Trinity Large

Trinity Large is a 400 billion parameter sparse MoE model developed through an ambitious pretraining project that combines cutting-edge techniques in data curation and efficient attention to achieve fast inference and high performance across various benchmarks. Three variants are being released: Trinity-Large-Preview, lightly post-trained for chat-readiness; Trinity-Large-Base, representing the best pretraining checkpoint after a complete 17 trillion token run; and TrueBase, an early checkpoint without instruct data or learning rate anneals. The model uses 256 experts with a high sparsity ratio, making it highly efficient compared to its peers. Despite budget constraints, the team managed to complete pretraining in just 33 days using 2048 Nvidia B300 GPUs, with a total project cost of $20 million. Trinity Large sets a new standard for frontier-class foundation models, demonstrating superior performance in math, coding, scientific reasoning, and multilingual capabilities. The model is currently available on OpenRouter, and integrations are in place with platforms like Kilo Code, Cline, and OpenCode, offering opportunities for users to test and provide feedback, which is crucial for its ongoing development.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.