Trinity Large Tech Report: 400B Open MoE Architecture Explained

Arcee AI’s Trinity technical report presents Trinity Large, an open-weight Mixture-of-Experts language model with 400 billion total parameters but only 13 billion active per token, alongside the smaller Trinity Nano and Mini models, all released through Hugging Face. Designed for enterprise deployment, auditing, adaptation, and efficient long-context agentic workloads, Trinity Large uses 256 routed experts, a shared expert, sparse activation, and a mix of local and global attention layers to reduce inference costs. It was trained on 17 trillion tokens using a large synthetic dataset curated by DatologyAI and 2,048 NVIDIA B300 GPUs, with stability improvements including SMEBU for router load balancing and RSDB for more consistent batch composition. The model was extended to 256,000-token context windows and showed strong retrieval performance beyond its training length, while benchmark results placed its base and instruct variants competitively against comparable models. In FP8 testing on eight H200 GPUs, Trinity Large reportedly exceeded DeepSeek-V3 and GLM-4.7 in throughput, time to first token, and end-to-end latency, though Arcee characterizes the current Preview instruct release as lightly post-trained and plans further refinement.

No tracked trend matches for this post yet.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.