Systems Engineering Playbook: Optimizing Qwen 3.5-397B MoE on Ironwood (TPU7x)

Deploying and serving the Mixture-of-Experts (MoE) model Qwen3.5-397B on specialized hardware like the Ironwood TPU v7x poses significant engineering challenges due to its massive weight footprint and complex architecture. The model's novel components, such as Gated DeltaNet (GDN) linear attention and Attention Data Parallelism, require a modular, model-agnostic optimization strategy to manage these complexities efficiently. By decomposing the model into independent building blocks and using pre-optimized modules, engineers can achieve significant performance improvements in inference workloads. Between April and June 2026, these optimizations resulted in a 3.1x performance increase for decode-heavy and a 4.7x increase for prefill-heavy workloads. Furthermore, by integrating these optimizations into open-source frameworks like vLLM and SGLang, legacy software barriers are minimized, supporting seamless migration paths for enterprise workloads. Through systematic hardware-aware sharding and custom kernel developments using the JAX/Pallas language, the team achieved high efficiency, extracting up to 82.4% and 79.6% of the TPU’s theoretical limits for compute-bound and memory-bound workloads, respectively. This approach not only optimizes the Qwen 3.5 model but also establishes a reusable software stack, enhancing TPU capabilities for future MoE architectures.

No tracked trend matches for this post yet.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.