Launching a trillion-parameter model at Sakana AI
Blog post from Modal
Modal and Sakana AI collaborated to deploy Sakana Namazu, an approximately one-trillion-parameter mixture-of-experts model designed for Japanese business contexts, web search, and code execution, using dynamically scalable GPU infrastructure rather than reserved capacity. To reduce memory demands while preserving Japanese-language performance, they applied NVFP4 quantization selectively to routed experts while retaining BF16 precision for attention, routing, and shared experts, calibrating the model with representative Japanese-style conversations. They also developed custom speculative decoding draft models trained on more than 30 billion raw production-style token IDs, later refining the approach with data from roughly one million real requests; Modal reports that this increased token acceptance, reduced decoding latency, and improved p99 latency. Additional serving optimizations addressed CPU–GPU synchronization, expert-matrix kernels, cached setup work, multi-GPU acceptance consistency, and grammar-constrained structured outputs. After launch, a KV-cache-aware router reportedly reduced queueing and time-to-first-token spikes by directing requests toward replicas likely to contain cached prompt prefixes, while elastic GPU scaling helped accommodate uncertain launch demand.
| Trend | Post Mentions | Total Month Mentions | Posts | Companies | MoM |
|---|---|---|---|---|---|
| Serverless | 1 | 1,385 | 326 | 109 | +13% |
Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.