How to self-host Kimi K3.1 Pro (2.8T parameters) on your own Kubernetes cluster: memory per precision, GPU instances and multi-node serving
Blog post from Qovery
Kimi K3.1 Pro is an open-weight, 2.8-trillion-parameter mixture-of-experts model with 104 billion active parameters per token, a million-token context window, image input, and an official 1,561 GB MXFP4 checkpoint; its weights, rather than its active parameter count, determine the main memory requirement. The guide estimates that serving also requires capacity for runtime overhead, KV cache, and per-sequence attention state, and identifies a single eight-GPU B300 node as the simplest configuration, while B200, H200, or H100 systems require multiple nodes and a fast RDMA-capable interconnect. It describes vLLM and SGLang support, an experimental NVIDIA NIM option, and Kubernetes deployment patterns ranging from a single pod to multi-node groups managed with LeaderWorkerSet. The model’s license permits commercial use subject to revenue and user thresholds, with internal use exempt from those conditions. The guide concludes that API access may be less costly for many teams, while self-hosting can suit workloads that need data to remain in a private cloud, sustained GPU utilization, or greater control over serving; it also emphasizes checking current GPU availability, pricing, engine support, and license terms before deployment.
| Trend | Post Mentions | Total Month Mentions | Posts | Companies | MoM |
|---|---|---|---|---|---|
| Kubernetes | 26 | 3,388 | 294 | 94 | -34% |
| MCP | 10 | 7,404 | 711 | 183 | -38% |
| LLM | 5 | 2,788 | 618 | 179 | -58% |
| AI Agents | 2 | 4,689 | 923 | 218 | -33% |
| TPUs | 2 | 125 | 19 | 7 | -59% |
| AI Coding Assistant | 1 | 1,395 | 336 | 123 | -29% |
| AI Model Fine-tuning | 1 | 533 | 114 | 51 | -39% |
| Platform Engineering | 1 | 1,080 | 220 | 61 | -46% |
Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.