Content Deep Dive

How to self-host Kimi K3.1 Pro (2.8T parameters) on your own Kubernetes cluster: memory per precision, GPU instances and multi-node serving

Blog post from Qovery

Post Details
Company
Date Published
Author
-
Word Count
4,305
Company Posts That Month
101
Language
English
Hacker News Points
-
Post removed?
No
Summary

Kimi K3.1 Pro is an open-weight, 2.8-trillion-parameter mixture-of-experts model with 104 billion active parameters per token, a million-token context window, image input, and an official 1,561 GB MXFP4 checkpoint; its weights, rather than its active parameter count, determine the main memory requirement. The guide estimates that serving also requires capacity for runtime overhead, KV cache, and per-sequence attention state, and identifies a single eight-GPU B300 node as the simplest configuration, while B200, H200, or H100 systems require multiple nodes and a fast RDMA-capable interconnect. It describes vLLM and SGLang support, an experimental NVIDIA NIM option, and Kubernetes deployment patterns ranging from a single pod to multi-node groups managed with LeaderWorkerSet. The model’s license permits commercial use subject to revenue and user thresholds, with internal use exempt from those conditions. The guide concludes that API access may be less costly for many teams, while self-hosting can suit workloads that need data to remain in a private cloud, sustained GPU utilization, or greater control over serving; it also emphasizes checking current GPU availability, pricing, engine support, and license terms before deployment.

Trends Found in this Post
Trend Post Mentions Total Month Mentions Posts Companies MoM
Kubernetes 26 3,388 294 94 -34%
MCP 10 7,404 711 183 -38%
LLM 5 2,788 618 179 -58%
AI Agents 2 4,689 923 218 -33%
TPUs 2 125 19 7 -59%
AI Coding Assistant 1 1,395 336 123 -29%
AI Model Fine-tuning 1 533 114 51 -39%
Platform Engineering 1 1,080 220 61 -46%
Use This Data

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.