How to Self-Host Mistral Large 3: Hardware, vLLM Setup & Function Calling (2026)

The comprehensive guide explores the deployment of the Mistral Large 3 model, a sparse Mixture-of-Experts (MoE) model with 675 billion parameters, focusing on practical considerations for production environments. It addresses crucial aspects often overlooked, such as hardware requirements across GPU tiers, precision formats like FP8 and NVFP4, context length settings, and function calling configurations, to optimize resource usage and performance. The guide emphasizes the importance of using the correct vLLM configuration to avoid issues with tokenization and function calls and highlights speculative decoding techniques to enhance throughput. Notably, Mistral Large 3 operates under an Apache 2.0 license, removing commercial restrictions and fees associated with self-deployment. It also provides insights into handling context length versus throughput trade-offs, secret management, and monitoring deployment through Prometheus metrics. The guide offers a detailed comparison of hardware setups, suggesting NVFP4 on A100s for efficient memory usage, while recommending FP8 for contexts longer than 64k tokens.