How to get started with Qwen3.8-Flash-Next on Runpod Serverless

Qwen3.8-Flash-Next is Alibaba’s preview serving stack for Qwen4, a 125B-parameter ultra-sparse mixture-of-experts model that activates 6B parameters per token, supports a native 262K-token context expandable to 1M with YaRN, and includes multi-token prediction, vision, Gated DeltaNet linear attention, and Qwen Sparse Attention for efficient long-context inference. Testing on Runpod with four NVIDIA H200 GPUs found that the standard vLLM worker template is currently incompatible because it uses vLLM 0.28, while this model requires version 0.29 or later and the dedicated vllm/vllm-openai:qwen38-flash-next image. Both Serverless load-balancing endpoints and GPU pods can run the model using TP4 with expert parallelism and Triton MoE support, with Serverless favoring bursty API traffic and pods better suited to debugging, interactive work, and higher sustained throughput. The deployment requires substantial storage and memory for its roughly 173 GiB FP8 checkpoint, and a critical NCCL_NVLS_ENABLE=0 environment variable prevents H200 NVLS initialization failures that otherwise cause all ranks to crash before weights load. Measured Serverless startup took about 11 minutes on a warm host, while compiled pods achieved much faster warm decoding performance, whereas eager Serverless mode reduced boot-time compilation delays but incurred slower early requests due to Triton kernel JIT compilation. The model was validated through OpenAI-compatible API calls with streaming, hidden reasoning traces, optional non-thinking mode, XML-based tool calling, and built-in vision support, demonstrating that the deployment can provide a 262K-context API after the required image, flags, hardware configuration, and cold-start considerations are addressed.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.