How We Achieved ~1000 tok/s and 16x Throughput with DSpark for Ideogram V4 Prompt Expander

At Team fal, significant improvements were achieved in the throughput of the Qwen 3.6 model for Ideogram V4's prompt expander, aiming to enhance user interactivity in text-to-image generation. By leveraging DSpark on SGLang, they managed to increase throughput by 16 times, primarily through the adoption of a 35B MoE model that balanced inference speed and image performance. This model was fine-tuned using PEFT for efficiency and stability. Despite initial challenges with speculative decoding and acceptance rates, the team incorporated DFlash, a diffusion-based model, and later integrated DSpark to further boost performance. The DSpark architecture involved using diffusion-based block predictors combined with Markovian heads to improve acceptance rates, which required modifications to both vLLM and SGLang for efficient model serving. Through extensive experimentation, they achieved ~1000 tokens per second with DSpark and ultimately settled on a configuration that provided 830 tokens per second, maintaining interactivity and efficiency while reducing the prompt expansion time to under two seconds.

No tracked trend matches for this post yet.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.