Multi-token Residual Prediction

Multi-Token Residual Prediction (MRP) is a lightweight module for accelerating diffusion language models, which generate text by progressively unmasking tokens rather than predicting them left to right. Unlike naïve multi-token prediction approaches that attempt to reproduce an entire next-step probability distribution and degrade sharply over repeated steps, MRP predicts the smaller residual change between adjacent denoising-step logits, allowing it to operate accurately across multiple steps while keeping the underlying model frozen. The researchers apply MRP in static decoding as either a speculative drafter that preserves backbone output quality while reaching up to 1.56× throughput in SGLang, or a direct decoder that achieves higher speed with modest accuracy tradeoffs. In dynamic low-threshold decoding, MRP can reassess newly revealed tokens and remask those whose confidence declines after neighboring tokens are considered, recovering substantial accuracy lost through aggressive parallel unmasking, with reported gains of up to 22.6 points on HumanEval. Evaluated across SDAR models ranging from 1.7B to 8B parameters and benchmarks including GSM8K, MATH500, HumanEval, and MBPP, the approach uses a small two- or three-layer transformer and is presented as a flexible inference component that supports both quality-preserving acceleration and quality recovery.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.