Toward RL on mixed-precision, mixed-accelerator samplers

Researchers demonstrate that large-model reinforcement learning can use heterogeneous, mixed-precision sampling fleets—including B200/B300 NVFP4, H100/H200 FP8 or BF16, and A100/RTX PRO 6000 BF16 accelerators—while training a BF16 model asynchronously on B200 GPUs. In agentic coding training of Qwen3.6-35B-A3B on repository repair tasks, uncorrected GRPO training collapsed even with homogeneous samplers because sampler–trainer mismatch grew over time, causing rising KL divergence, declining rewards, and repeated tool-use failures. The authors stabilize learning through masked importance sampling to reweight off-policy tokens, score centering to remove gradient drift, and top-p mask replay to ensure the sampler and trainer share the same candidate-token support. This combination achieved SWE-bench Pro V2 performance comparable to homogeneous BF16 B200 sampling despite an average sampler–trainer mismatch 4.7 times larger, while reducing sampler cost per update by about 15%. The findings argue that stability depends less on minimizing numerical mismatch than on keeping it bounded through off-policy correction, potentially allowing RL systems to independently optimize training and inference hardware, precision, kernels, scaling, and serving strategies.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.