August 2026 Summaries

Engineers adapted the 744B-parameter GLM-5.2 mixture-of-experts model, whose routed experts use 4-bit NVFP4 quantization, to play Super Mario Bros. through bf16 LoRA reinforcement learning on two 8×B200 GPU nodes. Making the system trainable required resolving several hardware and software issues: Transformer Engine’s dual quantized weight layouts doubled expected memory use, so the team retained only the forward layout and used bf16 dequantization during backward passes; chunked kernels eliminated large quadratic-memory buffers in a reference sparse-attention implementation; and specialized SGLang settings enabled serving an NVFP4 MoE model with continuously synchronized LoRA adapters. They also fixed distributed adapter loading, optimizer initialization that overwrote warm-started LoRA weights with zeros, and sequence-length-driven memory failures. Although standard GRPO training initially improved reward before collapsing into deterministic, identical rollouts with zero learning signal, switching to DAPO-style dynamic sampling and removing an entropy regularization term sustained training performance. Tests on smaller models indicated little meaningful difference between bf16 and NVFP4 bases for supervised fine-tuning or reinforcement learning, while the resulting full-scale policy completed roughly 77% of Mario level 1-1, demonstrating an end-to-end approach for parameter-efficient RL on a 744B model.