In agentic RL, faster tokens are not enough

Long-horizon agentic reinforcement learning is often limited not by raw model token speed but by the time required to assemble a fresh, valid training batch from trajectories that repeatedly alternate between generation, tool execution, sandbox operations, and verification. Experiments using GLM-5.2 on 32 GPUs showed that completion-order scheduling, surplus candidate groups, cache-aware sticky routing, isolated sandbox execution, optimized KV-cache storage, and speculative decoding reduced fixed-weight Terminal-Bench batch collection for 256 trajectories from 591.8 to 196.1 seconds, a 66.9% improvement. The system also avoided global stalls during policy updates by rotating updates across serving engines and preserving eligible partial work under explicit freshness limits, producing a complete Terminal-Bench interval of 240.1 seconds and limiting the added cost of live weight updates on SWE-Bench to roughly 11%. The findings argue that agentic RL infrastructure should measure time to a trainer-consumable batch rather than inference throughput alone, while balancing concurrency, cache locality, sandbox contention, and controlled policy staleness.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.