Learning from real-world experience

The report describes a post-training approach that combines rejection sampling fine-tuning with hint-guided on-policy self-distillation to learn from real-world model sessions, including both successful interactions and failures identified through user corrections or tool errors. Rather than imitating every step in successful sessions or discarding unsuccessful ones, the method applies cross-entropy loss to useful actions in successful sessions and uses grounded corrective hints to create KL-divergence targets for avoidable mistakes in either successful or failed sessions, while retaining other content only as context. Hints must be based on information available before the error, aiming to avoid hindsight bias; examples include confusing a requested W-3 form with a W-2 and using an invalid tool parameter despite documented allowed values. Privacy filters exclude opted-out users and sessions containing personally identifiable information, while multiple judges and validation stages assess outcomes, identify root-cause turns, and verify whether corrections are justified. Evaluations found that hints substantially improved the base model’s ability to regenerate corrected actions, and offline tool-error rates declined from 2.79% for stock GLM 5.2 to 0.87% for an RFT-plus-OPSD checkpoint, though differing training data prevent a controlled attribution. In a later live A/B comparison between two such checkpoints, tool-call failures fell from 2.24% to 1.77%, a statistically significant 21.2% relative reduction, while user dissatisfaction did not change significantly, indicating stronger evidence for improved tool reliability than for broader task success or user satisfaction.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.