Training on ComplexConstraints: Expert Rubrics as RL Rewards

ComplexConstraints is a comprehensive training set designed to enhance reinforcement learning (RL) models by using high-quality, expert-crafted rubrics to provide detailed, per-criterion feedback rather than simple pass/fail signals. This setup allows for the detection and correction of systematic grading weaknesses such as miscalibrated partial credit and exploitable phrasing. The study involved training the Qwen3-4B model with RLVR on a 1,000-example companion set, leading to significant improvements in task performance, as evidenced by a 15.5 percentage point increase in rubric pass rate on an in-distribution holdout, and notable gains on external benchmarks like AdvancedIF and MultiChallenge. The results highlight the efficacy of using densely annotated training data and expert rubrics, which ensure the model's ability to generalize beyond the original training set. This method emphasizes the importance of intent-aware, calibrated rubrics that are adversarially validated to prevent reward hacking, enabling models to better retain information across multiple interactions and maintain coherence in the face of user errors.

No tracked trend matches for this post yet.