Introducing Rubrics: Build Agents that Evaluate and Correct Their Work

RubricMiddleware enhances Deep Agents by enabling them to self-evaluate and iterate tasks based on predefined criteria, ensuring that complex tasks are completed accurately. This system introduces a grader sub-agent that reviews the agent's output against a rubric, providing per-criterion feedback and allowing for iterative improvements until all criteria are met or a set iteration limit is reached. This approach mitigates the issues of ambiguous instructions and non-deterministic errors, enhancing output quality and reducing the need for manual intervention by developers. By shifting the responsibility of identifying and correcting errors from developers to the system, RubricMiddleware ensures more reliable task completion, particularly in scenarios where correctness is critical.