Introducing Cube Evals

Cube Evals is a newly launched feature that addresses the challenges data teams face when AI agents answer business questions using a company's data model. As AI agents become integral to production systems, ensuring the accuracy of their responses is critical. Cube Evals allows teams to create evaluation cases—pairing natural-language questions with known-correct answers—and run these against the AI agent to obtain an objective accuracy score. This process helps identify discrepancies between the agent's output and the ground-truth data, allowing for targeted improvements. The evaluation cases are stored in the data model repository, ensuring that testing is integrated into the existing workflow and can be easily managed alongside code changes. This approach provides a deterministic grading system, ensuring consistent and reproducible results, and can be enhanced with optional model-based grading for more nuanced assessments. By automating what was previously a manual and error-prone process, Cube Evals streamlines validation in AI Studio, making it a native part of the development and deployment workflow for organizations using Cube.