Axiom has introduced a system for offline evaluations aimed at improving the quality and reliability of AI capabilities before deployment. The platform facilitates systematic testing by allowing teams to run AI capabilities against collections of test cases with known expected outputs, utilizing a flexible scoring system that can be customized to measure specific criteria. Built on Axiom's data platform, these evaluations are documented as distributed traces, enabling teams to query and visualize results alongside other telemetry data. This approach replaces the traditional, less structured method of development, where changes were made based on intuition rather than evidence, by providing tools to compare different models, prompts, and configurations using flag-based experimentation. Axiom's system is designed to integrate into continuous integration and deployment (CI/CD) pipelines, ensuring that quality is assessed and maintained throughout the development cycle. The platform aims to empower teams to make informed decisions by systematically measuring AI outputs, thus reducing the risk of regressions and improving overall product quality.