Evaluation metrics are essential for systematically measuring and improving the quality of large language model (LLM) outputs, as they convert subjective AI quality into quantifiable numbers. These metrics are necessary because AI-generated content is often non-deterministic and subjective, making traditional software testing methods ineffective. Key categories of evaluation metrics include task-agnostic metrics, which apply broadly and assess aspects like factuality, coherence, and safety, and task-specific metrics, which evaluate criteria unique to particular applications. Code-based metrics offer fast, deterministic evaluations, while LLM-based metrics handle subjective criteria, enabling the evaluation of complex quality dimensions. Braintrust provides a comprehensive infrastructure for implementing, tracking, and acting on these metrics, offering 25+ pre-built scorers and support for custom code-based and LLM-based scorers. The platform facilitates continuous monitoring, regression detection, and A/B testing by integrating with CI/CD platforms and providing tools for online scoring in production environments. Best practices for using evaluation metrics involve starting with simple metrics, combining multiple metrics to capture all quality dimensions, and tracking metrics over time to identify gradual quality shifts.