August 2026 Summaries

Evals is a new tool introduced by Hex to enhance the performance and reliability of analytics agents by allowing data teams to test context changes safely before deployment. The tool uses a large language model (LLM) as a judge to evaluate the entire conversation and the tools used by agents, thereby assessing not only the correctness of answers but also the reasoning and behavior behind them. Evals are integrated into the existing workflow using the Hex CLI, enabling version control and side-by-side comparisons of different context versions. This feature is particularly beneficial for companies like Chime, which adopted Evals to streamline their agent evaluation process across multiple data domains. By allowing for configuration sweeps and scheduled evaluations, Evals helps teams ensure their analytics agents remain accurate and adaptive, even as data and contexts evolve over time. The platform supports a continuous improvement cycle, with a focus on maintaining and enhancing the context that agents rely on, thus fostering confidence in their analytical capabilities.