July 2026 Summaries

In the evolving landscape of data-driven decision-making, evals (evaluations) serve a dual purpose: not only do they identify and rectify issues within data agents, but they also play a crucial role in building trust among users. The narrative underscores a shift from traditional dashboards, once the epitome of reliability, to AI-driven systems, which demand consistent proof of accuracy and reliability. Data teams are tasked with not only refining the accuracy of agents through regular evaluations but also effectively marketing these improvements to foster trust and encourage adoption across organizations. The challenge lies in demonstrating the tangible benefits of these AI systems, akin to the credibility once afforded to dashboards, by transparently sharing evaluation results and context improvements. This approach is essential for convincing business users to rely on data agents confidently, ultimately transforming data consumption from a mere metric of engagement to a genuine driver of informed decision-making.

In an exploration of optimizing AI cost management, Hex engineers Zach Kirby and Olivia Koshy discovered that a high cache hit rate, previously thought to be a reliable metric, was misleadingly concealing significant user expenses. Despite a 94% cache hit rate, high costs arose from sporadic uncached interactions that skewed user credit usage, leading the team to focus on improving caching strategies. They developed a cache visualization tool to analyze input token usage and costs, revealing that uncached responses could be disproportionately expensive. By restructuring cache prompts and implementing an automated review system, they aimed to mitigate cache misses. Additionally, adapting subagent models to reduce cache cooling and refining cache route keys to lessen mid-run cache misses were essential strategies. Through these initiatives, they learned that understanding and addressing outlier costs, rather than relying on averages, was crucial in managing AI expenses effectively.

In the journey toward achieving accurate AI analytics, data teams at companies like Ramp, Underdog, and Hover emphasize the importance of leveraging existing context to ensure trustworthy outcomes. These teams utilize Hex, a platform that helps them activate and manage context already embedded within their systems, such as dbt models, documentation, and stakeholder queries, to foster a continuous feedback loop for improvement. For instance, Ramp uses dbt as a single source of truth and syncs its domain documents with Hex for consistency, while Underdog focuses on activating existing context rather than creating new ones from scratch, utilizing semantic models and feedback loops to refine their analytics. Similarly, Hover prioritizes documenting key tables and columns to enhance the precision of the Hex agent, allowing users across various departments to engage with data more effectively. These strategies illustrate that successful data teams start with what they have, continuously refine their context based on active usage, and leverage platforms like Hex to streamline this process, making analytics more accessible and reliable across their organizations.

Analytics agents often produce incorrect results because they lack governed business context, such as trusted source tables, metric definitions, required filters, and institutional conventions, rather than because their underlying models are insufficient. The post argues that context engineering extends beyond prompt design by curating all information available to an agent, and cites benchmarks suggesting that concise, high-signal context can improve text-to-SQL accuracy more than changing models. It identifies four main forms of analytics context: curated warehouse metadata and endorsed tables, prior vetted analyses as memory, short workspace guides for business rules, and semantic models that codify consistent metric logic. It also warns that overly long, stale, conflicting, or poorly positioned context can degrade performance, recommending continuous evaluation through known-answer tests, reasoning-path reviews, session-length monitoring, and token-use tracking. At scale, prompt caching, targeted retrieval, and managed platform infrastructure can reduce cost and operational burden, while production readiness depends on progressively strengthening governance based on observed usage rather than waiting to build a complete semantic layer first.

In the evolving landscape of the AI era, data teams must adapt to new technologies that transform their roles and responsibilities, while still maintaining their core mission of making data accessible and useful. Although AI facilitates self-service data access and personalized distribution, it also introduces challenges like supervision and data governance, necessitating a focus on data quality, observability, and access management from the outset. Data teams are encouraged to remain adaptable, embracing cross-training and upskilling to navigate the increasingly blurred lines between roles, while still leveraging their unique combination of technical and business acumen to address complex data problems. As organizations strive for democratized data access, data teams must focus on ensuring data is used effectively and responsibly, fostering an environment where AI acts as a tool to enhance data usage rather than replace the human expertise that interprets and applies it strategically.

An AI gateway is a middleware layer between applications and large language model providers that centralizes routing, credential management, token- and dollar-based spending controls, logging, caching, content guardrails, and provider failover. Unlike traditional API gateways, it is designed for outbound model traffic and addresses LLM-specific concerns such as variable token costs, prompt privacy, prompt injection, and switching among models or providers. Gateways are generally most useful for organizations with multiple models, teams, production-critical AI workloads, compliance requirements, or significant AI spending, although they also create a centralized dependency that can affect all connected workflows if it fails. The piece emphasizes bring-your-own-key arrangements and enterprise controls that let organizations retain billing, logging, and data-processing oversight, citing Hex’s support for configurable model access as an example. However, gateways cannot determine whether model outputs are accurate or consistently apply business logic; trustworthy analytics also requires governed context, including trusted data tables, documented definitions, semantic models, workspace rules, and observability into user questions and agent quality.

Hex provides a playbook for building semantic models to create a governed and trustworthy metrics system, ensuring consistency in data interpretation across different stakeholders. The guide emphasizes the importance of a semantic model as a single source of truth, which aligns clean tables, metadata, and common business questions into one set of definitions. It highlights the necessity of using gold-standard, well-governed data from data warehouses as a foundation for building these models, underscoring that the model's reliability is contingent upon the quality of the underlying data. By curating specific business questions and pairing each with a verified SQL query, teams can establish a ground truth that prevents discrepancies in data interpretation. The process involves using Hex's semantic modeling agent to build and test the model against these verified queries, ensuring that every stakeholder receives the same accurate answer, thereby increasing trust in data-driven decisions. The playbook also advises maintaining the model over time, adapting it to new metrics and questions as the business evolves, and providing clear guidance on when to use the model for specific types of questions.

Kimi K2.7 is an open-weight analytics model now available in Hex, offering a cost-effective alternative to closed models like Opus 4.7 by using significantly fewer credits while maintaining comparable intelligence, particularly excelling in simpler, non-visual data tasks. Despite its slower performance due to its tendency to second-guess and validate its own processes, Kimi K2.7 proves more economically efficient, especially in straightforward semantic and analytically hard tasks, although it lags in visual data verification. As open-weight models have become more competitive in terms of both cost per token and accuracy, Kimi represents a strategic move by Hex towards integrating these models into their ecosystem. Hex aims to further refine Kimi's capabilities, addressing challenges like longer reasoning periods in certain scenarios, and expanding the versatility of open-weight models in analytics through continued experimentation and development.