January 2026 Summaries

The Open Semantic Interchange (OSI) specification aims to standardize the semantic context for AI-enabled analytics, ensuring consistent interpretation of metrics, dimensions, datasets, and relationships across different tools and platforms. At Coalesce 2025, dbt Labs introduced MetricFlow as an open-source project and announced collaboration with partners like Snowflake and Salesforce in developing OSI, which is available under an open-source Apache 2.0 license. This specification addresses the common problem of inconsistent metric definitions by providing a vendor-neutral interchange format that allows semantic definitions to be transferred without re-authoring. dbt Labs enhances this by operationalizing semantics through its Semantic Layer, enabling consistent metric execution with MetricFlow and supporting interoperability across analytics tools. The ongoing development of OSI involves collaboration with the OSI community, with opportunities for engagement and contribution available through various platforms.

The dbt Summit, previously known as Coalesce, is now accepting proposals for its annual event, which gathers the largest community of dbt users to discuss the future of data and AI. The event is seeking authentic and diverse perspectives from data practitioners and leaders who can share real-world stories about solving data problems, improving data quality, accelerating workflows, and optimizing costs. Participants can choose between traditional breakout sessions or interactive peer exchange sessions to present their experiences, focusing on specific problems, solutions, and lessons learned. Potential topics include analytics development best practices, data modernization, enterprise scaling of dbt, AI integration, self-service analytics, and cost optimization. First-time speakers are encouraged to participate, and dbt Summit track leaders will assist in preparing impactful presentations. Speakers receive a complimentary conference pass and resources to promote their work within the community. The call for papers is open until March 31, and proposals can be submitted online.

In the context of artificial intelligence, the importance of high-quality, centrally defined metrics is underscored as essential for successful AI implementation, as outlined in a piece by Kathryn Chubb. The text emphasizes the longstanding principle of "Garbage In, Garbage Out," highlighting that AI outputs are only as reliable as the data inputs, with 86% of business leaders recognizing the necessity of high-quality data for AI success. The complexity of modern data infrastructure, often resulting in "metrics chaos" due to inconsistent and outdated metric definitions across multiple tools, poses significant challenges to AI systems that lack the contextual understanding that humans naturally provide. The solution proposed involves creating a centralized, governed semantic layer that defines metrics in clear, machine-readable terms, ensuring consistency and trustworthiness. This layer, supported by tools like dbt, allows for version-controlled, tested, and documented metrics that can be seamlessly integrated across traditional BI tools and AI applications, ultimately driving operational efficiency and business value by providing a stable source of truth amid evolving data ecosystems.

In a discussion on The Analytics Engineering Podcast, Tristan Handy and Russell Spitzer delve into the intricacies of Apache Iceberg and the evolving landscape of open table formats and catalog layers. Spitzer shares his journey from working with Apache Cassandra at DataStax to joining Apple’s Apache Iceberg team, where significant efforts were made to replace legacy systems with Iceberg, facilitating easier migrations and reducing reliance on bespoke solutions like those used in Hive/HDFS. The conversation covers the governance model of Apache projects, emphasizing the community-driven process and the role of Project Management Committees (PMCs) in maintaining project integrity. They also explore the development of Iceberg through its different versions, highlighting enhancements made for transactional analytics, row-level operations, and support for streaming and AI applications. Furthermore, the discussion touches on Polaris, an Apache incubator project, designed to provide an interoperable lakehouse catalog with pluggable identity providers, aiming to simplify the catalog layer by supporting multiple table/file formats and ensuring robust identity integration.

The Language Server Protocol (LSP) is an open standard that facilitates communication between code editors and language tooling, allowing for consistent language features across different development environments. Developed by Microsoft for Visual Studio Code in 2016, LSP addresses the fragmented support and performance issues faced by developers by standardizing the communication between code editors and language servers, which handle tasks like code parsing and syntax analysis. This separation allows editors to remain lightweight and responsive while enabling developers to integrate various tools seamlessly. The protocol utilizes JSON-RPC messages and supports real-time document management and feature interactions. dbt leverages LSP through its Fusion engine to enhance SQL development in the dbt VS Code extension, providing features like live error detection, instant refactoring, and rich lineage context, thereby improving the efficiency and quality of data pipeline development. LSP's widespread adoption has led to improved tooling capabilities, enabling developers to write better code with fewer errors and deeper project insights.

dbt Labs has developed the dbt Fusion engine to address the inefficiencies in data pipelines caused by rebuilding models unnecessarily, leading to wasted compute costs and time for data engineers. The Fusion engine introduces state-aware orchestration, which only rebuilds models when necessary based on changes in the underlying data, resulting in potential cost savings of up to 64%. This innovative approach transitions dbt from a stateless tool to one that utilizes real-time model state, reducing the complexity and cost of data operations by maintaining a real-time cache of the environment and making informed decisions about when to rebuild models. Advanced configurations allow users to align model builds with business SLAs, maximizing reuse and further cutting costs. Additionally, Fusion optimizes testing by rerunning only necessary tests, further enhancing computational efficiency. By implementing these strategies, dbt Labs achieved significant improvements, including a 63% reduction in average job runtime and a 64% annual reduction in data platform costs. The company aims to continue enhancing cost-efficiency features and is working on integrating cost data visualization directly into the dbt platform.

Cribl's integration of dbt and Omni provides a robust framework for delivering trusted AI-driven analytics by closing the context gap that often undermines AI projects. dbt acts as a data transformation layer, centralizing governed analytics with version control, defined metrics, and data lineage, ensuring that AI systems operate from a single source of truth. Omni complements this by offering a business intelligence platform that enables flexible data exploration and consistent metrics, enhancing AI's ability to deliver accurate and reliable outputs. Cribl's transition from Looker to Omni was motivated by the latter's AI readiness, allowing Cribl to leverage enriched data models for AI training while maintaining governance through automated documentation processes. By integrating generative AI for automated metadata generation, Cribl streamlines workflows and reduces manual documentation efforts, supporting scalable AI adoption. The seamless integration of Omni with dbt facilitates synchronized development workflows, allowing teams to validate changes before impacting production, thus enhancing the reliability and trustworthiness of AI outputs. This approach not only optimizes the development lifecycle but also builds trust at the point of data consumption, ultimately leading to better AI adoption and streamlined processes through automated orchestration.

Sweetgreen, a popular restaurant chain, faced challenges in accessing consistent data insights due to multiple sources of truth, bespoke data flows, and manual processes that often led to inconsistencies and delays in decision-making. To address these issues, Sweetgreen undertook a comprehensive data transformation using dbt to standardize business logic and create a single source of truth with consistent metric definitions. This transformation enabled the use of conversational AI, allowing business teams to self-serve data insights by asking questions in plain English and receiving reliable answers. By leveraging dbt's Semantic Layer, Sweetgreen established governed, reusable data models, enhancing the accuracy and reliability of its analytics. This shift allowed data teams to focus on enabling rather than gatekeeping, as stakeholders could now explore and analyze data independently with tools like Claude, an AI tool for conversational analytics. The implementation resulted in faster insights, increased trust in data, and a significant cultural shift towards self-service analytics, reducing the data team's role as a bottleneck. Sweetgreen plans to continue migrating remaining dashboards to ensure consistent reporting across all platforms.

ADE-bench, developed by Benn Stancil and dbt Labs, is a new benchmark designed to evaluate the performance of AI agents on analytics and data engineering tasks, addressing the lack of specific benchmarks in the data community. While tools like SWE-bench assess software engineering, ADE-bench uses real-world dbt projects and databases to measure how AI models tackle the complex, messy problems faced by data practitioners. Initial results reveal varied performance across models and configurations, with dbt Fusion and the Model Context Protocol (MCP) significantly improving accuracy and efficiency. The benchmark consists of dbt projects, databases, and real-world tasks and creates a sandbox environment for agents to solve presented problems. Performance is assessed by test pass rates, costs, and runtimes, with dbt Fusion showing notable gains in pass rates. ADE-bench is open-source, encouraging community contributions to enhance its relevance and effectiveness in measuring AI capabilities in data work, with ongoing efforts to improve the dbt language framework and MCP tooling.

In a recent episode of "The View on Data," hosts Jerrie Kenney, Erica “Ric” Louie, and Faith McKenna discuss the significance of community and mentorship in the data field, emphasizing that community is more than just networking; it's about finding a supportive environment where people understand your work challenges without lengthy explanations. The hosts share personal anecdotes about forming relationships through local meetups, online platforms like Slack, and industry conferences, highlighting that community is often found through a mixture of luck and persistence. They redefine mentorship as a toolkit of skill-specific guidance and feedback loops rather than relying on a single advisor. The discussion also touches on maintaining community involvement without burnout, underscoring that kindness and small, consistent interactions can naturally keep connections alive. Ultimately, the episode encourages listeners to see community as a space for shared interests and problem-solving beyond mere career advancement, offering support and validation in navigating the complexities of working in data.

Lauren Anderson, leading the enterprise data platform at Okta, discusses the pivotal role of identity in navigating the evolving landscape of AI agents and open data lakes in an episode of The Analytics Engineering Podcast. She emphasizes the importance of central governance and a shared semantic layer for managing these shifts, arguing for a clear division of labor between analytics and data engineers as AI agents increasingly handle analytical queries. Lauren reflects on her career journey from banking to B2B data, highlighting the complexities of handling enterprise data at Okta, where she oversees data ingestion, transformation, and delivery processes. She explains how Okta's security-first approach is streamlined through repeatable frameworks, allowing for faster risk assessments and governance processes. The conversation also explores the integration of AI agents into data platforms, advocating for centralized, governed access to ensure security and efficiency, while predicting that agents will soon generate more analytical queries than humans.

Stora Enso, a leading renewable materials company, addressed the inefficiencies of its centralized data team by adopting dbt to enable decentralized data operations, significantly reducing data delivery times from months to days and empowering autonomous analytics across its five divisions. The shift involved halving the central data team and embedding engineers within divisions, transforming the central team's role from service delivery to platform provision, thus allowing divisions to prioritize their own analytics projects. This transition was facilitated by dbt's use of SQL, a universally understood language among analysts, and its built-in data lineage and testing frameworks, which maintained quality standards across teams. The success of this decentralized model has positioned Stora Enso to explore further innovations, including compliance with EU regulations, AI integration, and platform portability, thereby demonstrating how the right technological foundation can drive significant organizational change.

A data observability platform is essential for managing modern data stacks, which have increased complexity due to numerous data sources, transformation tools, and endpoints, creating potential failure points and reducing end-to-end visibility. These platforms provide comprehensive insights into data systems, combining monitoring, testing, and performance optimization to maintain data quality and operational reliability. Key components include metadata collection, anomaly detection, and testing frameworks that validate data quality and business logic. Their integration with tools like dbt enhances data transformation processes by monitoring data accuracy and creating test cases to prevent issues from reaching production. Effective alerting and performance monitoring help teams optimize infrastructure and reduce costs while maintaining data reliability. Successful implementation requires organizational commitment, leveraging existing infrastructure and skills to foster a culture of self-service and proactive issue resolution. As data systems evolve, observability practices must adapt, using AI and machine learning to enhance detection and alerting capabilities, enabling organizations to make data-driven decisions confidently.

Data observability offers significant advantages by enhancing system reliability, fostering trust in data, and enabling proactive issue resolution, as demonstrated by SurveyMonkey's experience. By integrating reactive monitoring with Monte Carlo and proactive data management using dbt, SurveyMonkey addressed data quality and processing challenges, converting anomalies into test cases for improved robustness. This approach not only reduced data anomalies but also led to notable cost savings, such as a 73% reduction in Snowflake credit usage, through performance monitoring and optimization. Data observability facilitates improved collaboration by democratizing access to data insights, reducing the burden on data engineering teams, and enabling business users to independently investigate and resolve issues. Furthermore, it supports scalable governance by automating audit trails and quality checks, ensuring compliance and maintaining high standards across data assets. Ultimately, data observability enhances decision-making capabilities, fosters organizational resilience, and positions businesses for competitive advantage, with potential future advancements in AI and machine learning for anomaly detection and alerting.

Data pipeline observability has become crucial as organizations transition to cloud-native data architectures, which use ELT pipelines and cloud data warehouses like Snowflake and BigQuery for efficient data processing. This evolution brings complexities that traditional monitoring struggles to handle, as modern pipelines involve interconnected components such as ingestion systems, transformation layers, and orchestration systems, creating potential failure points. Observability encompasses performance monitoring, data quality monitoring, lineage tracking, and sophisticated error detection, which are essential for identifying issues promptly and ensuring data reliability. Inadequate observability can lead to "data downtime," eroding trust and affecting decision-making, while effective observability supports rapid response to issues, performance optimizations, and resource management. Tools like dbt enhance observability through artifacts that offer deep insights into pipeline performance, and when integrated with broader data quality initiatives, they help create resilient systems. Ultimately, observability not only ensures the reliability and trustworthiness of data systems but also enables organizations to innovate and maintain competitive advantages in data-driven decision-making.

Modern data governance is a critical framework for ensuring the accuracy, security, and usability of data in analytics and AI initiatives, extending beyond mere compliance to encompass structural pillars, technical capabilities, and collaborative processes. It requires clearly defined roles such as executive sponsors, data stewards, and data owners, who work together to embed governance into organizational workflows. Modern strategies favor dynamic, continuous, and automated governance over traditional static methods, emphasizing distributed responsibility for high-quality dataset creation and collaboration across teams. Technical components like data cataloging, lineage tracking, quality monitoring, and access control are vital to maintaining a secure and efficient governance infrastructure. The rise of AI introduces unique challenges, such as bias and transparency issues, necessitating evolved governance strategies that focus on data products to verify and maintain data quality. Sustainable governance programs integrate naturally with existing data platforms and workflows, leveraging tools like dbt to automate traceability, ensure consistent data definitions, and support collaborative and efficient development practices, transforming data into a strategic asset that enhances decision-making and supports AI initiatives.