June 2026 Summaries

The text explores the distinct roles of data owners and data stewards within data governance initiatives, emphasizing the importance of clearly delineating responsibilities to avoid failures commonly predicted by analysts like Gartner. A data owner, typically a senior business leader, is accountable for a data domain, establishing policies, defining quality standards, and accepting risks, while a data steward is responsible for daily execution, such as enforcing access rules and monitoring data quality. The confusion between these roles often leads to stalled governance programs, and the text underscores the necessity of properly defining accountability and responsibility, as outlined in the RACI matrix, to ensure effective data management. Additionally, the text highlights the integration of these roles within a broader governance operating model, which includes other positions like the Chief Data Officer (CDO) and data custodians, and discusses the importance of suitable tooling, such as Soda, to facilitate collaboration and operationalize data governance efficiently.

AI for data quality is transforming data management from manual, rule-based tasks to automated, predictive processes that maintain datasets' accuracy and reliability, by utilizing AI to define, monitor, and resolve data quality issues at a scale that traditional methods cannot achieve. AI-driven data quality operates in two modes: assistive, where human direction guides AI tasks, and agentic, where AI autonomously executes tasks within set boundaries and returns changes for approval. This shift is driven by the increasing volume of data being generated and consumed, necessitating automation to keep pace, especially as AI agents become new consumers of data, unable to discern errors as humans do. The foundation of AI-driven data quality lies in data contracts, which are machine-readable specifications that define a dataset's expected quality, enabling AI to work effectively and autonomously. Tools like Soda AI facilitate the creation and management of these data contracts, allowing teams to automate data quality checks and scale operations without losing human oversight. Ultimately, AI enhances data quality by automating repetitive tasks, enabling teams to focus on higher-level analysis and judgment.

A data observability dashboard is a comprehensive monitoring tool designed to provide real-time insights into the health of data pipelines by tracking key metrics such as freshness, volume, distribution, schema, and lineage. This dashboard helps data teams detect, triage, and resolve issues before they affect downstream consumers by identifying anomalies and pinpointing where issues occur within the data infrastructure. Building such a dashboard involves a structured approach, starting with defining a monitoring scope, selecting key metrics, connecting data sources, configuring alerts, and establishing an incident response workflow. Unlike BI dashboards that focus on business performance metrics, data observability dashboards concentrate on the operational health of data systems, allowing teams to transition from reactive monitoring to proactive testing and data contract enforcement. The ultimate goal is to build a sustainable observability practice that evolves continuously to maintain data reliability and prevent recurring failures.

Data testing is a critical practice in ensuring data quality by verifying that data meets predefined standards before reaching end-users or systems, minimizing downstream issues. It encompasses a variety of checks, including data quality, structural, functional, and ETL/migration testing, each addressing specific aspects of data integrity. The practice is underscored by the data testing pyramid, which prioritizes efficient, cost-effective checks at the base, such as data quality checks, and more resource-intensive checks like ETL and migration tests at the top. Tools like dbt, Python-based frameworks, and Soda Collaborative Data Contracts provide essential support for implementing rigorous data testing practices, facilitating automation, and integrating testing into CI/CD pipelines. Effective data testing hinges on starting with critical datasets, employing reusable configurations, and fostering collaboration between technical and business teams to maintain data reliability. Measuring success involves tracking metrics such as test pass rate, incident rate, and mean time to detection, ensuring the continuous improvement of data quality practices.

The text explores the nuanced distinction between data governance and data management, emphasizing that while they are interconnected, they serve different roles within an organization. Data governance is defined as the strategic framework that outlines policies, roles, and decision rights, essentially setting the standards for what constitutes "good data" and who is responsible for it. In contrast, data management involves the operational execution of these standards, focusing on the lifecycle processes of ingesting, storing, transforming, and maintaining data. The article highlights the importance of distinguishing between these disciplines to prevent directionless management practices and ineffective governance programs. It introduces the concept of data contracts as a bridge between governance and management, enabling governance policies to translate into actionable management tasks while allowing feedback from management to inform governance. The text argues that organizations should focus on establishing a clear boundary between strategy and execution, rather than conflating the two, to ensure efficient and reliable data handling, especially in light of increasing regulatory demands and AI readiness.

Data governance is a crucial organizational system that ensures data is accurate, consistent, secure, and responsibly used across teams, addressing business challenges like conflicting data, lack of accountability, and inefficiencies. Unlike mere compliance or IT projects, it is a business capability involving policies, standards, roles, and processes that define data accountability, handling rules, and enforcement methods. Key components include data policies, standards, ownership, catalogs, quality management, access control, and lineage, all of which are enforced through continuous, automated checks. Effective data governance addresses regulatory compliance, enhances data quality and trust, boosts operational efficiency, and prepares organizations for AI and analytics by establishing a reliable data environment. Starting with a focused domain and securing executive sponsorship is recommended for initiating a governance program, emphasizing the importance of operationalizing governance through tooling rather than relying solely on documentation. Real-world examples demonstrate its application in healthcare, financial services, and e-commerce, where governance resolves inconsistencies and improves data reliability, proving its value as a business enabler rather than a restrictive measure.

In a transformative shift, 2K Games evolved from having no automated data quality checks to implementing nearly 2,000 checks across approximately 1,000 datasets within a year, achieving a 95% data quality service level agreement (SLA) with rapid alerts. This evolution was driven by an organizational shift rather than a new tool, emphasizing shared responsibility for data quality across six teams within the organization. Key strategies included assigning named owners to each dataset, establishing data contracts between data producers and consumers, and developing per-studio scorecards to measure and improve data trustworthiness. This proactive approach, which integrates data quality into the architecture and uses self-service checks, shifts the focus from detection to prevention, catching issues before they affect business operations. By embedding data quality responsibilities into the daily workflow and leveraging tools like Soda for automation, 2K Games has significantly improved its data handling processes, turning data quality into a visible and owned cultural component with quantifiable metrics for trust.

Modern data systems have evolved beyond simple pipelines, often becoming complex and distributed across various platforms and workflows. This complexity introduces challenges, particularly when upstream changes disrupt downstream systems. Schema registries and data contracts emerge as solutions to address these reliability issues, though they serve different purposes. A schema registry manages and enforces schema evolution in streaming systems, ensuring structural compatibility between data producers and consumers. In contrast, a data contract establishes a broader set of expectations, including data quality, freshness, ownership, and usability, often involving co-authorship between technical and business stakeholders. While schema registries focus on the write path by validating message structure at publish time, data contracts protect the read path by enforcing operational correctness against landed datasets. Mature data teams often use both tools to maintain reliability across their data infrastructure, with schema registries ensuring structural integrity and data contracts providing governance over quality and operational expectations. This layered approach is crucial as data systems become increasingly distributed and complex, necessitating a combination of structural and operational checks for maintaining data reliability and usability.