Governing data products at scale: Promoting collaboration between producers and consumers

The recent session on governing data products at scale emphasized the importance of collaboration between data product producers and consumers to effectively manage and scale data products. Laurent Dresse from DataGalaxy, a data catalog solution provider, highlighted that successful data governance hinges on understanding data products beyond technology, focusing instead on solving issues related to people, communication, and collaboration. This involves defining characteristics such as lifecycle status, quality expectations, security measures, and privacy dimensions and ensuring this information is accessible through a self-service platform. The session underscored the necessity of effective communication, where producers and consumers work together to document and define use cases, ultimately facilitating a comprehensive understanding of data flow from extraction to consumption. DataGalaxy's approach integrates a collaborative "WidiWig" strategy, where producers and consumers jointly identify data sources, build data product pipelines, and ensure real-time updates in their data catalog. This dynamic integration with platforms like Starburst allows for continuous evolution and governance of data products, making them visible and understandable across the organization and enhancing their scalability and adoption.