Introducing a New Domain-Specific Language for Data Reliability

Soda SQL, an open-source framework, was launched to help data engineers maintain reliable data pipelines by profiling, testing, and monitoring data using SQL. Following its success and community growth, Soda has developed a new domain-specific language, SodaCL, aimed at enhancing data reliability through a human-readable format that empowers Data Engineers and Analysts to manage data quality autonomously. This new language aligns with the data mesh concept, promoting decentralized data ownership and accountability, which supports the idea that data quality is a collective responsibility across all business domains. SodaCL features over 30 built-in metrics to facilitate easy data checks and encourages a scalable, self-serve approach to maintaining data quality, reducing the dependency on engineers for every data issue. The initiative is part of a broader trend towards Everything-as-Code, aiming to unify data teams and enhance collaboration in managing data products throughout their lifecycle, with a preview program available to gather feedback before its general release.