July 2022 Summaries

Datafold has introduced an integration with Hightouch, a data activation platform, to enhance the visibility of how changes to dbt models affect Hightouch models and syncs. This integration is crucial for ensuring that high-quality data is utilized effectively in operational systems like CRMs, SaaS applications, and ad tools, where it influences key functions such as customer communication, app experiences, and ad spending. By allowing analytics engineers to view the connectivity of every sync and model within a column-level lineage graph, directly within their pull requests, this integration helps reduce quality assurance time and minimizes accidental errors. It also accelerates code reviews by enabling reviewers to easily evaluate the impact of changes on Hightouch models and syncs. Datafold aims to provide comprehensive testing and visibility across data stacks and pipelines, making it an essential tool for users who rely on Hightouch for data-driven decision-making.

In the latest Analytics Engineering Roundup, dbt Labs CEO Tristan Handy emphasizes the importance of building systems that inherently prevent data quality issues rather than relying on detection and alerts after problems occur. Data quality is a multifaceted challenge involving technology, processes, and human error, with a significant portion of issues arising from changes made by people within an organization. Handy argues for a proactive approach to data quality, advocating for better change management processes that catch errors before they reach production. This includes implementing robust workflows with version control, staging environments, and automated checks to identify and correct potential issues early in the development process. The article highlights the importance of minimizing human error by understanding data dependencies and suggests that effective change management can lead to more robust data pipelines without sacrificing development speed.

An analytics engineer's first day at work highlights the importance of a structured onboarding process in data teams to maintain enthusiasm and avoid mistakes like breaking downstream dashboards. Kenny Ning, with experience in scaling data teams at prominent companies, emphasizes the need for efficient onboarding that includes merging code on the first day to familiarize new hires with systems and workflows. He advocates for investing in developer workflow tools like Spectacles and Datafold to automate testing and minimize risks associated with code changes. Moreover, using a company's staging environment or product version allows new hires to understand how product actions are logged in the database, enhancing their ability to navigate the company's data ecosystem. This technical onboarding approach ensures new hires quickly adapt, reducing the likelihood of errors while maintaining development velocity.