February 2025 Summaries

The new BigQuery Loader for Snowplow BDP Cloud offers enhanced performance, cost efficiency, and flexibility by enabling BigQuery as a destination across both AWS and GCP platforms. This update introduces the use of BigQuery Storage Write API, reducing infrastructure costs compared to the previous Streaming Inserts API, and employs a smarter schema evolution approach that minimizes column additions and avoids BigQuery's column limit. Users will benefit from a seamless migration process, with existing Snowplow BDP customers automatically upgraded and new users able to easily deploy the loader themselves. Additionally, the loader supports handling failed events by storing them separately in BigQuery for improved visibility and debugging. Snowplow encourages users to explore the benefits of this scalable and cost-effective solution, providing extensive documentation and support for those transitioning or starting fresh with BigQuery destinations.

Shift-left data governance is an approach that emphasizes performing data governance at the point of data collection rather than after the data has been collected and stored, and Snowplow offers tools to facilitate this process. This method addresses the persistent issue of poor data quality, which can consume a significant portion of IT budgets, and provides a more sustainable way to manage data, especially for organizations developing sophisticated data applications. By implementing shift-left governance, companies can ensure high-quality, reliable data from the outset, which is essential for AI-driven applications and real-time decision-making. Snowplow's schema-first design, early validation, and real-time quality monitoring help prevent bad data from entering storage systems, thereby reducing downstream cleanup costs and enabling faster deployment of data applications. This approach also fosters trust in data by making governance systematic and transparent, allowing data teams to focus on innovation rather than remediation.

Defining event and context schemas in Iglu involves balancing schema flexibility with data integrity, particularly regarding the additionalProperties attribute in JSON Schema, which determines whether undeclared properties are allowed in data processing. When set to true, this attribute offers flexibility by allowing new properties to be added without requiring schema updates, thus reducing overhead in rapidly changing environments. However, it also means that these extra properties, while present in raw JSON, are not available in downstream data models, potentially complicating data governance and quality management. In Snowplow data processing, such properties are retained but not loaded into data warehouses like Redshift, limiting their accessibility for analysis unless custom processing is applied. Best practices suggest using this flexibility selectively, employing structured metadata to maintain schema integrity, and establishing a versioning strategy to manage new properties. Ultimately, while this approach can facilitate rapid data collection, it requires careful management to avoid compromising data quality, especially in production-grade pipelines.

Snowplow is enhancing its customer data infrastructure (CDI) with the introduction of Snowplow Extensions, designed to expand the functionality of existing data structures with pre-built integrations. These Extensions aim to streamline and enrich data operations while maintaining stringent governance and quality control, allowing data teams to integrate operational tools seamlessly. The first two Extensions, Snowplow Reverse ETL and Snowplow Audience Hub, enable businesses to sync enriched behavioral data to over 200 business tools and facilitate the creation of audience segments through a no-code interface, respectively. This development supports Snowplow's objective to provide a composable, flexible data infrastructure that contrasts with traditional Customer Data Platforms (CDPs) by maintaining data integrity from the start. By integrating these capabilities directly into Snowplow, businesses can avoid the complexity and cost of separate CDP investments and ensure a consistent flow of high-quality data across their operations.

Following the release of the Javascript v4 tracker, Snowplow has expanded its tracking capabilities across multiple platforms, enhancing performance and developer experience while strengthening tracking capabilities. The updates include a new Snowplow tag for Google Tag Manager, which optimizes performance through asynchronous cookie access, a streamlined plugin architecture, and improved template and settings configuration. Additionally, the React Native tracker has been completely overhauled in TypeScript for full Expo support and web platform compatibility, aligning it with the JS v4 tracker core for consistent functionality across platforms. A new WebView plugin simplifies tracking in hybrid apps by enabling seamless event forwarding and compatibility with Android, iOS, and React Native trackers, integrating with the Unified Digital dbt data model. These updates aim to provide a more straightforward implementation process and improved tracking performance for diverse use cases, from simple websites to complex hybrid applications.