October 2021 Summaries

Integrating Snowflake with Salesforce offers significant advantages by transforming the data warehouse into a central hub for business operations, facilitating seamless data flow to frontline tools. The article discusses four methods for accomplishing this integration: Data Loader, Reverse ETL with Fivetran Activations, Data Import Wizard, and Snowflake Connector. Each method caters to different integration needs, with Data Loader allowing large-scale data transfer, Reverse ETL offering ease without custom code, Data Import Wizard providing a simple Salesforce-native option, and Snowflake Connector leveraging Tableau CRM Analytics Studio for syncing. These methods vary in complexity and suitability based on the frequency and nature of data integration required, enabling teams to choose the best fit for their specific workflows, thereby optimizing data accessibility and utilization for sales, marketing, and product teams.

Johnnie-O, a men's apparel brand, engaged Velir to build a customer data warehouse with advanced analytics and personalization capabilities after discovering cracks in its marketing technology stack during the pandemic. The challenge was to integrate several tools for ecommerce operations that were not integrated, leading to fragmented user metrics. Velir conducted workshops to uncover relevant data sources, destinations, and use cases, then provided a proposed data stack. They helped johnnie-O prioritize use cases by evaluating the level of integration effort against their business value. The initial setup included platforms like Snowflake, Data Build Tool (DBT), Fivetran, Full Circle (an ERP system), Shopify, Segment, Returnly, Klaviyo, and Back-in-Stock. Agile data activation was achieved through a series of two-week-long sprints using Scrum methodology. The result is an accurate calculation of customer lifetime value (LTV) and average order value (AOV), increased email open and click-through rates due to personalized emails, and a better understanding of customer purchases.

The article explains how to enhance Facebook Ads ROI by using machine learning training and Facebook's Conversion API in conjunction with reverse ETL tools like Fivetran Activations. It emphasizes the limitations of relying solely on the Facebook Ads conversion pixel due to obstacles like ad blockers and third-party cookie restrictions, which can lead to increased ad costs and inefficient targeting. The Conversion API, which allows advertisers to transmit conversion data directly from their systems of record to Facebook, helps create a more complete training dataset, enabling more effective machine learning-driven audience targeting. The article highlights the importance of configuring Facebook Ads pixel tracking to avoid duplicating conversion events and provides guidance on modeling data for the Conversions API to ensure accurate reporting, ads attribution, and deduplication. It also discusses how reverse ETL tools simplify the process of uploading conversion data into Facebook Ads, ensuring data integrity and improving ad targeting efficiency.

Fivetran's VP of Product discusses the company's future plans following its merger with HVR and participation in the Modern Data Stack Conference 2021. The focus will be on four key areas: connector reliability, data security, data transformation, and powering customer data products. Fivetran aims to improve reliability by increasing it from 99.75% to nearly 99.9%, while managing over 70,000 pipelines. The company will also continue to enhance its Fivetran Business Critical feature set for the highest throughput and security levels. Additionally, Fivetran is working on integrated data transformations and modeling with private preview availability of integrated scheduling for dbt transformations. Lastly, Powered by Fivetran (PBF) will be a critical part of the company's future offerings, allowing organizations to onboard customer data from external sources efficiently.

Fivetran has integrated seamlessly with multiple Google Cloud services, improving user experience. The partnership between the two companies allows users to discover and leverage Fivetran alongside BigQuery and within the wider Google Cloud ecosystem. With the recent acquisition of HVR by Fivetran, these integrations will only continue to scale, enabling real-time data replication technology for organizations. Joint customer GroupM highlights the value of the partnership, using Fivetran and Google Cloud to create a seamless data integration and visualization experience. The partnership has led to significant growth in joint customers globally, increased discoverability in BigQuery, and thought leadership in multi-cloud strategy.

The article provides a comprehensive guide on exporting CSV files from Databricks, detailing four distinct methods to achieve this task. It first outlines how to use Databricks Notebook, allowing users to download datasets directly or export them to DBFS for larger datasets, with options to customize CSV formatting and file size. The second method involves using the Databricks command-line interface (CLI) to transfer CSV files from DBFS to other locations, requiring Python and a personal access token for authentication. The third method utilizes JSpark, a Java-based tool, to execute SQL queries and save results as CSV files directly to a local machine. Lastly, the article suggests using external client tools like Visual Studio Code with a Databricks extension or standalone DBFS Explorer for easy file navigation and downloads. It concludes by recommending Fivetran Activations for a more streamlined data synchronization process if the outlined methods seem cumbersome.

The Modern Data Stack Conference 2021 brought together tech leaders to discuss the current state and future of data analytics. They highlighted that the modern data stack revolution was driven by cost changes, enabling companies to store and work with more data than ever before. However, they also pointed out that there is still a need for improvements in user experience and collaboration tools. The discussion also touched upon the debate between the "walled garden" of the data warehouse and the open-source Data Lakehouse, with participants expressing their preference for the latter due to its potential to provide more value from customers' data.

Successful analytics depends on choosing the right approach to storing enterprise data. Data warehouses, data marts, and data lakes are distinct technologies for storing business data and guiding business decisions. A data warehouse is a relational database designed for analytical queries, while a data mart is an organizational structure within a data warehouse containing summarized rather than raw data. A data lake is an object or file store that can accommodate large volumes of unstructured data such as free-form text, images, videos and other media, as well as structured data. Data warehouses are the most appropriate repository for structured business data used in analytics, while data lakes can store large quantities of media such as documents, images, videos, and audio. Some new technologies combine characteristics of both data warehouses and data lakes, known as data lakehouses.

The modern data stack (MDS) is a suite of tools used for data integration that includes a fully managed ELT data pipeline, a cloud-based columnar warehouse or data lake as a destination, a data transformation tool, and a business intelligence or data visualization platform. MDS differs from legacy data stacks in its cloud-hosted nature, ease of use, scalability, and reduced technical barriers to entry. The benefits of MDS include time, money, and effort savings due to low and declining costs of cloud computing and storage, off-the-shelf connectors, and the ability for analysts and data engineers to focus on higher-value analytics and data science projects. Key features to look for in each component of the modern data stack include prebuilt connectors, easy scaling, compatibility with destinations, version control, visualization flexibility, and user accessibility. Companies should consider upgrading to MDS when their current infrastructure is hindering growth or causing frequent downtime.

The article discusses the differences between ETL (Extract, Transform, Load) and ELT (Extract, Load, Transform), two major conceptual approaches to data integration. ETL is a traditional approach where data is transformed before being loaded into a destination, while ELT loads data first and transforms it later. The article highlights the challenges of ETL, such as constant maintenance, customization, and labor-intensiveness, which have become obsolete due to advancements in technology and cloud computing. ELT offers a more modern approach with decoupled extraction and loading from transformation, allowing for automated data integration workflows and easier management by analysts using SQL. The article concludes that while ETL may still be preferable in certain cases, ELT combined with automation can greatly simplify an organization's data integration process.

Fivetran has acquired HVR, with CEOs George Fraser and Anthony Brooks-Williams discussing the merger process and its implications for customers. The two companies have complementary capabilities, with Fivetran excelling in replicating SaaS data and HVR specializing in replicating mission-critical DBMS data. The combined company aims to offer a single place for all data movement needs and challenges, enabling new analytical use cases by consolidating more data sources.