November 2020 Summaries

The importance of analytics in business growth and success has been increasingly recognized across industries. Data-driven decisions have proven to improve revenue, reduce operating expenses, and maximize profit. Leading companies that prioritize data quality and continuously enhance their data are growing at a faster rate than those who do not. Automation and artificial intelligence adoption rates also play a significant role in this growth. The McKinsey Global Institute reports that data-driven organizations are more likely to outperform competitors in customer acquisition and profitability. The trend of analytics adoption is expected to grow as businesses worldwide race to adopt modern data infrastructure and analytics. This demand for data analysts and professionals skilled in handling digital data, known as "Gen D," is on the rise. The value of data analytics increases during uncertain times, with companies that make a significant investment in analytics during crises achieving higher returns on assets and investments. Despite recognizing the importance of analytics, many organizations struggle to establish successful analytics programs due to challenges in data integration from multiple sources. Traditional data integration methodologies are not equipped to handle this complexity, leading businesses to seek modern, automated methods for handling digital data.

Many businesses struggle with successful analytics programs due to the challenges of accessing timely, trusted data from numerous and unpredictable sources. Traditional ETL processes are hindered by resource-intensive workflows and custom code designed for specific use cases. Automated ELT or automated data integration offers a modern approach that simplifies data access and makes it as reliable as electricity. This method shifts the "transform" step to the end of the data pipeline, allowing analysts to load data before transforming it and creating a single source of truth in a data warehouse. Automated ELT leverages prebuilt connectors for various data sources, reducing manual intervention and bespoke code requirements.

The co-founders of Data Culture, Gabi Steele and Leah Weiss, emphasize that a data-driven culture is essential for companies to succeed. They suggest investing in the domain expertise of data people, sharing business challenges with them, and brainstorming solutions together. To support a data-driven culture, they recommend resisting tensions between business analysts and data analysts, investing in ambassadors and building a data community, embracing inclusivity, and focusing on diversity within the team. By fostering transparency and meaningful connections across both sides of a company, a data-driven culture can drive better outcomes and help organizations reach new heights of success.

Fivetran builds data connectors by designing normalized schemas to structure data from SaaS applications. The process involves understanding the application's purpose and major workflows, drafting an ERD (entity-relationship diagram) based on the underlying data model, studying the API, creating draft tables using the API response, applying Fivetran naming conventions, linking and normalizing tables, reviewing isolated tables, publishing the ERD, and developing the connector. The ERD is crucial as it visually represents interrelations between tables and dictates the behavior of the connector.

Implementing a modern, cloud-based analytics stack can be achieved in three steps: establishing success criteria, estimating total cost of ownership (TCO), and choosing the right tools. Firstly, understand the benefits an MDS will offer such as time and monetary savings, expanded capabilities for data teams, reduced report generation time, improved reliability, increased BI tool adoption, and new actionable metrics. Secondly, estimate TCO by comparing current workflow with available MDS technologies and considering factors like subscription costs, transformation requirements, and labor costs. Finally, choose the right tools by doing thorough research, considering compatibility and future-proofing, and testing them in a low-stakes manner before making a decision.

Michael Kaminsky from Gradient Metrics conducted a benchmark comparison of data architectures, specifically focusing on the performance implications of different warehouse distribution patterns under normal BI-style workloads within Redshift, Snowflake, and BigQuery. The results showed that denormalized tables resulted in faster query response times compared to star schemas for all three warehouses. The speed improvement ranged from 25% to 50%, depending on the warehouse used. This analysis was conducted using a subset of TPC-DS benchmark data and aimed to understand how different data architecture patterns perform once a warehouse has been chosen, rather than comparing warehouses themselves.

Fivetran has won the ISV Partners Innovation Award from Databricks at Data & AI Summit Europe 2020. The collaboration between Fivetran and Databricks aims to empower data professionals by accelerating time to insight with Delta Lake and the Lakehouse architecture for the modern data stack. Together, they enable customers to focus on core business value, not data engineering for data pipelines, as well as accelerate time to insight through reliable, ready-to-query data in various forms such as normalized tables, historical tables, and aggregated tables. Fivetran also plans to support Databricks' new SQL offering, making big data analytics more accessible to data analysts and BI professionals.

Fivetran has introduced a 14-day free trial for every new connector added to an existing account, allowing users to explore potential data sources and consider new analytical use cases before making a purchase. This feature is part of the company's mission to make accessing data as easy and reliable as electricity. The new consumption-based pricing model offers flexibility and ease in scaling up or down depending on business needs. Users can trial multiple connectors simultaneously, with each connector qualifying for its own 14-day free trial period.

F5 Networks, a Seattle-based application services and networking company, has modernized its data stack to improve business insights and support data-driven decision-making. The company's recent migration to a cloud-based data stack reduced delays in executing requests from internal stakeholders and improved the "code to customer" metric from weeks to days. F5 Networks achieved this by embracing a modern approach to data strategy, which involved moving its on-premise servers to the cloud, adopting a cloud-native data warehouse for company-wide data sharing, updating legacy BI tools and improving data access, duplicating data with ELT instead of ETL, modernizing the transformation process, and implementing a strong data success and customer success program.

Drizly, an ecommerce platform for alcohol delivery, successfully managed a 392% increase in sales during the pandemic due to its modern data stack. The company's analytics team had proactively updated their patchwork data infrastructure to a cloud-native modern data stack in 2019, which enabled them to handle the surge in new customers and data. Key insights shared by Drizly at Modern Data Stack Conference 2020 include anticipating dramatic scale when building your foundation, finding best-of-breed modern data stack tools, and planning to scale and grow in data maturity. The adoption of a modern data stack allowed the team to focus on higher-level activities such as enhancing data models and onboarding new analysts quickly using best-of-breed tools across the data stack.

Fivetran co-founder and CEO George Fraser shares his vision for how data infrastructures will continue to advance as innovators break new ground. He highlights five key developments within the modern data stack (MDS): 1) Automating data integration with a powerful cloud data platform, separating storage and compute with elastic scalability; 2) The rise of cloud-based data warehouses that separate compute from storage, offering better user experience and enhanced performance compared to legacy solutions; 3) Innovations in data transformation tools will grow, with automated pipeline solutions like Fivetran taking care of the "E" and "L" in "ETL," leading to a new category of technologies within the in-data-warehouse transformation ecosystem; 4) Making data actionable by closing the loop between insights from queries and actions taken out in the world, with some exciting use cases including automating payroll and billing, monitoring intrusion detection, and detecting marketing regressions that could save a company millions; 5) The modern data stack will shrink latency time to support new use cases, with fundamentally possible latencies of seconds and tens of seconds.