May 2021 Summaries

In honor of Mental Health Awareness Month, Fivetran's Solution Architect Nora Gentry shares her journey with mental health issues while working in the tech industry. She discusses how she transitioned from a career as a chemical engineer to a role at Fivetran after seeking treatment for depression and anxiety. Nora emphasizes the importance of reaching out for support, being open about mental health struggles, and finding a good therapist. She also highlights the positive work environment at Fivetran, where employees are encouraged to share their experiences and seek help when needed.

The article discusses how to choose the best ETL (Extract, Transform, Load) tool for data integration needs. It explains that ETL tools are essential for moving data from various sources to a place where it can be analyzed. Key criteria for choosing an ETL tool include environment and architecture, automation, and reliability. The article also provides a brief history of ETL and how it has evolved into ELT (Extract, Load, Transform) due to the rise of cloud computing. It emphasizes that automation is crucial in minimizing human intervention during data replication. Finally, the article mentions other considerations such as security, compliance, support for various data sources and destinations, and pricing models when choosing an ETL tool.

Customer success has evolved from an art to a science, becoming a crucial discipline for high-achieving companies alongside marketing, sales, and product. In the B2B SaaS industry, customer adoption directly impacts revenue, making customer health vital for long-term growth. Companies with consumption-based pricing models must focus on reducing churn rates to foster expansion opportunities and create true advocates. To nurture a customer success mindset, businesses should prioritize outward focus, define their purpose, and develop well-rounded team members with high IQ, EQ, and grit.

Data integration architectures are crucial for businesses to manage and analyze large volumes of data. ETL (Extract, Transform, Load) is a key enabler in this process, allowing data to be extracted from various sources, transformed into a standardized format, and loaded into a central data warehouse. This enables applications like master data management (MDM), cross-application data consistency, and sharing. However, traditional ETL systems face challenges such as monolithic architectures, labor-intensive development, and handling newer data formats. The future of ETL is reimagined with ELT (Extract, Load, Transform) and reverse ETL, leveraging cloud technologies to improve scalability, flexibility, and efficiency in managing and utilizing data for business insights.

Fivetran's Director of Core Experience Alexa Maturana-Lowe has been honored with the Partner Champion Award at the 2021 Databricks Partner Executive Summit. This recognition follows Fivetran receiving the Databricks Partner Innovation Award last year for their work in supporting Lakehouse architecture and SQL analytics. The partnership between Fivetran and Databricks is seen as vital, allowing businesses to focus on growth and innovation instead of basic data engineering tasks. By using Fivetran and Databricks together, companies can automate data ingestion, accelerate time to insight, and easily orchestrate trusted datasets. The two companies will be partnering for a session at the Data + AI Summit 2021 to discuss how they leverage Lakehouse and Fivetran for marketing analytics.

Data integration performance can be significantly impacted by network bottlenecks, particularly when dealing with high-volume data sources like operational databases. Common causes of slowdowns include insufficient resources on database instances, security protocols that limit bandwidth, and inter-region or inter-cloud transfers. Buffer management and compression techniques also play a role in determining network performance. To improve data integration speed, consider scaling up infrastructure, re-architecting the process to reduce points of transition, optimizing algorithms, parallelizing execution, and properly leveraging compression and decompression.

When dealing with dimensional modeling challenges, particularly the one-to-many joins problem in SQL queries, it's crucial to avoid duplicating aggregated values, as illustrated by the example of calculating total costs and quantities from orders and order items. A naive SQL query can inadvertently count order totals multiple times due to one-to-many relationships, leading to inaccurate results, such as inflating the total cost for states like Alaska. Looker addresses this with a patented technique called symmetric aggregates, which involves complex SQL to ensure each order contributes to the sum only once. However, this method has drawbacks, including complexity, potential performance issues, and the use of slower distinct aggregates. An alternative approach is to apply multiple levels of aggregation through subqueries, transforming the one-to-many relationship into a one-to-one, which simplifies the SQL and avoids expensive distinct operations while maintaining accuracy. Although subqueries can be inefficient in older systems, modern data warehouses handle them efficiently, making this a viable strategy for manually written SQL queries, despite not being commonly used by BI tools.

Autodesk has significantly improved its data ingestion process by replacing homemade ETL with automated ELT, reducing the time from six months to just six days. This transformation was led by Jesse Pederson, VP of Data Platform and Insights, who focused on four guiding principles: buy over build, keeping it simple, minimizing time to impact, and ensuring security and privacy from the start. By implementing Fivetran and Snowflake for data ingestion and storage, Autodesk has simplified its process and allowed its teams to focus on more valuable tasks, moving from a cost center to a revenue center.

Retailers are increasingly relying on data to navigate shifting customer priorities, especially amidst the COVID-19 pandemic. Modern data architecture is crucial for retail success, providing insights that help businesses remain relevant and outpace competitors. The type of data needed varies by sector, but generally includes customer views, product inventory, pricing information, supply chain insights, and cost reduction strategies. A modern data stack (MDS) can blend cloud-based sources with on-premise solutions, offering elastic scalability and automated updates to support decision-makers with up-to-date information. Key components of a successful MDS include data connectors like Fivetran, the Snowflake Data Cloud for warehousing and data lakes, and visual analytics platforms like Tableau. To leverage these tools effectively, retailers must prioritize data access, transparency, and literacy to create seamless workflows and generate valuable insights.

The author shares their experience of using Fivetran for rapid access to rich, query-ready marketing data, which has been key to transformative insights at the company. They discuss how centralizing all relevant sources into a data warehouse helped bridge the gap between product, marketing and sales analytics, enabling them to answer various questions related to customer behavior and campaign efficiency. The author also provides a high-level overview of the steps required for readily accessing information across marketing, product and sales tools on the fly. They suggest options for modernizing one's marketing analytics setup, whether through Fivetran or other means.

The text offers an in-depth exploration of building and managing a modern data stack, emphasizing best practices applicable to both large corporations and startups. It outlines a framework for establishing a cloud-native data infrastructure, incorporating data warehouses like Snowflake and BigQuery, business intelligence tools such as Tableau, and data pipelines including Fivetran. The author also discusses effective hiring strategies for data teams, focusing on technical, execution, and analytical skills, as well as cultural fit. Emphasizing the importance of a centralized data team, the text advises aligning metrics with leadership and viewing data teams as a blend of support, engineering, and product functions. It encourages adopting a systems thinking approach to data modernization, highlighting the complexity of enterprise data management and the potential of strategic, macro-level optimizations. The discussion includes leveraging engineering principles, customer-centric service, and the transformative potential of data to drive substantial business value.

The integration of Fivetran with the spend management platform Coupa allows businesses to enhance their financial analytics by combining spend data with data from other tools such as CRMs and issue-tracking systems. This helps identify gaps in operations, streamline processes, and improve performance. By centralizing data from various sources, organizations can gain a complete picture of their spend funnel and make informed decisions for growth.

The modern data stack (MDS) is revolutionizing analytics by simplifying and accelerating data collection and analysis. It consists of automated, cloud-based tools that replace traditional on-premise systems. Adopting an MDS allows businesses to harness the value of their data more efficiently without increasing headcount, providing a competitive advantage. The Modern Data Stack Conference features speakers from leading companies discussing how they leverage the modern data stack for innovation and growth.

Survival analysis is a statistical technique used to determine the expected duration until a specific event occurs, with applications across various industries for understanding customer and product lifecycles, predicting medical care costs, and assessing machine reliability. By leveraging survival analysis in Python, businesses can evaluate key metrics like active user survival rates, product time to purchase, campaign effectiveness, employee churn, and machine lifecycle. The analysis employs mathematical concepts such as survival time, survival function, hazard function, and the Kaplan-Meier method, which estimates survival probabilities based on observed events. The Python library lifelines facilitates this process, enabling companies to gain deeper insights into customer behavior, operational efficiency, and marketing strategies, ultimately enhancing data-driven decision-making.

Fivetran now supports Databricks on Google Cloud, allowing businesses to use an open lakehouse platform like Databricks across an open cloud platform like Google Cloud. This partnership offers greater customer choice and flexibility in a growing cloud ecosystem with diverse data tools and innovative infrastructure. Key benefits include the ability to purchase Fivetran through the Google Cloud Marketplace, support for all Fivetran connectors on Databricks Delta Lake, unified billing of Google Cloud products, use of Google Cloud credits to procure Fivetran, and a simplified procurement process.

Fivetran has received ISO/IEC 27001:2013 certification, recognizing its commitment to the highest level of information security. The company's Information Security Management System (ISMS) was extensively audited by Coalfire ISO, Inc., and this globally recognized standard for ISMS establishment and operation is designed to cover key areas of Fivetran's enterprise information security program focused on providing secure products and services for customers, partners, and employees. The certification enables customers to use Fivetran regardless of the type of data they choose to connect with its services. Combined with Fivetran's SOC 2 Type II audits and PCI validation (available in Q2), Fivetran services can be used with a wide variety of regulated data workloads. The company is committed to adding platform features and internal capabilities to provide the most secure and reliable data pipeline, emphasizing simplicity compared to other ETL tools.