March 2023 Summaries

The HiPPO, or Highest Paid Person's Opinion, is a phenomenon where senior leaders or executives rely on intuition or experience instead of data and analytics to make decisions. This can lead to a lack of buy-in from the wider organization, implementation of unvalidated strategies, and decisions that don't align with customer needs. However, in situations where data is limited or time is scarce, the HiPPO's intuition can serve as a valuable asset. To overcome the HiPPO, organizations must centralize their data sources, democratize access, and leverage modern data stacks to facilitate evidence-based decision making. This approach has been successful for companies like Square, which uses Fivetran's automated data movement solution to empower the C-Suite in overcoming the HiPPO and embracing data-driven decisions. By doing so, organizations can make faster, more informed decisions that drive better financial performance, resilience, and innovation.

Fivetran is an ELT (Extract, Load, Transform) platform that offers a cost-effective approach to data integration and analytics. To determine the cost-effectiveness of Fivetran, consider four key factors: building data pipelines, maintaining data pipelines, moving data, and transforming data. When it comes to building data pipelines, DIY options can be time-consuming and lack scalability, while using open-source connector development kits can speed up the process but also introduce inefficiencies. Maintaining data pipelines is critical, as pipeline breaks can have serious downstream impacts, and Fivetran's auto-schema change detection and response capabilities eliminate these issues. Moving data requires efficient capabilities such as data normalization, idempotence, incremental syncs, data selection, and programmatic pipeline management. Transforming data can be accelerated with features like version control, prebuilt data models, modularity, data lineage graphs, scheduling and managing transformations, and fully integrated scheduling. By considering these factors, organizations can choose the most cost-effective ELT solution for their business needs.

Automating data management is crucial for businesses to make informed decisions, as inaccurate and low-quality data can lead to costly mistakes. By establishing a seamless flow of high-quality data, organizations can drive innovation and save time, money, and talent. According to Fivetran's research, about 8 in 10 organizations rely on stale data, which can result in lost revenue. The company's solution helps turn raw data into usable insights, giving leaders confidence in decision-making. By outsourcing and automating mundane tasks, businesses can free up critical resources and focus on building trust through a single source of truth. With Fivetran's "plug and play" approach, businesses can quickly access pre-built data connectors and achieve operational efficiency without costly maintenance charges.

The modern data stack is evolving to help companies extract and transform data into analytics and insights, playing a critical role in driving business decisions and customer experiences. Experts from Andreessen Horowitz, Accenture, and Fivetran weigh in on the biggest challenges and opportunities facing the data world in 2023, emphasizing the importance of data-driven decision-making during economic uncertainty. Investment in technology and automation is essential to stay ahead, with companies needing to differentiate themselves through data-driven solutions. The modern data stack provides a suitable tooling infrastructure for companies to compete on what they extract from data, enabling differentiation and driving business success.

There is a new SQL execution engine called DuckDB that has been gaining attention for its simplicity and speed. In a benchmarking study, it was found to be faster than commercial systems like Postgres or SQLite for small data sizes on similar hardware. However, as the data size increases, other systems with more resources become faster. This suggests that DuckDB is not yet scalable to large datasets and multi-core processors. Despite this limitation, the study shows impressive performance, even on a 2021-era Macbook Pro with an M1 Pro CPU, which outperforms a server with 16 cores for certain data sizes. The findings highlight the potential of moving workloads from shared data warehouses to client machines, where the power of end-user machines can be leveraged to improve performance and reduce costs.

Fivetran and dbt are leading companies in modern analytics, with Fivetran providing efficient data integration and analysis, while dbt offers powerful data transformations and modeling capabilities. Companies such as Snowflake, SpotOn, and Red Ventures have seen significant benefits from adopting this duo, including reduced marketing waste and increased ROI, faster and more reliable reporting, and improved data democratization. By automating data ingestion and transformation, these companies can focus on driving value with advanced data science and machine learning models, attribution scoring, forecasting, and segmentation. The integration of Fivetran Transformations for dbt Core has enabled SpotOn to deliver analytics 5x faster while saving time and cost, while Red Ventures has seen up to 30 percent cost efficiencies in their marketing strategy.

Fivetran has launched AWS hosting in Japan, allowing businesses to securely integrate their data without worrying about data residency or compliance issues. This move further expands Fivetran's coverage in the Asia-Pacific region and provides customers with a low-latency data movement platform that meets strict security policies. With over 300 pre-built connectors, Fivetran's automated data movement platform enables customers to load data to cloud data warehouses on AWS in just a few clicks. The launch marks a significant milestone for Fivetran's Japanese customers who can now keep their data inside their network or solely in Japan due to strict security policies. Fivetran is committed to providing secure, reliable and cost-effective data movement solutions that comply with local regulations, allowing businesses to meet their data residency requirements wherever they operate.

The modern data stack (MDS) companies face a balancing act between data governance and business user self-service. The goal is to find the sweet spot where enough governance is enforced to mitigate data chaos while making progress toward empowering stakeholders with self-service capabilities. However, this delicate balance can be challenging due to technical challenges posed by data governance controls, such as the need for manual updates and reviews. To strike a balance, companies should categorize governance needs into foundational, intermediate, and advanced stages, allowing users to gradually gain more autonomy while maintaining data integrity. Ultimately, finding the right balance requires proactive conversations and collaborations between data teams and stakeholders, rather than relying solely on tools or technology.

Fivetran Inc. has partnered with data.world to create an effective data supply chain through data governance strategies that build trust and bring visibility to the data ecosystem. This enables teams to make critical business decisions by having the right data at the right time, reducing challenges such as low visibility into data, lack of trust in data and reports, and data and knowledge silos. By leveraging Fivetran's enterprise data catalog with data.world, organizations can unite their data ecosystem, increase data visibility, build a culture of trust and collaboration around data, break down data and knowledge silos, and remove bottlenecks with data governance. This results in a more efficient data supply chain that reduces risk and enables teams to make data-driven decisions quickly.

Data pipelines are critical in delivering reliable and trusted data, connecting various systems and handling aspects such as quality, security, and privacy. To achieve this, it's essential to ask the right questions and take the time to coordinate with your team before acting. Data intelligence from a data catalog like Alation provides contextual answers to table-stakes questions about data, enabling informed conversations among users about pipelines. This enables teams to pick the right data, identify potential issues, collaborate on pipeline development, and ensure everyone can access the data, ultimately leading to faster and more reliable data access.

Fivetran, a company, is promoting greater gender equity in the tech industry by breaking down biases and creating opportunities for women to succeed in technical roles. The EMEA team at Fivetran consists of remarkable women who share their career journeys and experiences in technical roles. They discuss the importance of critical thinking, empathy, clear communication, and creativity in solving problems. Imposter syndrome, self-doubt, and assumptions about abilities can be challenges faced by women in tech, but having a support network, advocating for oneself, and cultivating a growth mindset can help overcome these obstacles. The women emphasize the importance of research, seeking out mentorship, and being proactive in their career development to thrive in technical roles. They also stress the value of networking, forming connections with other men and women in tech, and inspiring others to pursue careers in technology.

The era of big data and cloud adoption has pushed data teams to innovate in constructing their data and analytics ecosystems. Data pipelines are supporting more operational, real-time, and event-driven analytics than ever before. However, with the increase in complexity and pressure on data teams, having clear insight into what's happening across the data estate is crucial. Active metadata can provide this insight by sending metadata back into every tool in the data stack, giving data teams context wherever and whenever they need it. It can be used for discoverability, root cause analysis, impact analysis, and cost optimization. By activating metadata, organizations can improve their daily workflows, build trust and visibility for the entire organization, and create an end-to-end view of the modern data stack.

Fivetran Activations has introduced an Open Beta for Git-backed Models, allowing data teams to manage SQL models with software development best practices through a bi-directional Git integration. This integration brings the advantages of Git, such as peer review, version control, and auditability, to Reverse ETL workflows, offering features like commit logs, approval flows, and automated CI testing checks. Users can specify SQL models in YAML configuration files and make changes either through the Fivetran Activations UI or by updating these files directly in a Git repository. The initiative aims to treat data resources with the same rigor as production-grade software, starting with SQL Models and planning to extend to Entities, Segments, and Syncs, thus enhancing governance and traceability of business logic changes. This feature is available to customers on the Fivetran Activations Platform Plan, with further developments anticipated to expand Git-based management to other resources.

Fivetran is an ELT provider that normalizes data within its own virtual private cloud (VPC), cutting compute costs for clients. This approach is unique among ELT tools, which typically normalize data in the client's warehouse or lake, increasing warehousing costs. By normalizing data within its own VPC, Fivetran reduces ingest compute costs substantially, with customers seeing savings of up to 10X compared to other providers like Stitch. Fivetran offers a free trial with 14 days of access to data connectors for 300+ sources, allowing customers to test the tool's efficiency before committing. The company also has multiple pricing tiers, including a free plan, to accommodate smaller organizations and limit initial financial commitment.

Businesses can enhance personalized customer experiences by utilizing a customer data platform (CDP) to collect and organize data on customer behavior and preferences, allowing for more targeted marketing strategies and optimized pricing. However, traditional CDPs often create new data silos, complicating data accessibility across an organization. Instead, leveraging existing data warehouses and tools like Fivetran Activations can help build a composable CDP that integrates seamlessly with existing systems and provides a unified, real-time view of customer data. This approach allows companies to achieve identity resolution, segmentation, omnichannel marketing, predictive modeling, and data governance more effectively, without the cost and complexity of traditional CDPs. Emphasizing warehouse-native data activation, this strategy ensures compliance with data protection regulations and maximizes the use of existing data infrastructure to enhance customer engagement and business growth.

Fivetran is an all-in-one data movement platform that has made significant progress in building a unified platform solution to help thousands of customers manage and move their most critical data. Starting March 1, 2023, users can access HVR capabilities rebranded as Fivetran Local Data Processing, enabling high-volume database replication while keeping data within secure on-premises or cloud environments. This technology will continue to be available to customers on the Business Critical Plan, offering flexible consumption-based pricing and fully-managed connectors and platform capabilities. Fivetran's single, integrated platform will provide automatic access to various sources and destinations, private networking and advanced security options, data controls with RBAC and SCIM detection, metadata integrations, free transformations for popular applications, and automated scaling and pipeline optimization. The platform also introduces self-hosted and fully managed deployment options, including Private Deployment, complex replication topologies, and AWS GovCloud, to support enterprise organizations in highly regulated industries.