April 2025 Summaries

The modern data lake, also known as a data lakehouse, has gained popularity due to its ability to decouple storage and compute resources, making it cost-effective and scalable. Open table formats like Apache Iceberg, Delta Lake, and Apache Hudi provide ACID transactions, efficient CRUD operations, schema evolution, partitioning, indexing, versioning, and metadata management. This separation of architecture enables universal storage, allowing end users to access data using their preferred tools without vendor lock-in. Decoupling storage from compute also enables easier distribution of data, reduced costs on ingestion, and improved governance through technical catalogs that track data locations and control access. The use of open table formats brings significant benefits, including the ability to read and write data without vendor-specific services, manage object stores however desired, and distribute data as needed. Additionally, modern data lakes are interoperable with other systems, supporting a wide range of tools for accessing and querying data. Fivetran's Managed Data Lake Service abstracts away complexity, providing near real-time data ingestion, cost-effectiveness, and full management capabilities.

Fivetran's Managed Data Lake Service makes using Google's Cloud Storage easy by automatically extracting data, converting it into open table formats, and normalizing, compacting, and deduplicating data as it lands in Cloud Storage. The service solves challenges related to data integration, management, and governance, allowing data teams to focus on higher-value projects. With Fivetran Managed Data Lake Service, organizations can use Google's Cloud Storage as a universal storage layer that supports any data use case, including batched data integration for reporting and decision support, real-time feeds for fraud detection and business process automation, and open table formats for governance, security, and query-ready relational structure. The service empowers data professionals to pursue higher-value analytics projects by assembling architectures from interoperable off-the-shelf tools and technologies. By leveraging Fivetran Managed Data Lake Service, organizations can build a sophisticated architecture that powers just-in-time inventory management and stock replenishment, starting from basic reporting and progressing toward system automation.

The GigaOm report compares the cost of moving data into a data lake using the Fivetran Managed Data Lake Service with the cost of moving data into a traditional data warehouse. The primary finding is that a data lake architecture results in substantial cost savings of 77% to 95%. This is due to modern data lakes offering interoperability, governance capabilities, and reduced vendor lock-in compared to traditional data warehouses. The report also highlights the benefits of adopting a modern data lake, including automated governance, schema migration, table maintenance, and query-ready data. Fivetran's Managed Data Lake Service eases adoption by providing a fully managed service that automates data integration, open table format conversion, and data lake management, while offering unparalleled convenience and peace of mind.

Fivetran has empowered three leading enterprises — Saks, Canva, and Condé Nast — to transform their businesses by centralizing data and fueling insights. By unifying data silos into a single source of truth, these companies are enhancing customer engagement, optimizing operations, and driving increased loyalty. With Fivetran's automated data pipelines, they can break down silos and make data-driven decisions to create exceptional customer experiences that meet and exceed customer expectations.

Managing a support team can be challenging due to the high volume and varied nature of incoming tickets, but Large Language Models (LLMs) such as OpenAI's GPT, Anthropic's Claude, or Google's Gemini offer an effective solution by automating ticket categorization with high accuracy. This automation streamlines the support process, leading to faster resolutions, improved resource planning, and valuable insights into customer issues that can inform product development and indicate potential churn risks. Implementing LLM-based categorization involves defining appropriate categories based on one's business needs, crafting precise prompts for the LLM, and testing and refining the system before scaling it across the support operation. The approach also allows for advanced applications like sentiment analysis, auto-response suggestions, and root cause identification, ultimately enhancing the efficiency of support teams without replacing human agents. By leveraging these AI tools, support teams can focus on solving complex issues and building stronger customer relationships, turning support data into actionable insights.

Fivetran has announced support for Google's Cloud Storage (GCS) as a data lake storage option, enabling organizations to leverage GCS and reduce overhead. This integration allows users to move data from 700+ sources directly into GCS in their preferred open table format, with seamless metadata management through catalog integrations like BigQuery Metastore. By automating data ingestion, format conversion, normalization, and deduplication, Fivetran reduces operational overhead, empowering organizations to focus on innovation and AI initiatives. With this integration, businesses can centralize diverse data sources into their GCS data lake, simplifying data movement while converting to query- and AI-ready formats, ensuring compliance with industry standards like GDPR. Fivetran users can now try the Managed Data Lake Service for free from April 9th to May 31st, 2025, with connectors set up for a new Google's Cloud Storage destination eligible for this promotion.

The industry is witnessing a shift towards open table formats, which are transforming traditional data lakes into data lakehouses by providing ACID compliance, schema evolution, and data versioning capabilities. Open table formats such as Delta Lake and Apache Iceberg offer robust transactional support and performance enhancements, but differ in key areas, with Delta Lake being optimized for Databricks and Spark, while Apache Iceberg has a broader query service ecosystem support through dedicated query platforms like Trino, Presto, and Flink. Despite growing adoption, the industry remains uncertain about which format will emerge as a leader, posing risks such as interoperability concerns, future-proofing data architectures, and operational complexity for teams that invest heavily in a single format. However, solutions like the Fivetran Managed Data Lake Service offer a way to enjoy the benefits of open table formats without adding unnecessary complexity, by supporting multiple query engines, providing optionality, and simplifying data integration.

Generative AI is transforming businesses by providing valuable insights and automating tasks, but it requires high-quality, real-time data to function effectively. Centralizing data using tools like BigQuery and Vertex AI enables secure access to proprietary data without compromising privacy concerns. To create AI-ready data, organizations need capabilities such as Extract, Load, and Transform (ELT) and solid data governance tools that protect data security and quality. Automating ELT processes with the help of Google Cloud and Fivetran accelerates access to high-quality data for various use cases beyond just AI, including reporting, predictive modeling, and customer analytics. By leveraging fully automated data integration, organizations can streamline their operations, focus on strategic AI development, and achieve enterprise intelligence readiness.

Fivetran Activations's AI Columns utilize AI to streamline data categorization, making complex datasets more navigable and trend-spotting more efficient. The process involves defining a set of categories, which can be manually provided or AI-suggested, and crafting prompts that determine how the AI should apply these categories to the data. This approach leverages large language models (LLMs) like OpenAI, Claude, and Gemini to facilitate the categorization of data such as user feedback or firmographic information at scale. The guide emphasizes prompt engineering's role in ensuring accurate categorization and suggests methods for evaluating and refining results. Users can perform quality checks by creating a new dataset to review AI Column outputs and can use SQL queries to analyze and validate categorized data. Advanced options allow AI to help define categories when none exist, offering flexibility in various data management tasks.

Fivetran's Managed Data Lake Service is designed to address the challenges of scaling data stacks, including vendor lock-in, compliance issues, and rising costs. By adopting a universal storage layer, organizations can reduce total cost of ownership, optimize data lake scalability, and automate governance through open table formats like Delta Lake and Apache Iceberg, integrated with Databricks' Unity Catalog. This approach enables interoperability, reduces vendor lock-in, and provides access to best-of-breed tooling, while delivering benefits such as reduced ingest costs, automated schema management, and enhanced data governance.

Moin Haque, Head of Enterprise Data, Analytics & AI at IFF, emphasizes the importance of synthesizing information to make informed decisions. He notes that data is a key ingredient in fueling growth and innovation, but its value lies not just in its quantity, but also in how it's combined with processes, people, and business objectives. Haque stresses the need for intentional data collection, processing, and integration to unlock meaningful insights. He also highlights the balance between innovation and governance, noting that AI is a powerful tool that requires careful consideration of its impact on core operational fundamentals. Moin Haque identifies three essential skills for effective leadership in fostering resilience and growth: empathy, adaptability (entropy), and efficacy, which prioritize people-first approaches, flexibility, and meaningful outcomes over perfection.

Fivetran has recognized its top partners with the 2025 Fivetran Global Partner Awards, acknowledging their expertise and commitment to helping customers harness the power of data and analytics. The winners include technology partners like Databricks, Microsoft Azure, Snowflake, Google Cloud, and Sigma, as well as global systems integrator (GSI) partners like Accenture, Capgemini, Tata Consultancy Services, phData, Hakkoda, Slalom, Devoteam, Infinite Lambda, Civica, Qrious, iZeno, and K.K. Ashisuto. These partnerships enable businesses to unlock new opportunities, gain a competitive edge through data-driven decision-making, and drive innovation in the rapidly evolving digital landscape. The awards recognize the dedication and expertise of Fivetran's partners in delivering best-in-class data solutions that simplify data integration and analytics capabilities for organizations worldwide.

The text highlights the growing demand for Artificial Intelligence (AI) and the need for high-quality data to fuel its success. Traditional storage solutions, such as traditional data warehouses, often fail to meet the scale and flexibility required for advanced AI projects. Data lakes are emerging as an ideal solution, offering a cost-effective, scalable, and flexible approach to data management that can support the growing volume, complexity, and variety of data. Modern data lakes combine the benefits of both structured and unstructured data storage, enabling organizations to handle large amounts of diverse data, reduce costs without compromising performance, and secure their data and brand reputation. The text also shares a success story from Tinuiti, a leading digital marketing agency, which transformed its AI-driven marketing operations with Fivetran's Managed Data Lake Service, accelerating client onboarding and eliminating manual data maintenance work, allowing engineers to focus on high-value initiatives like AI and predictive modeling.