November 2024 Summaries

Lyra Health, a leader in mental health benefits, uses advanced AI-driven matchmaking software to connect patients with therapists and coaches. Through a data-driven approach, Lyra has achieved remarkable results, including significant reductions in claims costs and healthcare expenses for children and teens. A key driver behind these results is Lyra's use of Fivetran Managed Data Lake Service, which combines the ease of use of a data warehouse with the flexibility and scalability of a data lake. This setup allows Lyra to maintain strict privacy and compliance standards while supporting data-driven insights and innovation in AI.

Fivetran has been exploring the potential of data integration to make generative AI practical since its emergence with ChatGPT. Retrieval-augmented generation (RAG), a core architecture supporting most current commercial implementations of generative AI, is a potent yet accessible way to turn proprietary data into useful, intelligent products. RAG requires two kinds of data integration: moving data from sources like applications and operational systems to data warehouses and data lakes, and turning data into embeddings that can be read by a vector database. Fivetran's internal chat tool, FivetranChat, is an example of how organizations can augment foundation models with proprietary data from text-rich systems such as CRMs and customer support applications for various use cases. The architecture involves extracting and loading data from every text-rich source closely involved with the product and operations, denormalizing it into text-rich tables using dbt transformations, and setting up a retrieval model with native AI tools like Snowflake Cortex and Databricks Mosaic AI to support generative AI. Overall, FivetranChat has been successful in providing accurate answers to internal questions about the product, operations, and company policies, reducing manual sifting through documents or asking for another person's time.

In today's fast-paced economy, companies are looking for ways to stay competitive while managing vast datasets, controlling costs, and streamlining operations with limited resources. Fivetran is a modern data integration platform that automates the process of moving data from multiple sources into a centralized data warehouse, enabling lean teams to manage complex data environments efficiently. Companies like Saks, Paylocity, and Dropbox have successfully used Fivetran to achieve significant productivity gains, cost savings, and faster insights, allowing them to focus on strategic initiatives and drive innovation.

GTMOps teams at leading companies are enhancing revenue outcomes by incorporating Large Language Models (AI) into their workflows and centralizing these transformations at the data warehouse and CRM-platforms level. This approach is facilitated by Fivetran Activations, which offers AI Columns to execute AI-driven prompts on customer data, syncing outputs to platforms like Salesforce and Hubspot. The blog outlines ten specific LLM prompts designed to aid RevOps and MOps professionals in gaining insights, personalizing engagements, and streamlining decision-making. These include intelligent lead scoring, churn risk prediction, hyper-personalized email content, industry classification, customer health analysis, campaign performance analysis, behavioral targeting, job title seniority classification, and personalized nurture emails. By centralizing these AI-driven prompts, the strategy ensures consistent insights across customer-facing applications, making AI-driven data transformation a single source of truth and enabling the delivery of hyper-personalized experiences across various engagement channels.

In a recent discussion with Fivetran's COO Taylor Brown and Chief Product Officer Anjan Kundavaram, they highlighted some of the pressing challenges in data management today, including persisting data silos and security concerns. They emphasized that unifying data access is crucial for gaining holistic insights. The rise of SaaS applications has led to a proliferation of data silos across various formats, which prevent enterprises from extracting meaningful insights. Fivetran's Hybrid Deployment model allows organizations to move and integrate data from both cloud-native and on-premises systems while meeting stringent security requirements. The company also anticipates a shift towards open data formats like Delta and Iceberg in the future, enabling companies to reduce costs and avoid vendor lock-in.

Fivetran Transformations has moved into general availability, offering automation and simplification of data transformations through its platform. The solution includes dbt Core and Quickstart Data Models for an end-to-end ELT solution that accelerates critical insights. Pricing is being introduced for Fivetran-hosted dbt Core transformations and Quickstart Data Models, with up to 5,000 model runs offered for free per month. New features are planned for January 1st, including user-defined jobs, compatibility for Hybrid Deployment connectors, support for cron scheduling in the Fivetran dashboard, and an account-level dashboard for jobs. Quickstart Data Models will continue to expand with more multi-connector data models coming soon. Customers can control their costs by adjusting run frequency, selecting specific output models, or refactoring models in their dbt Core project(s).

Since the release of ChatGPT in late 2022, enterprises have been exploring generative AI but struggle with implementation. To bridge this gap, organizations must first solve data integration and management challenges before optimizing interaction with foundation models like GPT-4. Retaining, augmenting, and generating (RAG) is a practical approach to enhance these models with accurate, context-rich data from various sources. Key challenges in implementing RAG include ensuring reliable data movement into accessible platforms and maximizing its capabilities for business needs. Automated data integration and effective prompt engineering, data curation, and knowledge graph usage are crucial strategies for successful RAG implementation.