July 2020 Summaries

Data automation is essential for enterprises as it bridges the gap between data analysts and IT departments, addressing common pain points such as non-reliable data flows, stale data sets, and schema changes. Automated data integration allows businesses to blend data sources from various internal systems, web applications, services, and APIs, improving business intelligence by creating dashboards that show insights and statistics from multiple critical sources. This approach also supports multiple destinations natively, enabling companies to segregate their data by business units or have distinct areas for raw data manipulation and reporting. Automated solutions are responsive and fault-tolerant, ensuring efficient data flow and allowing IT teams to focus on core projects such as machine learning models, cataloging, and governance.

This article discusses the advantages of using ELT (Extract, Load and Transform) over ETL (Extract, Transform and Load) in data processing. Traditional ETL processes require predicting all possible uses of data beforehand, leading to inflexible workflows that need redesigning whenever reporting needs change. In contrast, the ELT approach allows analysts to load raw data into a warehouse and transform it using cloud-based technologies as needed. This not only speeds up data ingestion but also lowers the barrier to entry for comprehensive analytics. The article highlights Fivetran's automated data pipeline service that helps companies automate extracting and loading data into their data warehouses in a cloud-agnostic manner, enabling them to focus on insights rather than building and maintaining pipelines.

Fivetran has introduced a dbt Package for HubSpot that helps users build sales and marketing dashboards by providing greater visibility into customer engagement. The package ties engagement activities to companies, deals, and contacts, offers SCD Type 2 tables for ongoing changes, and joins email campaigns to specific metrics on email activities. It can be used as the foundation of a consolidated view of marketing campaigns, allowing users to understand how campaigns drive successful interactions and improve future campaigns accordingly. The HubSpot API presents challenges in data segmentation and volume, but Fivetran's normalized ERD and dbt package help address these issues by simplifying engagement tracking and joining engagements with other objects in HubSpot.

The article discusses the process of assessing whether to build or buy a data pipeline as a product manager. It highlights the author's experience in building custom ETL for their previous product, which consumed significant engineering time and delayed time-to-value for users. The author then presents a three-step process for making this decision: defining must-have requirements, comparing costs of build vs buy, and assessing trustworthiness of vendors. This assessment is crucial in the ever-expanding market of data solutions.

No-code tools are becoming increasingly popular as they allow people from various roles and departments to build and experiment without relying on engineering resources. However, this also means businesses need to integrate data from more sources to get a comprehensive view of their operations. Fivetran is a solution that helps teams handle data produced by no-code tools, allowing non-engineers to take advantage of these platforms without creating siloed data or worrying about data infrastructure. By using Fivetran connectors for various no-code tools such as Airtable, SendGrid, and Stripe, businesses can facilitate experimentation, empower non-engineers, and maintain visibility into their operations.

Fivetran is an industry leader in data integration from various sources to a data warehouse. Despite having over 150 prebuilt connectors for popular sources like Zendesk, NetSuite, and Salesforce, some organizations struggle with obscure data sources. Fivetran's Google cloud function connector can pull data from any source using less than 100 lines of Python code. The integration process consists of three fundamental pieces: data reader, core processor, and data writer. For a well-architected integration tool, the core processor and data writer are agnostic to the data reader, allowing users to write custom data readers for unsupported sources. This enables seamless integration with Fivetran's existing core processor and data writer, making it possible to pull data from any source without a native prebuilt connector.

The article discusses the benefits of migrating on-premise data warehouses to cloud computing platforms. These include cost savings from eliminating hardware expenses and maintenance costs, scalability to accommodate fluctuating usage patterns, device agnosticism for improved interoperability, flexibility in hybrid solutions that combine public and private clouds, enhanced resilience against server downtime, and superior security measures provided by cloud service providers. The author emphasizes the importance of considering these advantages when deciding whether or not to adopt cloud computing technologies.

Fivetran has introduced a new dbt Package for Asana, a popular task management application. The package allows analysts to track individual tasks, user activity, and join data with relevant projects and teams to manage resources effectively. It also helps create an overall view of a project and associated tasks and users, determine task backlogs, and enrich tags with associated task metrics for tracking progress or priority. Fivetran's Asana dbt package first ingests and normalizes the project management data using its connector and then uses the Fivetran-created schema to help track the history of tasks, create a backlog of tasks, and join task activity to parent and individual activities.

The agriculture industry is increasingly relying on data analytics to improve operations and meet global demand for food. Modern data stacks are becoming critical as businesses need to analyze data from various aspects of their operations, including weather, soil, fertilizer, seeds, equipment, finances, and stock. Big data analytics can boost productivity and innovation in agriculture while helping manage environmental challenges, reducing costs, and improving supply chain management. However, a challenge remains in accessing the necessary resources for data analysis in this sector. Fivetran is highlighted as an effective solution for data integration in agriculture businesses, offering ease of use and cost-effectiveness. The adoption of modern architecture and BI solutions can significantly impact every piece of the agriculture supply chain, with large data companies needing to consider agriculture a serious business.

Fivetran Inc. has introduced a new connector for integrating YouTube Analytics data into businesses' analytics platforms. This integration allows companies to track standard and advanced metrics, as well as segment users by various criteria. The data obtained can help marketing teams improve their video content strategy and refine campaigns based on insights gained from the analytics.

"Powered by Fivetran" (PBF) is a new offering for modern data insights platforms that provide analytics as a service. These firms build data products on top of disparate solutions such as Tableau, Snowflake and Redshift, and offer insights to decision-makers in diverse verticals. PBF is designed to be the data backbone of these use cases by enabling companies to build 150+ Fivetran data pipelines into their applications with minimal coding or engineering overhead. This helps businesses manage cash flow, track payments, optimize inventories and supply chains, and understand revenue sources. The solution works well for organizations of all sizes, from small and medium-sized businesses to enterprises.

A customer inquiry about adding sparklines to Salesforce accounts led to the discovery of a hidden feature in Salesforce that allows for the integration of live sparklines using formula fields with an Image function. By using Fivetran Activations to sync data points into Salesforce, and leveraging the QuickChart API to create graphs from a comma-separated list of data points, users can set up real-time growth visualizations on Salesforce Account records without needing a third-party AppExchange app. This process involves creating a new formula field that dynamically generates a URL with the data, which QuickChart then uses to render the sparkline image. This method allows for immediate updates to the visual graph as data changes, offering an innovative way for Fivetran Activations customers to visualize data directly within Salesforce.