August 2019 Summaries

The article discusses Pipeline Automation Deficiency Syndrome (PADS), a condition affecting organizations that struggle to integrate large volumes of data. Symptoms include data file incontinence, impaction, slowed movement, and cognitive deficits. TC, a startup with high data consumption and rapid growth, suffered from this syndrome due to its inability to manage multiple data sources effectively. The treatment for PADS is the modern data stack, which centralizes data through highly-optimized connectors and cloud warehouses, reversing deterioration associated with the condition. Risk factors for PADS include heavy data consumption, integrating data from various sources, rapid company growth, understaffed BI or data science teams, and attempts to build AI or predictive analytics products.

The enterprise data warehouse (EDW) has been criticized due to problems associated with extract-transform-load (ETL) processes, such as latency, inability to handle real-time use cases, and difficulty adapting to changing business needs. However, these issues are not inherent to the EDW concept but rather stem from ETL. Alternatives like Query Federation, Data Virtualization, and Hybrid Transactional/Analytical Databases (HTAP) have been proposed, but each has its limitations. A successful data warehouse requires proper separation of concerns, with replication of all business data into the EDW followed by transformation into a user-friendly schema. This approach eliminates ETL problems like latency and data gravity issues while supporting non-relational data sources and allowing for flexible schema evolution.

Salesforce is a cloud-based CRM tool that helps businesses manage customer relationships. Fivetran can assist users in getting the most out of Salesforce by analyzing all data, combining it with other sources, and increasing reliability and ease of use. By replicating Salesforce data into a SQL-enabled environment, users can transform and display the data to gain insights into their CRM operations. Combining Salesforce data with information from other sources such as support desk software or event tracking apps provides a comprehensive view of customer interactions. Fivetran's connectors simplify this process by automating data integration and offering features like control over synced tables, column hashing for sensitive data, and comprehensive logging. This allows data scientists and analysts to focus on generating insights rather than wrangling data.

Chubbies, a brand known for its distinctive shorts, utilizes a modern data stack comprising Fivetran, BigQuery, and Looker to efficiently gather and analyze evolving consumer behaviors. Fivetran allows seamless replication of applications, databases, events, and files into a cloud warehouse with minimal setup and maintenance, enabling data teams to focus on insights rather than engineering challenges. BigQuery, Google's serverless enterprise data warehouse, provides scalable and cost-effective data analysis, enhancing productivity for data analysts. Looker integrates into applications to offer comprehensive data analytics and business insights across departments, facilitating informed decision-making.