May 2019 Summaries

Ethan Lockshin shares his experience at Fivetran, an independent ELT provider with fully managed automated data pipelines. Since joining in 2018 as employee #92, he has grown both personally and professionally, learning industry, product, leadership, and sales knowledge. The company has expanded from around 35 employees to approximately 120, emphasizing the importance of adaptability and continuous improvement. Fivetran encourages its employees to act as owners and adhere to core values such as integrity, initiative, trust, inclusiveness, leadership, and teamwork. Ethan highlights the diverse and supportive work environment at Fivetran, which contributes to its success in the industry.

Data engineers often spend their time building and maintaining data pipelines using classic ETL tools, which can be costly and inefficient. Modern data pipeline tools offer a more effective solution by automating these processes, allowing data engineers to focus on higher-value activities such as developing machine learning models and driving growth. As the demand for data engineers increases, companies must consider the return on investment of building their own data pipelines and prioritize staff morale and productivity.

Fivetran introduces the Tech Equity Pipeline Program (TEPP), an inclusive internship experience aimed at increasing equity and diversity in the technology industry. The program does not require a college degree or traditional background, focusing instead on positive attitude and basic skills. Underrepresented groups such as Black, Hispanic, Native American, Pacific Islander, individuals with disabilities, GLBTQIA, women, and veterans are encouraged to apply for this full-time, paid six-month internship program. Successful interns may be offered extension or conversion to full-time employment, including medical, dental, and vision benefits. TEPP is currently available in Oakland, CA with the engineering team and Denver, CO with the sales team.

MVF, a customer generation platform, integrates 14 data sources to create a single source of truth and identify new customers while uncovering revenue opportunities. This integration is made possible by Fivetran technology, which replicates applications, databases, events, and files into a high-performance cloud warehouse. The resulting insights are then used by businesses to drive growth in new markets and channels.

Advertising data can be difficult to replicate accurately and often lacks sufficient context for full-funnel analysis. Fivetran addresses these issues by providing predefined schemas, combating sampling error, accounting for restatements and attribution windows, syncing accounts into a single table, and capturing metadata. These features ensure accurate, full-funnel analysis that uncovers the true ROI of advertising campaigns.

When evaluating and trialing data pipelines for centralizing business data, consider the following factors: data sources supported by the pipeline service, its ability to handle large quantities of data while maintaining quality, ease of configuration, alignment with your organization's analysis needs, security measures in place, and customer service reputation. These aspects will help ensure a seamless transition to a modern data stack that aligns with your individual and company goals.

Fivetran, a data engineering company founded in 2014, initially developed its first data connector as an extension for their "real" product - a pivot table for big data built on Amazon Redshift. However, the genie escaped the bottle when customers started requesting more connectors and eventually led to Fivetran's focus on building automated data pipelines. The company aimed to solve complex engineering problems related to data replication and has since developed fully automated, fully managed data pipelines for various source types. Their ultimate goal is to be a true piece of infrastructure that can be used as a foundation for other tools in the industry. Fivetran's focus on reliability and quality is evident in their commitment to achieving 99.999% reliability, which requires engineering miracles.

Fivetran is a leading ETL solution that focuses on building efficient connectors for cloud data warehouses. The company's API connectors, data preparation techniques, and data loading methods set it apart from other solutions in the market. Fivetran's strategy involves pulling data from applications using an optimized approach to minimize API calls and ensure reliability. For databases, Fivetran leverages change data capture technology to incrementally detect and replicate data changes with minimal load on the production system. The company prepares the data by transforming it into a standard schema that is normalized for better integrity and ready for analysts. Finally, Fivetran loads the data into the warehouse as CSV files, automatically creates tables, and supports loading multiple sources at once to reduce costs. Security measures include control over data loaded into warehouses, encryption in transit and at rest, and deletion of data from their system after it is loaded into the user's warehouse.