August 2022 Summaries

Towards Data Contracts with Census: Iterate faster in dbt without breaking business workflows, Census released dbt Continuous Integration (CI) Checks to help users double-check that their dbt work won’t break their Census syncs. Census is a Leader in the new G2 Reverse ETL category and a Snowflake Premier Technology Partner for Reverse ETL. The company now offers Reverse ETL workflows on EU servers to meet regulatory requirements, including GDPR. New connectors have been added to Census, including PostHog, FullStory, Userpilot, Google Search Ads 360, and The Trade Desk. Census has helped businesses achieve operational efficiency, such as Guru saving 8 hours per week by making data more accessible to its business users.

To help European companies meet their regulatory responsibilities, Census has launched its first European region in Frankfurt (AWS eu-central-1), allowing customers to store and process customer data within the EU. The new region ensures GDPR compliance by storing data securely within the EU throughout a sync duration. This move aims to provide a secure and trusted environment for processing customer data, enabling personalized experiences like timely email marketing campaigns and relevant sales outreach. Census offers seamless migration to EU-specific pipelines, which can be completed in just a few minutes, ensuring minimal disruption to existing syncs and models.

For E-Commerce businesses to grow and become more profitable, developing a reliable analytics engine that consistently generates insights about customers, products, supply chains, and fulfillment operations is critical. Cloud data warehouses have become the central hub for various software-as-a-service tools and in-house operational systems to come together and deliver those critical data insights. Reverse ETL (rETL) enables businesses to put the ownership of their conversion data in the hands of the people who know their customer best, removing barriers to website updates, expanding and refining definitions of what "counts" as conversion, and enabling targeted outreach with personalized promotions. By leveraging rETL, businesses can optimize digital campaigns, automate marketing efforts, guide sales and customer support operations, and improve the customer experience by providing real-time updates on orders and fulfillment information. With a tool like Census or CorrDyn, businesses can operationalize their analytics and turn the flexibility of their analytics system into the flexibility of their business as a whole.

Google BigQuery and Slack are both powerful tools used by data professionals, making it useful to sync data between them to streamline daily workflows. There are three methods to sync data from BigQuery to Slack: manual, using APIs, and through Census, a no-code automation tool. The manual method involves running a SQL query in BigQuery, saving the result to a file, and then uploading it to Slack. This method is straightforward but time-consuming for frequent use. The API method requires coding and setting up APIs, which can be more involved but provides flexibility. Census offers a third option, allowing users to automatically sync data with notifications on updates, making it an efficient choice for regular syncing.

The author, a data analytics engineer at Wejo, participated in the CoRise dbt scholarship offered by Census as part of the Stand Out Data Show campaign. They won the scholarship and completed a four-week, hands-on course on dbt best practices. Through this experience, they gained knowledge on scalable and clear data models using dbt macros and jinja features, and learned how to apply these concepts in their role. The author recognizes the importance of staying up-to-date with best practices in the modern data stack and plans to continue expanding their knowledge with the dbt community. They also emphasize the value of standardization over innovation in this field.

Redshift users can export data from their SQL queries using the UNLOAD command, which allows data to be exported into an Amazon S3 bucket - essentially doing the reverse of the COPY command. The UNLOAD command is available through various methods, including the SQL IDE, AWS SDK, AWS CLI, and SQL client, each with its own set of options and parameters to customize the export process. Redshift users can unload data in different file formats such as CSV, JSON, or PARQUET, and also have options to compress the files using BZIP2, GZIP, or ZSTD. Additionally, there are methods to add quotes around values, partition the data by specific columns, and specify a delimiter for the export process. By understanding these options and parameters, users can achieve more flexibility in their data unloading process.

When a company like Census built the world's first Reverse ETL tool in 2018, its focus was on enabling product-led companies to drive better marketing, sales, and customer success. However, it wasn't until recently that interest in Reverse ETL grew exponentially over the past four years, proving its real business impact. Today, with the creation of Reverse ETL as a new G2 software category, Census is named the leader in this space, thanks to its customers who have synced billions of records to operational tools and used it to automate workflows like personalized email campaigns and account health scores. The company's title as a Leader validates its vision of Operational Analytics resonating with the community, and it has been recognized for its fastest and most reliable connectors from data warehouses to business apps. Reverse ETL is now considered a critical component of the Modern Data Stack, transforming data warehouses into hubs for business operations, empowering everyone with trustworthy and actionable data.

Census is releasing a feature called dbt Continuous Integration (CI) Checks that helps prevent issues with reverse ETL syncs by integrating with GitHub and providing end-to-end visibility into data flows. This new capability enables Data Contracts, which creates an implicit contract between Operations and Data teams to ensure trustworthy, high-quality, and available data. The feature will run integration tests in GitHub to confirm whether any Census syncs will break because of changes made to dbt models, providing early warnings before merging pull requests. With this release, Census deepens its support for software engineering best practices for DataOps and expands its integration with dbt, one of the most popular data transformation tools in the Modern Data Stack.

The data industry is experiencing rapid growth, making it challenging for entry-level professionals to navigate the numerous career options available. To find the right path, experts recommend staying calm and seeking opportunities to learn, as every step in your career journey has value. Prioritizing learning can lead to increased confidence and ultimately, higher earning potential. Be confident in your abilities, look beyond compensation packages when evaluating job offers, and focus on making a positive impact at your company. By anchoring yourself as a senior person and projecting a strong persona, you can command respect and drive change within an organization. Ultimately, the data field is about empowering individuals to overcome their limitations and achieve success.

Hugo is a meeting productivity hub that aims to simplify meeting workflows for entire teams by providing a one-stop shop for preparing for meetings, taking notes, and integrating with existing tools. The company uses data analytics to provide personalized user experiences for its customers, resulting in more conversions. Darren Chait, co-founder and COO of Hugo, shares his journey from being a corporate lawyer to setting up the modern data stack and how he learned about data tools like dbt through the no-code movement. With the help of tools like Census, Hugo was able to transform raw data into actionable insights that drive business impact, such as increasing conversions by 3-4 times through personalized email campaigns. Darren emphasizes the importance of getting the data stack right from day one, warehousing everything in a single spot, and democratizing access to data for all team members to deliver more value for users, quicker.

Reverse ETL is the final step in the data analytics process, enabling businesses to deliver product metrics or marketing audiences to stakeholders in minutes, not weeks. Census makes it easier than ever to scale the impact of data and analytics by providing a seamless integration with Snowflake warehouses. This partnership allows customers to operationalize their data and insights, empower business teams to self-serve, and multiply investments in existing tools like dbt, Airflow, and Looker. With reverse ETL, businesses can drive more actionable data and automate workflows with confidence.

The technical interview is a complex process that companies struggle to get right, despite having access to numerous resources on how to improve it. The lack of clear guidelines and best practices for conducting interviews leads to inconsistent results and poor hiring decisions. Key experts in the field, including CoderPad's David Corea and Census CEO Boris Jabes, emphasize the importance of creating a fair and unbiased interview process that measures a candidate's hard skills and imagination. They argue that traditional whiteboard-based interviews are not effective in assessing a candidate's ability to solve real-world problems. Instead, they recommend using tools like CoderPad to provide candidates with a familiar editor and unbiased problems to showcase their skills. Additionally, experts stress the need for companies to experiment and iterate on their interview processes to measure impact and improve results. By doing so, both companies and candidates can benefit from more effective hiring decisions.