Content Deep Dive
Updating our data stack
Blog post from Incident.io
Post Details
Company
Date Published
Author
Jack Colsey
Word Count
2,451
Language
English
Hacker News Points
-
Source URL
Summary
In this update on their data stack, incident.io has made several tweaks and improvements over the past two months. They have migrated Fivetran to a separate BigQuery project to deal with concurrent query limits, set up CircleCI to run the dbt pipeline sequentially, moved from using Google Sheets for GBP:USD FX rates to an API feed, materialized all of their dbt models as tables, introduced intermediate tables into their dbt pipeline, and used SQLFluff VSCode extension for automatic formatting and error detection. They have also updated their folder structure in Metabase to differentiate one-off analyses from those used by stakeholders across the company.