Company
Date Published
Author
David Wallace
Word count
849
Language
English
Hacker News points
None

Summary

At Good Eggs, a local organic grocery store, data modeling is crucial to manage marketing, purchasing, customer buying patterns, personalization, retention, and logistics strategies. The team uses a diverse stack of tools, including dbt, Snowflake, Mode Analytics, Jupyter Notebooks, Stitch, and custom Singer taps. However, managing this complex system poses challenges, such as tracking dependencies, ensuring data quality and freshness, and controlling resource usage. To address these issues, Good Eggs adopted Dagster, an orchestrator framework that provides features like support for custom data types, structured metadata, and testability, ultimately transforming their data platform team's software engineering lifecycle.