BestSecret’s data journey: Moving beyond Snowflake

BestSecret, a European online fashion retailer, is transitioning its data architecture from Snowflake to a more open data architecture using Trino and Starburst, driven by the need to accommodate emerging projects that their existing data stack could not effectively support. The company faced challenges in managing and transforming data with their previous setup, which included dbt, Snowflake, and Airflow, leading to inefficiencies and bottlenecks in their central data engineering team. By experimenting with Trino and conducting a Proof of Concept with Starburst, BestSecret opted for an open data stack that employs the Icehouse design pattern and the Iceberg format, allowing for greater flexibility and efficiency. This new architecture eliminates the need for a monolithic data warehouse, promoting a zero ELT approach with open table formats that prevent vendor lock-in and enable custom functionalities. The shift has resulted in improved efficiencies and has introduced optionality into their data management processes, although Snowflake remains part of their ecosystem.