Company
Date Published
Author
Pranav Bahadur
Word count
654
Language
English
Hacker News points
None

Summary

Cosmos, an open-source standard developed by Astronomer for orchestrating dbt with Airflow, is advancing to enhance data pipeline efficiency, intelligence, and observability. With over 21 million monthly downloads, it facilitates the seamless integration of dbt transformations into Airflow pipelines, streamlining data transformation management within an open-source framework. The latest update, Cosmos 1.11 alpha, introduces support for dbt Fusion, a new dbt engine that offers fast parsing speeds, state-aware execution, and reduced warehouse costs. This update is designed to simplify the adoption of Airflow for orchestrating dbt Core projects, as noted by a user from System1. Cosmos now includes full support for Airflow 3, flexible execution modes, and expanded database support, all while maintaining platform independence. The tool is positioned as a future-proof solution with ongoing support for all dbt versions, including dbt Fusion. Users are encouraged to contribute feedback via GitHub, and comprehensive guidance is available for those looking to integrate dbt Fusion with Cosmos.