June 2022 Summaries

Astronomer has been named a finalist in the Data category for the 15th Annual Ventana Research Digital Innovation Awards, thanks to its Astro platform, a fully managed data orchestration service powered by Apache Airflow. Competing alongside companies like Bigeye and MongoDB, Astronomer is one of 30 finalists across various categories, such as Analytics, Customer Experience, and Digital Business. The awards, presented by Ventana Research, a renowned market research and advisory firm, highlight technological innovation and its role in enhancing efficiency and productivity. Winners, including the recipient of the Overall Digital Innovation Award, will be announced on June 29.

Astro is a newly launched, fully managed data orchestration platform that leverages Apache Airflow® and OpenLineage, providing an enhanced experience for building, running, and monitoring data pipelines. Designed by the creators of Airflow and OpenLineage, Astro offers features such as integrated data lineage for comprehensive pipeline visibility, a cloud-optimized distribution of Airflow for reliable operation, and an Astro CLI for efficient local development. Available on AWS and Google Cloud, with Azure support forthcoming, Astro provides a single control plane for centralized management while the data plane remains in the user's cloud. This architecture allows organizations to manage their data pipelines effectively, ensuring reliability and cost-efficiency, and enabling seamless scaling and observability, ultimately reducing the need for extensive DevOps involvement and offering rapid deployment of the latest Airflow features.

Astronomer has introduced two new commands, astro dev parse and astro dev pytest, to enhance the testing of Directed Acyclic Graphs (DAGs) in Apache Airflow using the Astro CLI. These commands streamline the development and testing process by allowing users to identify syntax and import errors quickly, ensuring that DAGs render correctly in the Airflow UI, and facilitating the creation and execution of pytests locally. The astro dev parse command accelerates error detection, saving time in debugging, while astro dev pytest enables users to run DAGs against custom or default pytests to ensure compliance with organizational requirements. These enhancements simplify the traditionally complex and time-consuming process of setting up a local Airflow environment, reduce overhead by eliminating unnecessary component setups, and improve feedback on errors. Additionally, the astro deploy command now integrates the parse functionality, offering a final assurance that DAGs are error-free before cloud deployment, thereby improving reliability and security without requiring sensitive credentials to be stored locally.