August 2026 Summaries

Astronomer announces major performance and reliability enhancements to its managed Astro platform, re-engineering key Apache Airflow scheduling, execution, scaling, and recovery components while retaining Airflow compatibility without requiring customer changes. The platform is designed for modern data and AI orchestration workloads, including ETL, dbt transformations, model training and evaluation, and event-driven pipelines, and is reported to support up to 500,000 concurrent tasks. In load tests, Astro achieved p95 task-start latency of about 230 milliseconds at 100,000 concurrent tasks and roughly 300 milliseconds at 300,000 tasks, compared with Celery’s reported 23.582-second latency at 50,000 tasks. Astro attributes these results to event-driven scheduling, an Airflow-aware scaling and healing layer, high-availability components, and multi-region database recovery capabilities, while claiming 85% fewer failures under load than open-source Airflow.

Astronomer has made cross-region disaster recovery for Astro generally available on Azure, completing its rollout across AWS, Google Cloud, and Azure for Enterprise Business Critical customers. The feature allows customers to enable a secondary Azure region through the UI, API, or Terraform, with Astro managing replication of Airflow metadata through Azure PostgreSQL geo-replicas, task logs and container images through Blob Storage object replication, and a warm-standby AKS environment designed to restore workloads in under one hour with a target recovery point objective of 15 minutes for supported task-log region pairs. Astro retains a one-cluster, two-region control-plane model so cluster identities, hostnames, and customer-facing endpoints remain unchanged after failover, while DNS redirects traffic to the promoted region. Azure’s replication capabilities allow DR to be enabled on existing clusters without downtime or separate data migrations, though customers must prepare secondary-region workload identities, private networking connections, registry secrets, and any custom routing within their own Azure tenants. Replication health is monitored directly using Azure storage policy status and PostgreSQL replication-delay metrics, and failover promotes the database replica, creates workloads on the standby cluster, updates DNS, and establishes reverse replication for eventual failback.