Introducing Apache Airflow® 3.3

Airflow 3.3, released on July 6th, introduces significant features like the task state store and pluggable retries, aimed at enhancing the resilience and failure management of data pipelines. The task state store enables tasks to persist small pieces of information across retries, allowing for reconnection to ongoing external jobs without starting over, while pluggable retries allow users to customize retry policies based on the type of failure encountered. These updates are particularly beneficial given the emphasis on managing failures in data engineering. The release also includes improvements such as updates to asset partitions, a new language task SDK for Java and Go, and various UI enhancements, making it a compelling upgrade for users. With contributions from the Apache Airflow community and the first managed service support from Astronomer, Airflow 3.3 promises to streamline pipeline operations and reduce the time spent on failure management.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.