July 2026 Summaries

Kestra's deployment model, particularly for enterprise customers, revolves around a clear understanding of its four main components: the webserver, scheduler, executor, and worker, each of which plays a distinct role in managing workflow execution and scaling. The webserver handles UI and API requests, the scheduler monitors triggers to initiate execution events, the executor coordinates the workflow's execution logic without making external calls, and workers execute tasks by interacting with external systems. Deployments typically run on Kubernetes, allowing for scaling and failover through multiple replicas of each component, with tuning primarily focusing on the executor and worker counts as load increases. Kestra's flexibility extends to multi-region, hybrid, and hardware-specific deployments, with worker groups facilitating task routing across different infrastructures, ensuring compliance with data residency requirements and optimizing resource utilization. The system can scale from standalone development setups to extensive distributed deployments, with PostgreSQL as the standard backend, upgradeable to Kafka and Elasticsearch for higher throughput and execution history management. This open-source platform offers a unified orchestration backbone that can adapt to diverse organizational needs and complex environments, with options for further customization and scaling depending on workload demands and regulatory constraints.

Leroy Merlin transitioned from Airflow to Kestra for orchestrating their workflow executions, achieving significant improvements in performance and scalability. Their previous system was prone to slow execution and risked entire platform failures due to single task issues. The new Kestra architecture comprises three layers designed for scaling execution, isolating teams, and controlling execution locations. This architecture allows teams to operate independently through constructs like namespaces and tenants, providing isolation and user management tailored to organizational needs. Kestra's design enables tasks to be routed based on specific requirements such as data residency and GPU routing, using worker groups that can be scaled independently. This multi-layered approach not only supports extensive workflow executions but also addresses compliance and geographical constraints without the need for significant infrastructure rebuilding. The platform's open-source nature and ease of deployment make it a versatile choice for enterprises with complex orchestration needs.