July 2026 Summaries

The Plugins & Ecosystem Squad manages over 200 repositories and has developed Plugin Devtools, a collection of standalone shell scripts to automate repetitive tasks and streamline operations across the Kestra ecosystem. These scripts aim to simplify daily processes, reduce errors, and enhance efficiency by providing a single command solution for tasks that would otherwise require multiple steps. They are designed to be used both by developers and AI agents, allowing for consistent and reliable execution of tasks such as deploying plugins, managing repositories, and releasing updates. The scripts also facilitate fleet management, API coverage assessment, and automated updates, reducing the need for manual intervention and enabling smoother workflows within the organization. This approach turns scripts into shared contracts between humans and agents, ensuring that improvements are implemented once and benefit all users, thereby saving time and resources.

Kestra's deployment model, particularly for enterprise customers, revolves around a clear understanding of its four main components: the webserver, scheduler, executor, and worker, each of which plays a distinct role in managing workflow execution and scaling. The webserver handles UI and API requests, the scheduler monitors triggers to initiate execution events, the executor coordinates the workflow's execution logic without making external calls, and workers execute tasks by interacting with external systems. Deployments typically run on Kubernetes, allowing for scaling and failover through multiple replicas of each component, with tuning primarily focusing on the executor and worker counts as load increases. Kestra's flexibility extends to multi-region, hybrid, and hardware-specific deployments, with worker groups facilitating task routing across different infrastructures, ensuring compliance with data residency requirements and optimizing resource utilization. The system can scale from standalone development setups to extensive distributed deployments, with PostgreSQL as the standard backend, upgradeable to Kafka and Elasticsearch for higher throughput and execution history management. This open-source platform offers a unified orchestration backbone that can adapt to diverse organizational needs and complex environments, with options for further customization and scaling depending on workload demands and regulatory constraints.

Leroy Merlin transitioned from Airflow to Kestra for orchestrating their workflow executions, achieving significant improvements in performance and scalability. Their previous system was prone to slow execution and risked entire platform failures due to single task issues. The new Kestra architecture comprises three layers designed for scaling execution, isolating teams, and controlling execution locations. This architecture allows teams to operate independently through constructs like namespaces and tenants, providing isolation and user management tailored to organizational needs. Kestra's design enables tasks to be routed based on specific requirements such as data residency and GPU routing, using worker groups that can be scaled independently. This multi-layered approach not only supports extensive workflow executions but also addresses compliance and geographical constraints without the need for significant infrastructure rebuilding. The platform's open-source nature and ease of deployment make it a versatile choice for enterprises with complex orchestration needs.