February 2025 Summaries

Kestra engineers have implemented a series of performance optimizations in versions 0.19 and 0.20 to enhance the resource efficiency of their orchestrator. Key improvements include enhanced data serialization through the use of the ION format, reducing CPU and memory usage in the PostgreSQL backend by modifying JSONB handling, and adding a missing index to JDBC backends for improved queue execution. They also adjusted the default number of Worker threads to better align with available CPU cores, optimized log handling by introducing asynchronous logging and batching, and allowed logging to file to mitigate execution time impact for log-heavy tasks. Additionally, the Worker component was refactored to avoid unnecessary CPU cycle consumption by preventing the mutation of WorkerTask, thereby reclaiming processing time for task execution. These updates maintain Kestra's position as a scalable and high-performing orchestration platform.

Kestra 0.21 introduces several new features and improvements, including no-code forms for easier workflow creation, customizable dashboards for enhanced monitoring, and a new core property called "finally" for executing cleanup tasks at the end of workflows. The update also features a Log Shipper, which efficiently manages and distributes logs across various platforms like Elasticsearch and AWS CloudWatch, and offers a redesigned no-code flow editor for better user experience. Additionally, Custom Dashboards allow users to tailor monitoring to their specific needs, and Maintenance Mode facilitates platform updates without disrupting ongoing workflows. Other enhancements include improved UI and UX, OpenTelemetry support, and new plugins such as the AWS EMR and HuggingFace Inference plugins, enabling integration with external services and tools.