How Kestra engineers optimized orchestrator performance

Kestra engineers have implemented a series of performance optimizations in versions 0.19 and 0.20 to enhance the resource efficiency of their orchestrator. Key improvements include enhanced data serialization through the use of the ION format, reducing CPU and memory usage in the PostgreSQL backend by modifying JSONB handling, and adding a missing index to JDBC backends for improved queue execution. They also adjusted the default number of Worker threads to better align with available CPU cores, optimized log handling by introducing asynchronous logging and batching, and allowed logging to file to mitigate execution time impact for log-heavy tasks. Additionally, the Worker component was refactored to avoid unnecessary CPU cycle consumption by preventing the mutation of WorkerTask, thereby reclaiming processing time for task execution. These updates maintain Kestra's position as a scalable and high-performing orchestration platform.