Home / Companies / Kestra / Blog / Post Details
Content Deep Dive

Kestra 0.4 release, Performance & Kafka, JDBC and Singer plugins

Blog post from Kestra

Post Details
Company
Date Published
Author
Ludovic Dehon
Word Count
1,766
Language
English
Hacker News Points
-
Summary

Kestra's 0.4 release introduces significant performance improvements and new features, particularly enhancing the processing of large clusters and reducing CPU usage and latency in environments like Leroy Merlin's production setup. The update optimizes Kafka processing to minimize queuing delays and enhances the efficiency of task execution, notably during heavy overnight workloads. New and updated plugins include JDBC plugins for bulk queries, expanded Kafka capabilities, and enhanced Singer plugin functionality for managing multiple data sources and destinations. Additionally, the release includes a CustomJob integration with GCP Vertex AI for scalable machine learning deployments and improved BigQuery task retries to prevent failures. The Enterprise Edition now offers recursive flow dependency visualization, aiding in impact analysis across data pipelines. Subsequent minor updates, versions 0.4.1 and 0.4.2, address bugs and refine the user experience, and further plugin enhancements are anticipated for stronger ecosystem integration.