June 2025 Summaries

Kestra version 0.23 has undergone significant performance enhancements, largely driven by contributions from Xiaomi Engineering and insights from the open-source community. These improvements include faster execution, smarter scheduling, and reduced resource usage. Notable advancements involve optimizing the merging process of task outputs, refining flowable task execution to eliminate unnecessary overhead, and enhancing the JDBC scheduler's efficiency by introducing a dedicated monitoring loop. Additionally, the introduction of missing indices on the JDBC backend and changes to the parallelization of JDBC Backend queues have further enhanced performance. Xiaomi's engineering team played a crucial role in these optimizations, demonstrating substantial improvements in execution times and system responsiveness, particularly in large-scale scenarios. The collaboration exemplifies how real-world usage and targeted contributions can propel a platform's development, with Kestra now better equipped to handle extensive workloads efficiently.

The 0.23 release of the Kestra platform introduces Unit Tests for Flows, a feature designed to enhance the reliability of workflow automation by enabling automated, isolated testing of individual flow components. This functionality allows users to verify flow logic without triggering side effects or interfering with production data, making it possible to safely test workflows that interact with external systems such as APIs and databases. Unit Tests in Kestra are written in YAML, allowing for language-agnostic and accessible test creation and management directly from the Kestra UI. The tests support mocking of inputs, files, and task outputs, enabling users to simulate various scenarios and ensure that tasks behave as expected through assertions. This feature facilitates parallel testing at scale without degrading system performance and keeps test executions separate from production executions for a clean workflow environment. Available in the Enterprise Edition and Kestra Cloud, Unit Tests aim to prevent regressions, maintain workflow integrity, and provide clear feedback on test outcomes, contributing significantly to a robust workflow automation process.

Kestra 0.23 introduces several upgrades to enhance the orchestration experience, including a multi-panel editor that allows for simultaneous code and no-code workflows, unit tests for flows using a declarative syntax, and faster UI filters with plain text editing. It provides tenant-based storage isolation to ensure data separation across tenants, customizable dashboards, and Python dependency caching to reduce task execution times. The release also integrates Git for version control, supports new plugins like Salesforce and HubSpot, and introduces enhancements such as more efficient data storage and retrieval methods. Additionally, Kestra 0.23 features a revamped approach to managing workflow outputs, notable plugin enhancements, and breaking changes that require migration for existing users. The update aims to improve usability, data governance, and performance while fostering community contributions and feedback.

Retrieval Augmented Generation (RAG) enhances generative AI by combining creative AI outputs with real-time, contextually accurate data, offering more precise and relevant responses. The blog outlines the process of building a RAG pipeline using Google Gemini AI and the Langchain4J plugin in Kestra, emphasizing tasks like document ingestion, embedding creation, retrieval strategies, and augmented generation. By leveraging Kestra's Langchain4J plugin, users can create complex AI workflows that are provider-agnostic and utilize embeddings stored in a vector database, ensuring that responses are enriched with contextual information from custom data. The setup facilitates the splitting of documents into segments, storing embeddings for later retrieval, and generating context-rich responses, thus improving the accuracy and relevance of AI outputs. The process demonstrates the integration of AI models like the Google Gemini, highlighting their capability to manage large context windows efficiently and cost-effectively, making it suitable for automated workflows. This approach offers a scalable solution for incorporating precise data into AI-driven tasks, allowing for the generation of tailored and up-to-date responses while maintaining flexibility in sourcing documents from various inputs.