September 2026 Summaries

Kestra reviewed seven plugins to reduce repeated hot-path work and address silent data-loss or error-reporting failures, using JMH benchmarks and real local services to measure performance changes where applicable. JDBC now pools remote database connections by default and caches result-set metadata per result set, yielding roughly 20–25 times faster connection operations with about 200 times less allocation and improved row-mapping throughput. Serdes reuses date formatters and an error-path JSON mapper, reducing parsing overhead and making exception-message processing about 66 times faster, while Redis batches list pushes into variadic LPUSH commands, reducing 10,000 pushes from 10,000 round trips to 20 and substantially improving latency-sensitive workloads. MongoDB loading now passes maps directly as documents instead of serializing and reparsing JSON, producing roughly a 25-fold speed improvement and far less garbage. Reliability fixes ensure GCP Pub/Sub waits for in-flight messages during shutdown, BigQuery supports concurrent appends, AWS Athena and S3 paginate beyond 1,000 results, SQS batches sends, and Kafka propagates producer failures rather than reporting successful tasks when messages were not delivered.

Kestra 2.0 replaces the platform’s prior engine architecture to address database-dependent workers, queue bottlenecks, and the maintenance burden of separate JDBC and Kafka Streams implementations. The new design decouples queue and repository backends, retaining JDBC support for open-source deployments while offering Redis, AMQP, and Kafka queue options in Enterprise, with a single executor, scheduler, and worker implementation across configurations. A new Controller service intermediates communication between workers and backend systems through secured outbound gRPC connections, allowing workers to operate in separate clouds, regions, on-premises environments, or restricted networks without direct database access or exposed inbound ports. The release also reduces queue traffic by loading task outputs only when required, introduces an Indexer Service for non-JDBC backend configurations, supports workload routing through Worker Groups, and enables logs to be stored separately in JDBC, Elasticsearch, Datadog, or Splunk to reduce pressure on primary databases.