Company
Date Published
Author
Lucia Cerchie, Justine Olshan, Josep Prat
Word count
1696
Language
English
Hacker News points
None

Summary

The sticky partitioning strategy in Apache Kafka 2.4 aims to improve performance by sending all records to one batch and potentially filling it earlier, reducing latency when producing keyless messages. The new partitioner, DefaultPartitioner, implements this feature, which can lead to significant reductions in latency compared to the original default partitioning strategy. The sticky partitioner is particularly beneficial in scenarios with high throughput, low latency requirements, or a large number of partitions. By sending fewer but larger batches, the producer experiences improved performance, including reduced CPU usage. This improvement becomes more pronounced as the number of partitions increases.