Company
Date Published
Author
Matthias J. Sax, Victoria Xia, Wade Waldron
Word count
3103
Language
English
Hacker News points
None

Summary

This blog post is part of a series examining the Streams API of Apache Kafka, focusing on a new application reset tool designed to simplify the process of reprocessing input data from scratch in Kafka Streams applications. The tool, introduced with Confluent Platform 3.0.1 and Apache Kafka 0.10.0.1, addresses the complexities and potential errors of manually resetting applications by managing global resets, while local resets are handled within the application code using KafkaStreams#cleanUp(). The post details how to prepare an application for reset, the steps involved in using the reset tool, and the importance of managing offsets and internal state to ensure accurate reprocessing. It also highlights the drawbacks of using workarounds like changing the application ID, emphasizing the need for proper cleanup of internal topics, state directories, and metadata to avoid resource wastage. The article concludes with an encouragement to try the new tool and provides resources for further learning about Kafka Streams.