Introducing Kafka-Kit: Tools for scaling Kafka

Kafka-Kit` is a collection of tools developed by Datadog for managing Kafka clusters. It includes `topicmappr`, which handles partition to broker mappings, failed broker replacements, storage based partition rebalancing, and replication auto-throttling. The tool's primary inputs are topics and brokers, and it provides deterministic output and minimizes movement broker replacements. It also exposes a selection of partition replica placement strategies, including the `count` strategy, which ensures leadership is maximized among brokers and partitions are evenly distributed. Another strategy, `storage`, prioritizes even storage utilization across brokers while satisfying locality constraints. The tool can be used for capacity planning, data mapping, and making data movement and recovery hands-off. It's designed to work with Kafka 0.10.1 and has features such as dynamic throttle adjustment and automated recovery.