Apache Kafka Needs No Keeper: Removing the Apache ZooKeeper Dependency

KIP-500` is a proposal to move metadata management into Apache Kafka itself, breaking its dependency on external systems like ZooKeeper. This change aims to reduce complexity and improve efficiency by storing metadata in a partition inside Kafka, eliminating the need for an external metadata system. The project introduces a Raft protocol for the metadata quorum, allowing for self-managed replication and minimizing controller failover time. KIP-500 mode will be experimental initially but is expected to become the default as it gains confidence. The proposal also includes bridge releases to enable zero-downtime upgrades from older ZooKeeper-based versions of Kafka. By simplifying architecture and leveraging the event log abstraction, KIP-500 aims to prove that Kafka needs no external keeper like ZooKeeper.