Exorcising Ghost Apache Kafka Controllers

A Kafka defect affecting versions 3.9 through 4.3 could leave removed controllers recorded in cluster metadata as “ghost” controllers, preventing clusters from enabling newer features such as Share Groups or upgrading their metadata version. Although Kafka 4.4 introduced a native controller-unregistration command, affected clusters faced a circular dependency because outdated ghost-controller registrations blocked the metadata upgrade required to use that command. Aiven developed patches for Kafka 4.0–4.3 that reuse existing controller-registration metadata to mark permanently offline, non-voter controllers as decommissioned, allowing patched clusters to ignore them during feature and metadata-version validation without introducing incompatible metadata records. The patch was also carried forward into version 4.4 so that clusters could transition safely to Kafka’s official unregistration mechanism and eventually remove the obsolete registrations. Aiven plans to deploy the fix automatically through maintenance updates for users of its managed platform, while patches are available for organizations maintaining their own Kafka forks.

No tracked trend matches for this post yet.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.