In early 2021, a new data team was formed to develop a real-time data platform for Kong's SaaS platform, Konnect, aiming to deliver advanced real-time API analytics. The initial architecture included an ingestion service, a message queue, and real-time OLAP data storage, with Apache Druid selected for backend data storage and AWS Kinesis for the message queue due to its managed service benefits and ease of setup. As the platform evolved, the team identified limitations in Kinesis, especially its lack of transactional writes across multiple topics, prompting a migration to Kafka, which offered better transaction support and a more active community. This transition involved integrating AWS MSK with Druid, requiring complex infrastructure setup across AWS accounts, which was facilitated using AWS Private Link for secure connectivity. The team implemented a zero-downtime data migration strategy by double-writing data to both Kinesis and Kafka, ensuring data parity during the transition. Additionally, a feature flag controlled the UI rollout to test the new Kafka integration internally before a full release. The migration to Kafka improved development productivity, resource utilization, and overall engineering efficiency, offering better community support and the ability to operate independently of external services for local development.