In recent years, Apache Kafka has gained significant popularity among companies such as Uber, Twitter, Netflix, LinkedIn, Yahoo, Cisco, Goldman Sachs, etc., due to its scalability and real-time pub/sub capabilities. To overcome the limitations of traditional databases, specialized systems have been built to focus on a single task, leveraging commodity hardware and open-source projects, which reduces costs and enables faster development. These systems often rely on data streaming platforms like Kafka to ingest and process large amounts of data in real-time. The trend is shifting towards stream data platforms powered by distributed pub/sub systems like Kafka, where multiple specialized systems co-exist, requiring a rethinking of the data curation process to handle schema evolution and duration reasoning early on. Companies can leverage this architecture to build scalable applications, such as Gen AI applications, using frameworks like Confluent's Kappa Architecture.