Company
Date Published
Author
Andy Coates
Word count
5391
Language
English
Hacker News points
None

Summary

ksqlDB 0.10 introduces major updates to key management, enhancing support for multiple key columns and formats like Avro, JSON, and Protobuf. In this version, both tables and streams in ksqlDB manage keys uniquely: tables have a PRIMARY KEY for unique row identification, while streams can declare a KEY column without inferring uniqueness. This update removes the confusing WITH(KEY) syntax, allowing users to name key columns freely and eliminating the need to duplicate keys in Kafka record values. Streams are now supported without key columns, though this may necessitate data repartitioning for operations like joins and aggregations. The new AS_VALUE function allows users to replicate the previous behavior of copying keys into values when necessary. Key column naming has been refined, and users must now explicitly include key columns in projections, impacting materialized views and joins. These changes, while not backward compatible, aim to provide a more intuitive and expressive framework for managing data in Kafka.