Keys in ksqlDB, Unlocked

The latest release of ksqlDB, version 0.15, introduces significant enhancements, particularly in its handling of message keys in Apache Kafka. Previously limited in the types of data it could process from message keys, ksqlDB now supports a broader range, including multiple columns and advanced data types like ARRAY and STRUCT, across all supported serialization formats such as JSON, AVRO, and PROTOBUF. This update allows for more complex operations, including the use of PARTITION BY and GROUP BY clauses with multiple expressions, resulting in streams and tables with multiple key columns. Despite these advancements, certain limitations persist, such as the inability to support JOIN operations on sources with multiple key columns, although workarounds involving STRUCT-type key columns are possible. The update also improves integration with Confluent Schema Registry, enabling automatic schema inference for both key and value columns. Future developments promise further enhancements in handling multiple key columns and more sophisticated data operations, positioning ksqlDB as a more robust tool for managing and querying streaming data in Kafka environments.