Company
Date Published
Author
Vicky Papavasileiou, Vinoth Chandar, Victoria Xia, Wade Waldron
Word count
2278
Language
English
Hacker News points
None

Summary

ksqlDB has improved its high availability by implementing heartbeating for failure detection, allowing pull queries to access underlying state during rebalancing while Streams standby replication brings the new server up to speed with the failed server, and enabling lag-aware routing where servers can quickly determine which other servers are alive and how caught up their tables are. This enables highly available pull queries by specifying a maximum allowed offset lag for each query, allowing users to trade off some consistency for more availability. The new design also allows ksqlDB to be configured as an AP system, trading inconsistency within specified bounds for availability, though once the inconsistency exceeds the bounds, it returns to being a CP system.