Highly Available, Fault-Tolerant Pull Queries in ksqlDB

ksqlDB has improved its high availability by implementing heartbeating for failure detection, allowing pull queries to access underlying state during rebalancing while Streams standby replication brings the new server up to speed with the failed server, and enabling lag-aware routing where servers can quickly determine which other servers are alive and how caught up their tables are. This enables highly available pull queries by specifying a maximum allowed offset lag for each query, allowing users to trade off some consistency for more availability. The new design also allows ksqlDB to be configured as an AP system, trading inconsistency within specified bounds for availability, though once the inconsistency exceeds the bounds, it returns to being a CP system.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.