Validating consistency and the absence of data loss in Redpanda

Redpanda is presented as a Kafka replacement for mission-critical systems, emphasizing the importance of rigorous testing to ensure system reliability. The text highlights the role of chaos testing, a method used to validate systems under fault conditions, which was popularized by Netflix and involves intentionally injecting faults like network partitioning and process termination to test system resilience. Redpanda incorporates the Raft protocol for linearizability tests, ensuring correct concurrent operation execution, and has developed an in-house consistency checking tool, Gobekli, to address the challenges of validating long operation histories. This approach allows for real-time validation of linearizability with low computational complexity, essential for maintaining the integrity and reliability of distributed systems. The article also details the fault injection process and the observed effects on latency and availability, concluding with Redpanda's commitment to providing a trustworthy and robust system, inviting users to try the product and engage with their community.