March 2018 Summaries

Aerospike has released version 4.0, which introduces strong consistency while maintaining high performance. This new feature makes Aerospike suitable for applications that require PostGres, MySQL or Oracle primary key correctness with lower latency and higher throughput. The development of this technology took two years and includes a high resolution Lamport clock, improved transaction commit model, enhanced cluster management, and an optional "commit to disk" feature. Aerospike's Early Access program has been successful in testing the technology and business use cases with industry-leading companies. Performance tests using YCSB show that multi-million TPS systems remain practical even with strong consistency enabled.

The text discusses the validation of Aerospike's strong consistency claims through Jepsen tests, which are designed to test distributed systems for problems that can lead to inconsistency and data loss. Kyle Kingsbury ran existing Jepsen tests as well as custom-crafted "nemeses" on Aerospike 3.99.0.3 and higher versions. The results showed no sign of nonlinearizable histories, lost increments to counters, or lost updates to sets. Aerospike's architecture includes distributed record masters that are robust and well suited for high performance Lamport clock implementation. The system is capable of handling clock skew of 27 seconds, which can be caused by real-time clock hardware or operating system "pauses". NTP (Network Time Protocol) open source software can be used to achieve millisecond level time synchronization. Aerospike's performance with strong consistency was tested and it showed high performance even with strong consistency. The availability benefits of committing each write to a storage device were also measured. Kyle Kingsbury found no data loss or linearizability violations in Aerospike 4.0, demonstrating its ability to handle the most demanding data environments with strong consistency guarantees.

Aerospike 4.0 has achieved strong consistency, a revolutionary leap forward for high-fidelity data uses, with dramatically lower latency and higher throughput compared to traditional databases like PostGres, MySQL or Oracle. The technology overcomes an extraordinarily challenging technical problem that took two years to solve, featuring a high-resolution Lamport clock, improved transaction commit model, and cluster management. Aerospike's Early Access program has proven the technology and business use cases with industry-leading social media companies and existing advertising customers. Performance measurements using YCSB tests show nearly identical performance between session consistency mode and strong consistency mode, while multi-million TPS systems remain practical. Key users include AppNexus, which manages over 50 billion unique data points globally, and IBM Fellow Don Haderle, who calls Aerospike a revolutionary tool for creating rich consumer and machine-to-machine experiences.

Aerospike has successfully validated its claims of strong consistency through a series of tests conducted by Kyle Kingsbury, which included running existing Jepsen tests and creating custom "nemeses" to challenge the system. The tests showed that Aerospike does not lose data or allow stale or dirty reads even under high concurrency, challenging network conditions, and crashes. High performance was also demonstrated with session consistency and full linearizability, achieving comparable read and write latency to its AP mode counterpart. Additionally, issues such as node crashes and clock skew were addressed through feature updates and improvements. The results provide strong evidence for Aerospike's ability to handle demanding data environments with strong consistency guarantees.