Company
Date Published
Author
Alexander Gallego
Word count
86
Language
English
Hacker News points
None

Summary

At the Hydra Conference 2021, Alex and Noah discussed with Alexey Fyodorov the process and insights gained from building a new storage engine specifically designed for the Kafka API, incorporating a thread-per-core execution model alongside the Raft data replication model. This innovative storage solution eliminates the use of virtual memory and page cache, opting instead for custom read-ahead and write-behind strategies to achieve predictable latencies. The discussion emphasized the co-design approach that integrates these strategies with Raft, aiming to optimize data handling and replication efficiency.