Company
Date Published
Author
-
Word count
1004
Language
English
Hacker News points
None

Summary

Memgraph, an in-memory graph database, has recently introduced support for data storage exceeding memory limits, enabling users with limited budgets to load large datasets without incurring high costs for additional RAM. This development involves a shift from purely in-memory databases, which are fast but limited by memory capacity, to a larger-than-memory architecture that balances in-memory computation with disk storage. By identifying and transferring "cold" data to disk while keeping "hot" data readily accessible, Memgraph employs RocksDB for disk storage, utilizing Log-Structured Merge-Trees to minimize write amplification. Despite the advantages, such as cost-effective data storage and maintained performance, challenges remain, including the requirement that transactions fit into memory, which limits the execution of large analytical queries. The Memgraph team continues to enhance this architecture, addressing the complexities and limitations introduced by integrating disk-based storage.