Home / Companies / Memgraph / Blog / Post Details
Content Deep Dive

Improve Load CSV

Blog post from Memgraph

Post Details
Company
Date Published
Author
Antonio Filipovic
Word Count
3,880
Language
English
Hacker News Points
-
Summary

In a detailed exploration of optimizing the LOAD CSV clause in Memgraph, the text delves into the challenges of memory consumption and performance issues that arise when importing large datasets. LOAD CSV, a Cypher clause, is crucial for importing data from CSV files into Memgraph, especially when transitioning from older systems. However, the process can be memory-intensive and suffer from performance degradation due to the creation of numerous delta objects during a single transaction. Memgraph 2.7 introduces two storage modes: the ACID-compliant IN_MEMORY_TRANSACTIONAL mode and the newly developed IN_MEMORY_ANALYTICAL mode, which disables delta object creation to alleviate memory and performance issues. The text also discusses the use of different memory allocation strategies and profiling tools like massif, gperftools, and flamegraph to identify and address inefficiencies in memory usage and execution speed. The improvements culminated in better memory management and faster data import processes, allowing Memgraph to handle large CSV imports more efficiently, though the creation of delta objects still poses a constraint on ultimate performance gains.