Introduction to Benchgraph and its Architecture

Benchgraph is an in-house benchmarking tool developed by Memgraph to ensure the consistent performance of its graph database through rigorous testing after every code commit. The tool's architecture comprises Python scripts for managing benchmark execution and a C++ client for executing the benchmarks, which involve running Cypher queries over the Bolt protocol. Benchgraph supports various workloads, such as isolated, mixed, and realistic, to simulate different production scenarios, and it collects metrics like latency and queries per second. The process includes starting the database with specific configurations, importing and exporting data, and executing queries to measure performance. This custom benchmarking solution offers flexibility and is tailored to Memgraph's specific requirements, despite the challenges of building and maintaining such an infrastructure. The tool is integrated into the CI/CD pipeline to provide daily performance reports to Grafana, ensuring the database's reliability and scalability.