May 2022 Summaries

Docker is a versatile open platform used for developing, shipping, and running applications within isolated environments known as containers, which include all necessary libraries for operation, making applications easily shareable and consistent across different hosts. Docker Compose enhances this by allowing the definition and management of multi-container applications through a YAML configuration file, facilitating deployment across various environments such as production, staging, and testing. An example of its use is in creating demo web applications for Memgraph, where a Dockerfile specifies the Docker image and necessary libraries for each component, such as backend, frontend, and database services, all integrated into a bridge network for seamless communication. The Memgraph Platform image, often employed due to its comprehensive features, supports graph algorithms and modules, with configurations adjustable via the MEMGRAPH environment variable. Demonstrative applications like the Twitch analytics demo utilize services such as Kafka and Zookeeper, showcasing Docker Compose's ability to simplify deployment and operation of complex setups, thereby offering a streamlined development process for graph applications.

Knowledge Graphs are advanced data structures that facilitate complex queries across multiple data silos by organizing data into tables and graphs, allowing for high connectivity and adaptability to new information. Various Python packages are available to work with knowledge graphs, each offering unique features and benefits. Pykg2vec focuses on knowledge graph embedding with a framework built initially on TensorFlow and later adapted to PyTorch, offering Bayesian hyperparameter optimization and interactive visualizations. PyKEEN provides tools for building and evaluating knowledge graph embeddings with features like automatic memory optimization and community-driven tools. AmpliGraph democratizes graph representation learning, enabling users to generate and utilize knowledge graph embeddings for tasks like link prediction. LibKGE aims to enhance research in KGE models by providing tools for training, hyperparameter optimization, and logging, supporting a wide range of models and configurations. GraphVite combines a C++ core library with a Python interface to enable high-speed processing of large-scale graphs and supports dynamic data types and high-dimensional data visualization.

Graph query languages (GQLs) are essential for managing and extracting data from graph databases, with each language offering unique features and catering to different needs. GraphQL is a modern alternative for REST-based architectures, providing flexibility and precise data requests, especially beneficial for complex microservices and legacy infrastructures, though it may face performance issues with nested queries. Gremlin, part of Apache TinkerPop, excels in path-oriented graph traversals across different programming languages but can be challenging to read and write for complex queries. Cypher, associated with Neo4j, is intuitive and pattern-based, ideal for data analytics and application development, though it struggles with high write loads and lacks some data types. SPARQL, designed for RDF data, supports federated queries across different repositories but is complex and limited in recursive querying. AQL, the query language for ArangoDB, is human-readable and supports complex computations, yet it lacks data definition capabilities and has execution limitations. Each language has its pros and cons, and the choice depends on the specific data and operations needed.

Graph Data Zagreb is hosting its fourth meetup on June 8, 2022, at WESPA Spaces in Zagreb, attracting nearly 200 graph and data enthusiasts. The upcoming event will feature Boris Agatić, a data scientist and tech entrepreneur, who will discuss the transformation of Bitcoin blockchain transactions into a graph to analyze the decentralized distributed payment network. Agatić, known for his expertise in blockchain technology and artificial intelligence, will leverage graph databases to explore Bitcoin's public and real-time transaction data. Attendees are encouraged to click "ATTEND" to confirm their participation and can look forward to networking opportunities with drinks and burgers following the talk.

When implementing a new feature or switching libraries, application performance can suffer, as experienced by the team at Memgraph when transitioning their Bolt server to use Boost.Asio. Initially, their handwritten Bolt server faced performance issues after they shifted to support WebSocket, a decision driven by the increasing popularity of WebSocket and the introduction of a monitoring server via WebSocket in Memgraph 2.2. Despite successfully passing tests, performance benchmarks revealed significant degradation, prompting the team to use profilers like Callgrind and gperftools to diagnose the issue. While Callgrind provided detailed call statistics, it failed to pinpoint the slowdown, leading them to gperftools, which revealed that the send function was unusually slow due to the absence of the MSG_MORE flag, causing every call to send an actual package over the network. Correcting this oversight by utilizing boost::asio::basic_stream_socket::send instead of boost::asio::write restored performance closer to original levels, highlighting the importance of understanding both networking intricacies and profiling tools. Through this process, the team learned valuable lessons about package handling, the significance of correctly instrumenting POSIX sockets, and the complementary use of different profilers to diagnose performance issues effectively.

At the third edition of the Graph Data Zagreb Meetup hosted by Memgraph on May 4, 2022, over 60 participants gathered to explore graph data applications, including a presentation by Katarina Šupe focusing on a Game of Thrones dataset. Šupe demonstrated how graph algorithms and traversals can identify and visualize character connections and deaths within the series. Attendees engaged in discussions over burgers and craft beer, celebrating Star Wars Day as well. The event also announced the next meetup on June 8, featuring Boris Agatić, who will discuss blockchain's relevance to graph data. Participants are encouraged to join the Graph Data Zagreb community to stay informed about future events and topics.

Memgraph 2.3 enhances its graph database capabilities by allowing developers to connect it to more application stacks, facilitating faster application development. The release introduces user-defined functions in C/C++, Python, or Rust, which can be used in Cypher expressions without altering the graph, offering performance optimization and flexibility, especially for complex filter expressions. The implementation of Bolt over WebSocket enables querying Memgraph directly from web browsers or WebAssembly clients, enhancing interoperability with cloud environments. Although there is a slight performance drop in high-throughput scenarios, the update makes the communication stack more robust and cross-platform. Additionally, the FOREACH clause is introduced as a procedural tool within the Cypher environment to enable more sophisticated queries. While some bugs were fixed in this release, Memgraph anticipates further improvements and major bug fixes in future versions.

Kostas Kyrimis from Memgraph's Core Team introduces a three-part series exploring the integration of WebAssembly into Memgraph's architecture, specifically focusing on client adapters. The motivation behind this exploration stems from the complexities involved in implementing client adapter libraries, which traditionally require either reimplementation of networking protocols for each programming language or reliance on specific bindings APIs, each with its unique challenges. Kyrimis highlights the limitations of using tools like SWIG for code generation, as they often complicate debugging. Instead, Memgraph aims to leverage WebAssembly as a platform-independent intermediate representation, compiling the C client library into a WebAssembly module to facilitate a universal runtime through WASMER. This approach centralizes changes within the C implementation, reducing cognitive overhead and isolating bugs within the networking stack, while eliminating the need for understanding diverse binding libraries. Kyrimis teases a deeper exploration of WebAssembly in the subsequent articles, culminating in the development of Memgraph's first WASM-based client adapter library for JavaScript, the jsmgclient.

Kostas Kyrimis details the creation and development of jsmgclient, a WebAssembly (WASM)-based JavaScript client adapter for Memgraph, as part of his WebAssembly series. Utilizing Emscripten for its smooth integration with JavaScript and CMake, Kyrimis made WASM a compilation target for mgclient, enabling the creation of a WASM module that facilitates communication with Memgraph's database via the Bolt protocol. Despite initial challenges with asynchronous networking calls and Memgraph's lack of Websocket support, Kyrimis adapted the networking stack to be asynchronous, ultimately achieving a successful connection and running a "Hello world" example. He highlights the ease of implementing JavaScript wrappers for the WASM module and suggests the potential for generating wrappers in multiple languages using tools like wasmer, hinting at future innovations in database client libraries. The project concluded with the successful launch of the first WASM-based client for Memgraph, as documented in the jsmgclient repository.

Kostas Kyrimis delves into the intricacies of the WebAssembly (WASM) ecosystem, highlighting the process of compiling C libraries into WASM modules using the Emscripten SDK, which transforms C programs into modules consumable by browsers or JavaScript engines. The article addresses the challenge posed by the restricted environment of browsers, particularly the limited access to system interfaces like POSIX, and discusses how Emscripten emulates system libraries through JavaScript files, thereby constraining the modules to JavaScript's limitations. To overcome these restrictions, the WebAssembly System Interface (WASI) is introduced, offering a complete sandbox environment that allows for standalone WASM modules to be executed on any WASI-compliant runtime, thus providing seamless interoperability across different execution environments. The author also touches on the potential for Emscripten to fully support WASI in the future and introduces Wasmer, a WASM runtime supporting WASI and available in multiple programming languages, which could unify runtime semantics across various languages. The discussion sets the stage for exploring jsmgclient, Memgraph's first WASM-based client adapter, which will be covered in the subsequent part of the series.