January 2019 Summaries

DigitalRoute's Usage Data Platform (UDP) aggregates data from diverse systems and locations, analyzing and acting upon it in real-time. The platform processes 300 billion transactions per day and provides real-time insights to improve user experience. A new trace feature was introduced to track the path of records through a customer's system, requiring a robust back-end capable of efficient storage and fast presentation. Redis' pub/sub messaging capabilities were used to improve communication between services, providing simplicity and speed. The use of Redis Streams is believed to be an even better fit for the platform's use case due to its built-in data persistence and many-to-many data channel capabilities.

This application allows users to track when materials are "dry" or cured, using Node.js and Redis Enterprise Cloud. It provides two main operations: `paint` to add a coat of paint to a room, which sets a timer for the material to dry, and `readytopaint` to check if it's safe to apply another coat. The application uses Redis's set data type to track rooms and their associated timers, allowing for efficient management of multiple users and coats. The `paint` operation takes a user ID, room name, and duration as arguments, while the `readytopaint` operation takes only the user ID and room name. The application also includes help text and error handling using yargs. With a small JSON file providing connection options, the application can be run from the command line to track paint drying times.

Redis supports modules, which are custom extensions that add new commands, data types, and functionalities to a Redis database. Writing a module can be done using C, but this requires setting up a new C project and dealing with cross-platform compilation issues. Zig, a new language developed by Andrew Kelley, is a more modern alternative that allows writing fully C ABI compatible binaries while providing additional safety features such as advanced error checking and generics. Zig makes it easy to write Redis modules by allowing the import of C header files directly or translating them to Zig using the `translate-c` command. The translation process involves changing type signatures to spare unnecessary optional unwrapping, and the resulting code is straightforward and easy to read. With Zig, building a Redis module can be done with minimal effort, even cross-compilation for 64-bit Linux is supported in a single command.

We recently published benchmarking results for RedisGraph and responded to criticism from TigerGraph regarding the accuracy of our findings. We initially introduced a parallel requests benchmark to better represent real-world scenarios, but incorrectly extrapolated benchmark times for TigerGraph. To correct this, we re-executed the full benchmark against TigerGraph, revealing that RedisGraph is 5-15 times faster than TigerGraph in most cases, with the biggest difference seen in one-hop queries. Our new results also show that average query times are a more accurate metric for this test, with RedisGraph's average query times being 5-20 times faster than TigerGraph under parallel load. We're making our modified benchmarking code available to everyone and documenting known issues to increase transparency and accuracy.

Redis offers several approaches for creating content filters, including using Redis Sets, SISMEMBER, BF.MEXISTS, and Bloom filters. The choice of approach depends on the desired trade-off between response time and storage efficiency. In this case, a Bloom filter was initially thought to be the fastest solution, but it was found that plain old SISMEMBER ran faster in all tests. However, Bloom filters offer significant storage efficiency advantages, making them suitable for large-scale applications where storage is a concern. The results of benchmarking tests showed that while response times were similar between Bloom filters and Redis Sets, Bloom filters were much smaller in terms of memory usage, which can be a critical factor in high-traffic applications.