August 2017 Summaries

This announcement marks a significant release of Redis Enterprise Pack version 5.0, which promises improved high availability, better scaling performance, advanced security features, and real-time search capabilities at Redis speeds. The new release introduces Geo-distributed, Active-Active Redis Applications with Conflict-Free Replicated Databases (CRDBs), enabling sub-millisecond latencies for globally distributed applications. Additionally, the new RediSearch module provides high-performance integrated query and full-text search over efficient in-memory indexes, while the ReJSON Module offers native JSON data handling. Other notable features include native JSON processing, Redis 4.0 compatibility, fast "Existence" Search with Bloom Filters, and integrated account management with LDAP. The preview program is expected to start around September 11th, 2017, and will run for a few weeks, offering developers the opportunity to provide feedback on the new capabilities and be part of the inner circle with engineering teams.

The Redis-ML module provides matrix operations for processing Matrix data using the Redis database, adding matrices as a native Redis data type and supporting mathematical operations like multiplication and addition. It allows users to read and write matrix values through specific commands, such as ML.MATRIX.SET and ML.MATRIX.GET, which use row-major format. The module can perform basic matrix equations like y = Ax + b by combining multiple matrices using commands like ML.MATRIX.MULTIPLY and ML.MATRIX.ADD. Additionally, it enables users to scale a matrix using the ML.MATRIX.SCALE command, resulting in a new matrix with the same dimensions but scaled values. Matrices are used for various applications including linear transforms and multivariate probability distributions, making Redis-ML a valuable tool for data processing and analysis.

This post introduces logistic regression as a linear model for building predictive models from observed data, specifically designed to predict binary values. It discusses how logistic regression can be used in conjunction with Redis using the Redis-ML module, which provides ML.LOGREG.SET and ML.LOGREG.PREDICT functions to create and evaluate logistic regression keys. The post demonstrates how to perform a logistic regression on the Fisher Iris Plant Data Set and uses Redis to emulate the One vs. Rest procedure for multiclass classification. It shows that using Redis can provide a highly available, real-time classifier for data, and will explore matrix operations supported by Redis-ML in the next post.

We've been working on video walk-throughs of common situations with Redis Enterprise. In the first video, we discussed replication between different clusters, a feature useful for disaster recovery, read performance boost, geo-located databases, and testing or reporting purposes. Replication can be done entirely from the Redis Enterprise UI in a point-and-click operation. The second video focused on high availability setup techniques, introducing core features like sharding and clustering, and showcasing high-performance capabilities, including hundreds of thousands of operations per second. We also demonstrated regional replication with short clicks, allowing users to replicate between shards within the same cluster.

This post discusses linear regression and its implementation using Redis-ML module. It revisits the housing price predictor example from a previous post, but uses R instead of Python to perform the linear regression. The code demonstrates how different toolkits can be used in conjunction with Redis, emphasizing that the model is more important than the toolkit when using Redis for serving models. Additionally, it explains multiple linear regression and how to set up a key in Redis to store the represented multiple linear regression. The next post will cover logistic regression and building a classification engine with Redis.

Redis is a Data Structure Store that allows dealing with Key-Value Pairs as well as data structures such as Hashes, Lists, Sets, and Sorted Sets. It leverages the power of these data structures to support operations like exact match queries through hashes and range queries through sorted sets. Redis also supports Modules, including RediSearch for full-text indexing and searching capabilities. The complexity of accessing an entry in a hash or a sorted set is O(1), while the complexity of scanning a sorted set is O(log(n)+m).

Intel has announced the availability of Optane, an NVMe-based SSD device built on top of the new persistent memory technology from Intel and Micron, 3D XPoint. Redis Enterprise Flash (RF) product was built with a similar approach in mind. The RF benchmark using Optane drives showed impressive results compared to the previous generation of Intel’s NVMe-based SSD, the P3700. RF maintains performance similar to that of RAM while dramatically reducing server infrastructure costs. In tests on AWS, the new Intel Optane SSD delivered more than nine times the throughput using Redis Enterprise Flash.

The Redis search engine module, RediSearch, aims to provide fast and feature-rich real-time search capabilities while maintaining the simplicity of Redis' single-threaded architecture. However, this simplicity comes with a cost: long-running commands can block the entire server for extended periods. To address this challenge, Salvatore Sanfilippo introduced Thread Safe Contexts and the Global Lock in Redis 4.0, allowing modules to run multiple threads while acquiring the lock when accessing or modifying data. This enables concurrency without sacrificing atomicity of reads and writes, although some limitations apply, particularly with high update/delete loads. The new design allows for concurrent execution of queries and indexing, improving performance and making RediSearch a more powerful search engine suitable for large datasets and high loads.