March 2022 Summaries

This post delves into the exciting world of probabilistic data structures and Redis modules. It highlights that large-scale low-latency data processing is challenging, but Redis' high performance and versatility make it ideal for solving such challenges. The Count-Min Sketch (CMS) is a probabilistic data structure that sacrifices some accuracy to gain space, making it useful in scenarios where reduced accuracy is acceptable. CMS works by aggregating the count of all items in your dataset into several counter arrays and returning the item's minimum count upon query. It has been a Redis module for several years and was recently rewritten as part of the RedisBloom module v2.0. The post also discusses other probabilistic data structures, such as HyperLogLog, and their applications in high-scale low-latency data processing.

RedisDays 2022`, a three-stop virtual tour, concluded with a focus on artificial intelligence/machine learning (AI/ML) in real-time data in New York. The event featured keynotes and sessions on AI/ML innovation in financial services, operational machine learning, overcoming challenges of microservices architecture, and a fireside chat between Redis' CMO and Daymond John. Attendees were entered into a raffle for Oculus Quest 2 VR systems and Nintendo Switches, as well as eligible to win gift cards after completing an event survey. The sessions showcased the power of Redis in driving AI/ML innovation with real-time data, including vector similarity search and MLOps lifecycle support.

The RedisBank application is an online banking platform that utilizes Redis, a fast and accurate data store, to provide real-time access to financial information. The app allows users to monitor and plan their finances accurately by using Redis core data structures and modules, such as Redis Streams, Sorted Set, TimeSeries, and Hashes. The application also uses Spring Boot APIs and a frontend, and is built on top of the Redis module for Spring. The app generates bank transactions every 10 seconds or so and puts them in a Stream, which is then processed by the app and sent to the user interface. It also adds every transaction to a Sorted Set, allowing users to identify the biggest spenders. Additionally, the app uses TimeSeries to store bank account balances and Hashes to index transactions for search functionality. The application provides real-time data transmission, eliminating lags, and allows users to access their banking information with just a few taps on their mobile device.

Redis OSS and Dynomite are peer-to-peer geo-distribution layers for non-distributed datastores. While Redis OSS has an open-source community, Dynomite's lack of maintenance and limited functionality compared to Redis OSS have led organizations to migrate to Redis Enterprise, which offers a more scalable and maintainable solution with features like multi-model operations, conflict-free replicated data types, and high availability mechanisms that guarantee 99.99% uptime and 99.999% uptime for Active-Active deployments. In contrast to Dynomite's AP system, which requires manual failover and conflict resolution strategies, Redis Enterprise provides automatic failover and causal consistency through its CRDTs implementation. Additionally, Redis Enterprise offers multi-tenancy, scalability, and deployment options that are more flexible than Dynomite's self-managed deployments within AWS autoscaling groups or containerized environments.

The Data Economy is a video podcast series that explores how leaders use data to make positive impacts on their business, customers, and the world. CIOs and CDOs must innovate ahead of the business need by leveraging emerging capabilities such as artificial intelligence, machine learning, and IoT, while working with diverse data sources and enabling real-time experiences. According to Ron Guerrier, HP's CIO, innovation requires finding new markets, identifying different ways of working and servicing customers, and understanding customer needs through metadata or other means. He emphasizes the importance of data diversity, capturing and processing real-time data, and democratizing data to provide personalized, real-time experiences. To succeed, CIOs must partner with business leaders, build diverse teams, and create a learning culture that fosters innovation and different perspectives.

RedisDays, a three-part worldwide virtual event, kicked off with a stop in London where Redis experts and guests discussed the power of sub-millisecond speed and real-time data. The event highlighted how many people use real-time data without noticing it, which is used daily to improve customer experience across various applications and industries such as manufacturing, healthcare, and online advertising. Key takeaways from the event included best practices for modernizing data architectures hampered by legacy technologies, product announcements, and new use cases for real-time data. The event also featured sessions on building real-time natural language processing data pipelines using Redis, showcasing its capabilities in transforming financial services processes and improving customer experience.

Redis Stack is a unified product that consolidates the capabilities of leading Redis modules into a single solution for building modern, real-time applications with the speed and stability of Redis. It offers a suite of three components: Redis Stack Server, RedisInsight, and object mapping libraries for several programming languages. The product unifies the capabilities of Redis search, document, graph, time series, and probabilistic data structures into a single package that includes open source Redis, making it easy for developers to start building with these advanced features from the moment they start using Redis. Redis Stack is not a replacement for Redis, but rather an extension of its capabilities, and it will continue to be developed as an open-source project, supporting one of the largest developer communities on earth.

The Redis team has relaunched Redis.io with a modernized design, updated infrastructure, and revised documentation principles. The new site is designed to better serve the Redis community and improve the user experience for developers building with Redis. The team has also introduced Redis Stack on redis.io, which unites various Redis modules under a single product licensed under the Redis Source Available License (RSAL). The site now clearly distinguishes between open source and RSAL-licensed projects, aiming to effectively sponsor and promote open source Redis while providing an onramp to the RSAL-licensed Redis Stack.

The Data Economy` is a video podcast series about leaders who use data to make positive impacts on their business, customers, and the world. To collaborate with technical and business colleagues, identify legacy application issues such as slow "batch world" processing limitations, frontend user-experience issues, architecture deficiencies, and data storage concerns. Real-time streaming flow and analytics can be achieved by changing from batch processing to real-time paradigms, simplifying app development with a robust multi-model data platform. This shift can create differentiating capabilities, making quicker and more accurate predictions and aiding in real-time decision-making. Architects must design real-time capabilities while considering technical challenges such as multicloud, microservices, in-memory databases, caching solutions, and reference data storage. The key business technology challenge is creating versatile, real-time data processing architecture that serves multiple business needs, requiring ongoing experimentation with machine learning and real-time analytics to develop algorithms and deliver applications.

ABettaMe, a healthy lifestyle app, uses Redis as its backend to measure user progress and analyze experiments. The application is built using various technologies such as TypeScript, React for the UI interface, Java and Spring Cloud Gateway for the application gateway, C# and .NET 5 for the recommendation service, and Google Protocol Buffers for shared contracts. ABettaMe allows users to create new experiments, log metrics, view progress graphs, and track milestones, providing a seamless experience through Redis' advanced data processing capabilities. The app is built using Docker and can be easily deployed and managed. It also provides access to the Redis Launchpad, which features innovative applications that are making an impact on everyday life.

Redis Enterprise is an enterprise-ready version of Redis, which is an in-memory database, providing management, automation, resiliency, and security capabilities to make it suitable for large-scale businesses. It helps legacy applications and databases scale by front-facing them as an enterprise cache, modernizing NoSQL and SQL databases, and simplifying the tech stack, leading to cost savings. Additionally, Redis Enterprise enables faster time-to-market, improves digital income generation, avoids penalties in regulated industries, and alleviates DevOps teams' workload, resulting in significant financial benefits, including a 350% ROI over three years and a payback period of less than six months.

Redis can be installed on Windows 10 through various methods, including downloading precompiled versions from GitHub or compiling Redis from source using Microsoft's official port. Additionally, users can leverage the Windows Subsystem for Linux (WSL2) to run a GNU/Linux environment directly on Windows and install Redis there. The WSL2 method offers several benefits, including no need for virtual machines or Docker. To enable WSL2, users must open PowerShell as Administrator and run the command `Enable-WindowsOptionalFeature -Online -FeatureName Microsoft-Windows-Subsystem-Linux`. After enabling WSL2, users can download and install Ubuntu or another preferred Linux distribution, then proceed with installing Redis using apt-get commands. Once Redis is installed, users can use Python to interact with it. However, the primary drawback of running Redis on Windows is that Windows isn't officially supported by Redis, which can lead to performance issues, particularly with database saving methods.

Active-Active Geo-Distribution` allows for placing Redis database cluster instances and centers close to users, enabling real-time responses. This is achieved through `CRDTs` (Common Replicated Data Types), which are a family of replicated types with specific properties that ensure operations converge to a final state among all replicas. CRDTs work by swapping standard commands with equivalent ones, allowing nodes to independently apply changes and enable geo-distributed applications. The approach offers significant advantages for geo-distributed applications, including achieving high availability and resilience in case of network partitions. However, CRDTs have limitations, such as not being suitable for inherently transactional problems like bank account balances, and require careful consideration when testing Active-Active Geo-distributed Applications.

Music enthusiast Franco Chen developed an application that allows users to listen, share and discover new music with their friends irrespective of their location. The app uses Redis for real-time data transmission, processing, and retrieval, ensuring hyper-responsive commands and interactions between users. It creates private or public online rooms where people can invite others to join in the experience, fostering deeper engagement with different genres, tastes, and preferences through interaction with music lovers worldwide. The app also features a chat window for sharing opinions on each song and a feature that skips songs if over half of the room dislikes them.

The Data Economy is a video podcast series that explores how leaders use data to make positive impacts on their businesses, customers, and the world. Experian, one of the three major U.S. credit bureaus, shares its experiences with big data, real-time data processing, and managing personal data. Eric Haller, EVP & GM in Identity, Fraud & DataLabs at Experian, discusses how his organization uses detection algorithms to pinpoint fraudulent transactions, analyzes over an exabyte of data, and processes data in under 200 milliseconds for retail fraud detection. He also emphasizes the importance of finding new data sources, signals, and machine learning features to improve accuracy and performance. The podcast highlights Eric's insights on optimizing multicloud architectures, leveraging graph databases, and centralizing multi-business-unit data sources, as well as his advice to developers and data scientists to focus on solving business problems first and then seek technology partners and solutions that enable efficient implementations.

RedisDays is a three-part virtual roadshow that highlights the latest real-time data innovations at Redis, making its first US stop in San Francisco on March 23rd. The event will focus exclusively on developers and feature live demonstrations, fireside chats, and practical workshops on building Redis-based apps with newly introduced tools. Attendees are eligible to win Oculus Quest 2 VR systems and Nintendo Switches as well as gift cards by completing an event survey. Redis Co-Founder Yiftach Shoolman and other guests will share the latest developments in Redis evolution, including the recently released Redis 7.0, which boasts new capabilities such as Redis functions, ACLs, scalable Pub/Sub, and more. The event aims to help developers pivot to a streamlined real-time database platform by providing flexible data types and tools necessary for building faster applications. Additionally, there will be fireside chats on topics like harnessing the power of Redis in unique shopping experiences and how open source drives innovation in the beauty industry.

Financial institutions face challenges in today's digital era, as customers demand seamless omnichannel banking experiences. Real-time data plays a crucial role in meeting these expectations and surpassing consumer and business expectations. Banks must adopt cloud and microservices architectures to leverage real-time data and maximize the user experience. A unified real-time data layer can provide banks with speed, flexibility, and innovation capabilities while transitioning from legacy IT systems. Real-time analytics is essential for financial institutions to stay competitive in today's fast-paced market environment.

Tecton's Feature Store is designed to handle the steps required for productionizing an ML use case like fraud detection by abstracting away data pipeline work, allowing data science and data engineering teams to focus on building models. To support this, Tecton integrates with Redis Enterprise Cloud, a high-performance database that delivers best-in-class performance, scalability, and cost-effectiveness across cloud vendors. This integration enables real-time machine learning applications by providing ultra-low-latency inference capabilities and cost-effective solutions for high-scale ML applications. With the integration, Tecton users can take advantage of Redis Enterprise's 3x faster latency and 14x lower costs compared to Amazon DynamoDB, while also benefiting from its operational capabilities such as high availability, multiple database persistence, backup, and recovery options.

The Data Economy podcast is a thought leadership destination where Chief Information Officers (CIO), Chief Technology Officers (CTO), and IT Executives share their knowledge, strategies, and use-cases on leveraging data to work smarter, not harder. The podcast explores how organizations are using data in innovative ways to increase business value, overcome infrastructure constraints, and accelerate business growth. It features interviews with industry leaders discussing topics such as fraud detection algorithms, real-time streaming flow, data diversity, and digital transformation in various industries including retail, logistics, and cybersecurity. The podcast is presented by Redis and hosted by Michael Krigsman, an Industry Analyst and Publisher of CXOTalk.

RedisDays is a three-part virtual roadshow featuring informative sessions with members of the Redis community, including customers and data experts, highlighting the products and use cases that power modern digital businesses. The event kicks off in London on March 15th, followed by stops in San Francisco and New York City. Attendees can expect to learn about how Redis technology is transforming enterprise and helping leaders meet pressing demands, as well as hear from industry experts on best practices for building real-time data pipelines and multi-model real-time databases. The event will also cover the use of RedisJSON document store to improve application performance and explore the possibilities of building modern data architectures with legacy technologies in the financial services industry.

Redis 7.0 is nearing its general availability with a second release candidate available now. The release includes significant improvements to the Access Control List (ACL), allowing for more flexible and secure access control, as well as enhancements to the server's introspective abilities, including improved documentation and command management mechanisms. These changes bring Redis closer to completeness and provide clients with more information about executing commands, improving latency and reducing network bandwidth. The release also promotes subcommands to first-class citizens of the server's commands table, enabling more fine-grained distinctions between their behaviors.