How We Power Box Logs with Upstash Redis Search

Box faced challenges with log management, initially relying on Redis lists which were efficient for ordered reads but lacked search capabilities, making it difficult to filter logs at scale. The solution involved implementing a dual-write system where each log entry is written both to the Redis list and asynchronously to an Upstash Redis Search index as a JSON document. This approach allows for efficient log retrieval and filtering by time range, level, and source, and enables full-text search across all logs without overhauling the existing storage infrastructure. By using Upstash Redis Search, Box leverages a single database to maintain ordered storage and search functionalities without additional services, ensuring that both storage types remain synchronized and accessible for rebuilding indexes if necessary.