Using New Relic Observability to reduce our Redis costs by 50%

New Relic faced challenges with increased costs and inefficiencies in its distributed tracing pipeline, primarily due to the high-performance demands of processing massive telemetry data streams. The team embarked on an optimization journey, first by upgrading and right-sizing Redis clusters to improve infrastructure efficiency, which led to initial cost savings and better resource utilization. They identified further opportunities by rethinking data storage with Redis batching, which involved storing and compressing batches of spans instead of individual spans. This approach significantly improved compression efficiency, reduced Redis memory and bandwidth usage by 66%, and halved annual costs without compromising system reliability or performance. The experience highlighted the importance of combining infrastructure tuning with smart data storage strategies and demonstrated the potential of observability for identifying and validating improvements in large-scale systems.