Elastic on Elastic: How we saved $100,000/month by keeping our own software up to date

Upgrading to Elasticsearch 7.15 led to a significant reduction in inter-node traffic for Elastic's internal observability clusters, decreasing from 464TB to 204.5TB per day, and saving the company $100,000 monthly or $1.2 million annually in data transfer and storage costs. The improvements were achieved through the implementation of network data compression using the lz4 algorithm and new data relocation and recovery methods that reduce cross-zone data transfer costs. Elastic's observability clusters, which span 207 production clusters across four cloud providers, benefit from these innovations by maintaining efficient data management and monitoring capabilities while minimizing expenses. The upgrade not only enhances Elastic's internal operations but also presents potential savings for its customers by lowering the total cost of ownership.