A New Era for New Relic’s Lambda Extension: From Go to Rust

The New Relic team undertook a significant rewrite of their Lambda Extension, transitioning the codebase from Go to Rust, which resulted in considerable performance improvements and enhanced reliability. By moving to Rust, they achieved a 40% reduction in billed duration and a 13% decrease in memory usage, thanks to the language's zero-cost abstractions and lack of a garbage collector. Rust's strict ownership system prevents common bugs, making it more reliable, while also enabling the use of asynchronous runtimes and connection pooling for better performance. The architectural overhaul included using non-blocking I/O, concurrent data structures, and a sophisticated telemetry pipeline with retry mechanisms, which contributed to reduced data loss by 95%. The transition to Rust presented challenges such as a steeper learning curve and a smaller ecosystem, but the benefits of increased efficiency and safety justified the effort, providing up to 40% faster performance, reduced costs, and zero customer Lambda failures.