How We Scaled Private Link Routing Across AWS, Azure, and GCP Using Envoy and Wasm

Managing cloud infrastructure at scale often leads to challenges with Private Links like AWS PrivateLink, Azure Private Link, and GCP Private Service Connect due to high infrastructure costs and strict cloud provider limits. To address these issues, a shared Layer 4 routing tier was developed, allowing multiple private links to be routed through a single Network Load Balancer (NLB) and an Envoy proxy using a custom C++ WebAssembly plugin. This plugin dynamically routes connections based on PROXY Protocol v2 headers containing unique customer VPC Endpoint IDs, which are differently injected by each cloud provider. The approach necessitated upstream contributions to the Envoy project to overcome limitations in its Wasm API and required a shift from Golang to C++ for better memory management. This method significantly reduced infrastructure costs and simplified the control plane by managing connections through Kubernetes services, though it requires careful orchestration of connection terminations to avoid service disruptions. Despite the complexities introduced by managing an intermediary proxy layer, this innovative solution provides a scalable and cost-effective alternative to traditional 1:1 infrastructure mappings, especially beneficial for those facing scaling limitations with existing private link services.