Throwing Out The Phonebook

Fly.io rebuilt its 6PN private IPv6 networking system to replace a fleet-wide, eBPF-based “phonebook” of migrated Machine locations with an on-demand routing model. The earlier design encoded original host information in Machine addresses and relied on address translation plus a synchronized map containing hundreds of thousands of entries on every host, creating scaling, observability, debugging, and consistency problems. The replacement uses VXLAN over the existing WireGuard mesh and lets hosts request a route only when they first need to reach an unknown 6PN address; a userspace daemon queries the local Corrosion-replicated SQLite database, installs a route, and periodically refreshes it. This reduced typical per-host route tables to roughly 100–200 entries, removed address rewriting, improved monitoring, and allows stale routes or degraded links to be handled through forwarding and extra hops. The design retains eBPF only for network-isolation validation and fails closed if validation is unavailable, though initial connections between previously unconnected Machines may incur about a one-second TCP retransmission delay. Fly.io migrated gradually by running the old and new paths simultaneously, enabling hosts individually, and ultimately removed the prior swapping-based system, while gaining a simpler foundation for broader network-level rerouting.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.