Networking Is a Black Box, We Used eBPF to Open It
Blog post from Railway
Phin Walton, a network engineer at Railway, has introduced Network Flows, a feature designed to demystify networking for application developers by providing real-time traffic visualization between services. This tool aims to simplify network debugging by offering a visual representation akin to pipes on a canvas, allowing developers to see service connections and throughput without resorting to traditional, complex methods like tcpdump. The feature was developed in response to the high volume of support tickets related to perceived networking issues, which often turned out to be application problems rather than actual network faults. By leveraging eBPF and ClickHouse for efficient data processing, Network Flows enriches raw socket data to provide meaningful insights into network flow, helping developers identify and resolve issues more efficiently. This initiative reflects Railway's broader mission to abstract lower-layer networking concepts for developers while still offering detailed data for those who need deeper insights, aiming to bridge the knowledge gap in networking among application developers.