High-performance GPU memory transfer on AWS Sagemaker Hyperpod

Perplexity developed a custom RDMA-based networking system for AWS p5 instances to transfer non-contiguous GPU memory regions between machines while supporting dynamic Kubernetes membership and peer-to-peer communication, requirements that did not fit NCCL’s static, collective-oriented model. Using AWS Elastic Fabric Adapter, the Scalable Reliable Datagram protocol, libfabric, GPUDirect RDMA, two-sided SEND/RECV control messages, and one-sided RDMA WRITE data transfers, the system bypasses CPU and kernel data copies for direct GPU-to-network-card communication. Performance depended on managing application-owned registered buffers, pre-posted receives, polling completion queues, and awareness of the instance’s NUMA and PCIe topology, which links H100 GPUs and four 100 Gbps network cards beneath each PCIe switch. After adding queuing, connection warmup, dedicated pinned threads, state sharding, batching, lazy posting, and NUMA-aware resource allocation, the implementation reached 3,108 Gbps across 32 network cards, or 97.1% of the platform’s theoretical 3,200 Gbps bandwidth.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.