RDMA point-to-point communication for LLM systems

TransferEngine is a Rust-based, open-source RDMA communication library developed by Perplexity AI to support flexible point-to-point communication for emerging LLM workloads such as disaggregated inference, Mixture-of-Experts routing, and asynchronous reinforcement-learning post-training, where conventional collective communication frameworks can be restrictive. Designed to operate across NVIDIA ConnectX-7 and AWS Elastic Fabric Adapter hardware, it assumes reliable but potentially unordered transport, uses an ImmCounter mechanism for order-independent completion notifications, and transparently distributes transfers across multiple NICs. Its compact API supports registered memory, two-sided control messaging, zero-copy contiguous and paged writes, scatter and barrier operations for peer groups, and UVM Watchers that coordinate GPU computation with CPU-initiated RDMA transfers. Perplexity reports using the library to independently scale prefill and decode services through layer-by-layer KV-cache transfers, synchronize model weights for RL post-training in under two seconds, and enable low-latency trillion-parameter MoE operations on AWS EFA.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.