September 2026 Summaries

WebSockets are presented as a stronger transport than Server-Sent Events for AI applications that require real-time client input during streaming, such as cancelling responses, approving tool calls, redirecting agents, or synchronizing sessions across devices. Unlike SSE’s server-to-client model, WebSockets provide bidirectional, ordered communication with native binary support and lower per-message overhead, although they do not inherently provide reconnection, cross-device fan-out, state recovery, or exactly-once delivery. Self-hosted tools such as Socket.IO and Centrifugo can supply some reconnection and pub/sub capabilities but still require teams to operate brokers, clusters, security updates, scaling, compliance, and on-call support, while delivery guarantees remain an application responsibility. The piece argues that managed realtime platforms can shift much of this operational work to a provider through service-level commitments for availability, ordering, delivery, failover, and compliance, highlighting Ably as an example. It also notes that OpenAI and Anthropic agent-session products manage model execution but do not solve last-mile delivery to end-user devices after disconnects. SSE may remain suitable for simple, single-turn streaming interfaces, while self-managed WebSockets may fit small deployments or organizations with existing realtime infrastructure; more complex agent workflows increase the operational case for managed services.

WebSocket message ordering is provided by TCP only within a single open connection, so production systems can exhibit apparent reordering after reconnects, across load-balanced backend instances, during simultaneous replay and live delivery, or when multiple processes publish to the same stream. Application-level asynchronous handlers can also reorder otherwise correctly delivered messages when awaited work completes out of sequence, which can be addressed by serial processing through a queue. Maintaining ordering across distributed infrastructure requires ongoing measures such as sticky routing, shared coordination layers, and replay sequencing, rather than a one-time code fix; the text notes that concurrent publishers are a common explanation for apparently disordered LLM token streams. It suggests comparing the engineering and operational costs of self-managed realtime infrastructure with managed platforms, and presents Ably as an example offering per-publisher, per-channel ordering, automatic recovery and ordered replay, while emphasizing that teams must still define and coordinate what constitutes a single logical publisher.

Raw WebSockets provide efficient, bidirectional connections for AI streaming but do not inherently support session recovery, targeted cancellation, multi-device message delivery, or reliable detection of agent crashes. Building these capabilities requires shared persistent state, idempotent replay, run-level identifiers, authorization controls, connection registries, fan-out mechanisms, and agent lifecycle reporting; common production stresses such as deploy-driven reconnect storms, mobile handoffs, concurrent cancellations, and device switching can expose weaknesses in in-house implementations. The text argues that teams should distinguish isolated new failures, which may be ordinary bugs, from recurring failures under expected conditions, which may indicate an underlying infrastructure gap. It cites survey figures suggesting substantial development, maintenance, and outage costs for self-managed WebSocket infrastructure, alongside Intercom Fin’s move from its in-house realtime system to a managed platform. Ably presents its AI Transport as a managed layer that retains WebSockets or SSE while adding resumable delivery, run-specific cancellation, native multi-device synchronization, and presence-based agent status, though product-level user experience decisions remain the application’s responsibility.