March 2026 Summaries

In the rapidly evolving field of AI agent development, starting with HTTP streaming is common, but as products mature, teams often encounter limitations that hinder performance and user experience. This progression is mapped onto a maturity curve, highlighting stages from simple prompt-response interactions to more complex, production-grade AI experiences. As AI products develop, they require a session layer to manage longer sessions involving multi-step workflows, real-time collaboration, and seamless agent-to-human handoffs. These demands often necessitate infrastructure beyond HTTP streaming, such as durable transport, collaborative state management, and bidirectional communication capabilities, which can be provided by solutions like Ably AI Transport. This shift allows engineering teams to focus on product features rather than transport reliability, ensuring that AI applications can scale effectively while maintaining a high-quality user experience.

Matthew O'Riordan discusses the challenges AI teams face due to the absence of a dedicated session layer between agent frameworks and user devices, which affects the stateful interaction necessary for agentic AI applications. Current solutions like HTTP streaming and SSE are insufficient for maintaining long-running, bidirectional sessions, leading to fragile connections that can't recover from network disruptions and lack mechanisms for user interaction or human handover. O'Riordan highlights how AI companies like Intercom and HubSpot have had to build custom workarounds, such as message buffers and state serialization, to address these issues, but these efforts often lead to complexity and inefficiencies. The emerging solution is a "durable session" layer, which provides persistent, addressable connections that support multi-device continuity, agent visibility, and seamless human-AI coordination. Companies like Ably are developing infrastructure to address these gaps, offering a more reliable foundation for AI applications that allows engineering teams to focus on higher-value tasks rather than maintaining session infrastructure.

AI support systems in customer service face significant challenges due to the limitations of HTTP streaming, which is not designed for sessions that persist through changes in participants or devices. Two main failures, session continuity and efficient human escalation, often occur because the existing infrastructure ties session state to a single connection, which doesn't accommodate the multi-device, multi-participant nature of customer support. These failures are frequently misdiagnosed as issues of state persistence or workflow but are rooted in an architectural constraint where each participant requires a separate connection with no shared session. The solution lies in decoupling session state from individual connections and creating durable sessions that any authenticated participant can access, thereby allowing seamless transitions across devices and participants. This concept of durable sessions is gaining traction as a necessary architectural evolution in customer experience engineering, enabling more reliable and efficient support interactions without necessitating extensive custom infrastructure.

LiveObjects, a feature from Ably, addresses the challenge of managing shared state in real-time applications by eliminating the need for infrastructure-heavy solutions to problems like conflict resolution and state recovery. Initially released as an experimental API, LiveObjects has evolved into a production-ready solution with a redesigned API based on user feedback, featuring path-based operations that enhance flexibility and ease of use. This new API allows developers to interact with data paths instead of specific object instances, providing seamless updates even when objects are replaced. LiveObjects supports features like object resets, reliable data expiry, and revised object limits, making it ideal for applications such as live polls, leaderboards, and collaborative forms. It's particularly beneficial for AI applications that require real-time state synchronization across multiple clients, ensuring consistency and automatic conflict resolution. Currently available in JavaScript, with Swift and Java SDKs forthcoming, LiveObjects aims to streamline the development of real-time features without the overhead of managing a state coordination layer.

AI applications face challenges in maintaining seamless user experiences due to the limitations of existing transport layers, particularly in handling durable sessions across multiple devices and interruptions. Traditional request-response models struggle when connections drop, users switch devices, or agents need to hand off tasks to humans, leading to corrupt state and duplicate outputs. The absence of a robust transport layer results in issues like state corruption during reconnects, lack of visibility into agent actions, and inefficient resource use when users disconnect. This has led teams to create makeshift solutions that often fail under production loads, diverting resources from core features. The emerging solution is a dedicated infrastructure layer for durable sessions, akin to durable execution for back-end workflows, which ensures continuity and state persistence. Ably AI Transport offers a drop-in solution that simplifies these complexities by providing resumable streaming, multi-device continuity, presence management, and bi-directional communication without altering existing AI models or orchestration processes.

Amber Dawson explores the intricacies and challenges involved in implementing resume tokens and last-event IDs for live streaming in AI applications, which are designed to enhance user experience by allowing streams to resume seamlessly after disconnections. She explains the foundational components of resumable streaming, including message identifiers, client state tracking, reconnection protocols, and catchup delivery, and highlights the limitations of Server-Sent Events (SSE) in handling bidirectional messaging and distributed infrastructure. Dawson discusses the complexities of building resume functionality into WebSockets, emphasizing the need for custom session management and token-level storage, which can become performance bottlenecks when handling large volumes of data. The article also addresses the issues of duplicates and gaps in message delivery, the challenges of maintaining multi-device continuity, and the significant effort required for developing a reliable, production-grade resumable streaming system. Ultimately, Dawson suggests that while building custom solutions is feasible, leveraging transport infrastructure that integrates resume capabilities can alleviate many of these challenges and allow teams to focus on application logic rather than infrastructure concerns.

Building reliable AI agents for production involves addressing infrastructure challenges beyond just AI model development, particularly in maintaining real-time synchronization between agents and clients. This requires solving issues related to distributed systems such as connection management, client-specific state tracking, distributed agent routing, and ensuring continuity between historical and live data. Effective solutions involve implementing a robust transport layer that manages delivery, synchronization, and state management, allowing for seamless token streaming, message appends, and annotation handling. By employing these technical patterns, AI applications can handle network interruptions, device switches, and multi-user sessions more efficiently, reducing complexity and enhancing user experience. The focus shifts from model capabilities to overcoming the distributed system hurdles, ensuring that AI applications can function resiliently across various conditions.

Amber Dawson's article discusses the challenges of maintaining AI response continuity during network interruptions or page refreshes, a problem not stemming from AI models but from delivery mechanisms. Most AI applications utilize HTTP streaming, which fails to preserve response continuity if a user refreshes the page, loses connection, or switches devices. Traditional solutions like client-side buffering or response regeneration prove inadequate due to their limitations in handling network interruptions and session continuity. To address this, a system for resumable streaming is proposed, involving session identity, offset tracking, and ordered delivery with history. This infrastructure allows users to resume AI responses seamlessly, even across multiple devices, enhancing reliability and user trust in AI applications. The article emphasizes the need for systems like Ably AI Transport to manage session continuity and offset synchronization, ensuring that AI delivery is as dependable as the intelligence it provides.