June 2026 Summaries

AI chat streams often rely on HTTP streaming and Server-Sent Events (SSE), which lack built-in session recovery, necessitating additional infrastructure for reliable resumption. The Vercel AI SDK offers a solution with its resumable streams feature, utilizing Redis to buffer tokens for page-reload recovery, but it is limited to single-device use and requires additional operational responsibilities. This approach is suitable for stable desktop connections with short responses, but becomes less feasible for complex scenarios involving long queries, multiple devices, or enterprise environments. In contrast, Ably AI Transport provides a more robust alternative, offering durable sessions that maintain the complete state of conversations independently of connections, enabling seamless multi-device continuity and overcoming the limitations associated with Redis-backed setups. The choice between these options depends on the specific UX needs, infrastructure capabilities, and the complexity of the AI support product in question.

AI chat platforms, traditionally seen as solitary experiences, are evolving to facilitate collaborative efforts involving multiple users and AI agents. While existing systems like ChatGPT and Claude permit sharing of conversations, they don't fully support real-time collaborative interactions. Companies like Anthropic are addressing this gap, introducing features like a “Comments” section for team discussions. A new model explored involves "durable sessions," which allow multiple people and AI agents to share a single chat, enabling seamless collaboration. The Ably AI Transport SDK is an implementation of this concept, offering features such as resumable streaming, session continuity across devices, and multi-agent coordination. A demo outlined in the text showcases how friends can plan a day out with the assistance of an AI, with the session storing all conversation states and allowing new participants to catch up instantly. This approach highlights the potential for more interactive and shared AI experiences, paving the way for innovative applications that integrate human and AI interactions effectively.

Live polls at conferences or events can be effectively implemented using Ably's Pub/Sub system, offering a scalable and low-latency alternative to traditional server-based CRUD applications. By utilizing Ably's annotations, which allow for attaching information such as votes to messages, the system aggregates and summarizes votes efficiently without requiring each vote to pass through a central server. This setup consists of three roles: the voter, who participates via their phone; the presenter, who displays results on a big screen; and the admin, who manages the poll process. Annotations offer the advantage of separating summaries from raw events, allowing different clients to access only the data they need while maintaining security by granting least-privilege capabilities to each role. This architecture ensures one vote per person, as enforced by Ably's unique aggregation type, and accommodates a large number of concurrent voters without latency issues.

AI adoption programs often fail to fundamentally change team operations, as initial increases in tool accessibility and usage do not necessarily translate to improved capabilities. The focus on usage metrics can be misleading, as they do not measure the effectiveness or impact of AI on team performance. Instead, two key performance indicators (KPIs) are recommended: one that assesses whether AI is enabling new outcomes previously unattainable, and another that evaluates how integrated AI is in daily operations. These KPIs are scored monthly by team leads based on evidence, not assertions, with an emphasis on continuous improvement rather than initial high scores. The goal is to determine whether AI is genuinely enhancing team capabilities or simply making existing processes faster. At companies like Ably, honest self-assessment reveals most teams start at a low capability score, with gradual progress reflecting genuine integration of AI into workflows rather than superficial adoption.

The discussion highlights a critical issue in AI chat applications using the Vercel AI SDK, where invoking the `chat.stop()` function or closing a browser tab only terminates the HTTP connection without halting the server-side process, leading to continued billing for token consumption. This behavior is not erroneous but inherent to resumable stream architectures, which are designed to withstand disconnections without stopping the process. The solution involves creating a dedicated stop endpoint with features such as idempotency checking, persistence of partial assistant snapshots, and backend-specific cancellation logic, none of which the SDK inherently provides. In contrast, the Ably AI Transport offers a more efficient approach by leveraging a bidirectional session model, allowing explicit cancellation signals to be sent, thus handling race conditions and disconnect distinctions more effectively. This model eliminates the need for a separate stop endpoint by integrating cancel, stop, and redirect as first-class session signals, ensuring that intent, rather than mere disconnection events, guides the process termination.

Wayfarer is a demonstration of a collaborative AI app designed to assist users in planning trips by providing a live, interactive experience through a combination of chat history, synthetic UI, and structured itinerary states. The app leverages a durable session on Ably, which allows multiple users and an AI agent to interact with the shared trip data seamlessly across various devices and refreshes. This approach eliminates the need for additional infrastructure such as a stream store, sync layer, or presence service by attaching to a single addressable session. Key features include resumable streaming, multi-device continuity, and bidirectional communication for stopping the AI agent mid-task, all facilitated by Ably AI Transport integrated into the Vercel AI SDK. Through this setup, Wayfarer maintains a coherent and synchronized user experience without duplicating efforts or tokens, and supports presence detection by showing who is currently viewing the trip.

The integration of Ably AI Transport with Vercel's AI SDK enhances the useChat feature by introducing advanced capabilities such as resumable streams, cross-device and multi-user synchronization, conversation branching, history compaction, and stop-and-approve controls. By swapping the default transport of useChat for Ably AI Transport, conversations can now be resumed after interruptions, synchronized across multiple devices and users, and managed with branching and compaction features that streamline the stored conversation history. This setup allows for a collaborative environment where multiple users can engage in a shared session and see the full conversation history, with the ability to approve or stop actions before they are executed. The demo illustrates these capabilities, showing how the new transport maintains session durability and enhances the chat experience without altering the Next.js app's model, tools, or message components.

AI Transport v0.3.0 builds on the foundational concept from v0.2.0 that a session is a channel by introducing features like presence and LiveObjects, which enable real-time updates and shared state management in AI sessions. With presence, users and agents can see who's online and manage participant states, enhancing interaction through indicators such as "typing..." or cursor positions. The update also simplifies codec creation by making the codec interface declarative, allowing the transport to manage event ordering, de-duplication, and replay, thus reducing the developer's workload. LiveObjects, which need to be manually enabled, allow for synchronized shared states, such as task progress or shared documents, on the same channel as conversations, ensuring all clients are updated in real time. These advancements make AI Transport a more robust and user-friendly platform for developing multi-party, resumable AI conversations.

Ably's experience with AI-driven dashboard creation highlighted significant challenges in data governance and trust, as inconsistent metrics led to a proliferation of ad hoc, unreliable dashboards. Initially, overwhelmed by the inadequacies of their chosen tool, Metabase, employees turned to AI agent Claude for solutions, which only exacerbated misinformation due to the lack of a cohesive data strategy. The company addressed these issues by introducing a "data warehouse genie," a governed system that centralizes knowledge and ensures consistent data querying, and by transitioning dashboards to a code-based, reviewable format that integrates directly with their existing repositories. This approach not only enhanced transparency and trust in data by allowing dashboards to carry their own definitions but also democratized the creation process, enabling non-engineers to contribute governed dashboards with the assistance of AI. The new system fosters continuous improvement by capturing learning from each dashboard build, thereby enriching the AI's knowledge base and ensuring that the answers it provides are increasingly accurate.

The development of an AI chat app using the Vercel AI SDK functions smoothly in a development environment but encounters limitations when deployed to production due to the default transport layer, DefaultChatTransport, which is designed for HTTP and Server-Sent Events (SSE). These protocols, while suitable for a stateless serverless platform, present challenges such as ambiguous cancellation signals, lack of multi-device support, and the need for significant infrastructure for stream resumption. DefaultChatTransport's limitations become apparent when production requirements exceed the single-user, single-device model it supports, leading to potential costs like orphaned AI generations and silent failures on additional devices. Vercel's design allows for a pluggable ChatTransport interface, enabling the replacement of DefaultChatTransport with a WebSocket-based transport layer, which offers a more durable session management solution by decoupling sessions from individual connections. This replacement resolves issues like cancellation ambiguity and supports multi-device delivery without altering existing agents, tool calls, or UI elements. The decision to switch transport layers should be based on specific production needs, such as reliable server-side cancellation and multi-user session handling.

The Vercel AI SDK's DefaultChatTransport, designed for stateless serverless platforms, uses HTTP POST and Server-Sent Events (SSE) to facilitate chat functionalities. While effective for single-user scenarios with stable connections, this setup reveals limitations in production environments. These constraints include an inability to reliably cancel server-side operations, lack of multi-device support, and the need for custom infrastructure, such as Redis, to resume streams. The transport is not inherently flawed but is bounded by HTTP's structural properties, which do not support persistent connections or multi-user scenarios. Vercel's architecture does not accommodate persistent WebSocket connections, prompting the design of a pluggable ChatTransport interface. Transitioning to a WebSocket-based transport layer, like Ably AI Transport, allows for durable sessions that mitigate these limitations by enabling persistent connections, reliable cancellation signals, and multi-device delivery without altering the existing application logic. This shift addresses the shortcomings of DefaultChatTransport by decoupling sessions from connections, thus resolving issues related to session persistence and multi-user engagement.

AI products are often hindered by user experience issues rather than model capabilities, primarily due to failures in the delivery layer between the AI agent and the client. Over nine months, discussions with over 40 companies revealed that most AI user experience problems stem from the transport layer, where sessions break, context is lost, and user trust erodes. The proposed solution is to implement "durable sessions," which are persistent and shared resources that remain intact despite connection drops, device switches, and agent crashes. This approach requires a transport-layer swap rather than a complete rewrite, allowing existing agents to be adapted without changing their core code. By decoupling the session from the connection, durable sessions enable features like multi-device synchronization, bidirectional control, and seamless human-AI handoffs, addressing common user frustrations such as dropped sessions and context loss. The concept was demonstrated using the Vercel AI SDK, showing the benefits of maintaining session continuity and coherence across different devices and agents.

Version 0.2.0 of @ably/ai-transport introduces significant enhancements to the SDK, focusing on durable AI session management that supports complex interaction patterns beyond the traditional request-response model. This update shifts the framework from a simple token streaming setup to a comprehensive session-based structure, where every interaction—inputs, outputs, agent lifecycle events, and control signals—is durably recorded in Ably channels. This approach allows for multi-step, multi-party, and long-running conversations that can span devices and remain consistent across reconnections. The new version supports sophisticated agent behaviors, such as pausing for human input and resuming tasks, making it possible to build more interactive AI user experiences. The SDK, compatible with various AI frameworks, helps manage agent execution as runs that can suspend and resume independently from the initial invocation, leveraging Ably's durable message logging and exactly-once delivery features to ensure seamless session continuity and multi-client synchronization.

AI agents face unique challenges with WebSocket disconnections due to their mid-execution silences, which can be misinterpreted as idle time by infrastructure like AWS Application Load Balancer and Cloudflare, leading to unintended connection drops. These disconnections result in the loss of session context, including tool call results and generated tokens that occur during the gap. While WebSocket transport reconnection restores the connection itself, it does not recover the session state, necessitating a session recovery layer to store and replay in-flight data. Solutions like Ably AI Transport address these issues by providing automatic reconnection, history compaction, and protocol fallback, ensuring that client sessions can resume seamlessly without requiring complex application code. Configuring WebSocket servers to send regular ping frames can prevent idle timeouts, keeping connections alive under the thresholds set by AWS ALB and Cloudflare.

Temporal and Ably AI Transport both address challenges in maintaining continuity in backend processes and user sessions, respectively, but at different layers of the tech stack. Temporal provides "durable execution" by ensuring that workflows can survive server crashes and restarts, maintaining their state without the need for custom idempotency logic. However, it does not address the durability of user-agent sessions, which is where Ably AI Transport comes in. Ably AI Transport offers a durable session layer for AI applications by creating persistent, addressable channels that retain session state across disconnections and reconnections, thus resolving issues that arise when users switch devices or lose network connections. This session durability is crucial for multi-message AI assistants, which require stable session identities, message buffering, and seamless reconnection capabilities that existing tools like Redis, WebSockets, and job IDs do not fully provide. While Temporal is focused on workflow execution, Ably AI Transport ensures that the conversational context between an AI agent and a user remains intact, allowing for a coherent user experience even when connectivity is interrupted.

Temporal introduced a new paradigm by providing a durable execution context that ensures workflow persistence across server restarts, making backend jobs crash-proof without necessitating custom idempotency logic. This contrasts with traditional queuing systems, which address different concerns. Similarly, in AI applications, the need for a persistent session layer is highlighted, as current solutions like WebSockets, Redis, and sticky sessions do not adequately maintain session continuity across device switches or connection drops. Ably AI Transport addresses this gap by offering a durable session infrastructure that persists independently of individual connections, allowing both AI agents and users to maintain conversational context seamlessly across reconnections. While Temporal secures backend workflows, Ably AI Transport ensures the continuity of user-facing sessions, complementing each other in managing different layers of AI systems.

Ably has revolutionized its SDK development process by leveraging large language models (LLMs) to enhance efficiency and quality. Traditionally, Ably's SDKs spanned a wide range of platforms and languages, relying on a comprehensive specification to ensure consistency. However, inconsistencies in interpretation, development effort, and test coverage posed significant challenges. The introduction of AI-powered LLMs allowed Ably to construct a Universal Test Suite (UTS), which employs natural language to specify tests that LLMs can translate into code for different platforms. This advancement led to the creation of a Dart library with improved test coverage and traceability compared to previous libraries. By allowing AI to handle the mechanical aspects of code translation, human developers can focus on critical design decisions, resulting in a substantial boost in development speed and reliability. This approach not only streamlines the process but also sets a new standard for future SDK enhancements, such as the AI Transport SDK, demonstrating the transformative potential of AI in software development.