We built a Custom Transport for Vercel's AI SDK

Ably, a real-time messaging platform, has been adapted to work as a transport layer for the Vercel AI SDK, providing enhanced features such as multi-device and multi-user support, resumable streams, human handoff, interruptions, and history compaction. The integration aimed to expand the capabilities of the Vercel AI SDK, which traditionally uses HTTP streaming for its UI SDK and is limited by single-request single-response assumptions. Ably's pub/sub model allows for a more dynamic message exchange, overcoming issues like lack of multi-user visibility, resumability, and bidirectional communication that HTTP streaming faces. By implementing a custom transport through Ably, developers can bypass these limitations and extend useChat's functionality, albeit with challenges like bypassing the state machine and lifecycle hooks, which Ably addresses with features like the setMessages function. This adaptation demonstrates how Ably's transport can significantly elevate the interactivity and resilience of AI-driven applications on the Vercel AI SDK, offering developers a broader range of options for managing real-time conversations between AI models and users.