How to talk to an LLM (with your voice)

The Daily's developer platform powers audio and video experiences for millions worldwide. The company is exploring voice-driven AI applications, leveraging large language models (LLMs), WebRTC, and video capabilities. LLMs are good at summarizing text, answering questions, and conversing. To build a voice-driven LLM app, developers need to consider speech-to-text, text-to-speech, and the LLM itself. The platform recommends running everything in the cloud for improved reliability and lower latency. WebRTC is preferred over web sockets for real-time audio streaming due to its ability to deliver audio at low latency across various network connections. The demo showcases a choose-your-own-adventure story with DALL-E generative art, highlighting the potential of combining these technologies.