Building voice agents for the real world: 7 takeaways from our NYC meetup with LiveKit

A New York meetup hosted by AssemblyAI and LiveKit highlighted how Boardy, a networking agent, and Flagler Health, a clinical calling agent, encounter similar production challenges despite serving very different users and operating models. Panelists emphasized that conversational freedom should match the task, with Boardy encouraging open-ended discussion while Flagler tightly constrains calls and escalates medical questions to humans. Both teams favor immediate AI disclosure to build trust, while reliable caller identification and voicemail handling are essential for outbound telephony. The discussion identified multi-party turn-taking, low latency, interrupted instructions, and the gap between what an agent says and what a listener actually hears as major technical issues. Evaluation remains difficult because automated measures can misjudge failed calls and simulated agents do not fully represent human behavior, particularly in regulated healthcare environments with HIPAA requirements. Boardy’s memory design combines compact user profiles with semantic retrieval of past conversations, but voice interactions require this retrieval to happen without creating awkward silence. Across all these challenges, accurate transcription was presented as a foundational dependency for collecting information, evaluating outcomes, managing conversation flow, and retrieving useful context.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.