VAQI, Revisited: How OpenAI’s gpt‑realtime Stacks Up — With Sensitivity Analysis for Real‑World Priorities

A follow-up benchmark applies the Voice-Agent Quality Index (VAQI), which combines interruptions, missed end-of-thought windows, and response latency into a weighted 0–100 score, to compare Deepgram, OpenAI’s gpt-realtime, and ElevenLabs in a repeated food-ordering dialogue with pauses, corrections, noise, and natural speech fillers. Across 231 runs per provider, Deepgram recorded the fewest interruptions and maintained sub-second latency, OpenAI achieved the lowest missed-turn rate but had the slowest average response time, and ElevenLabs delivered the fastest responses while producing substantially more interruptions and missed turns. Deepgram ranked first under all tested weighting schemes, while OpenAI generally performed better when conversational turn-taking was prioritized and ElevenLabs improved when latency carried greater weight. The evaluation finds that gpt-realtime has improved OpenAI’s latency and turn detection relative to earlier tests, but argues that provider rankings depend on application-specific priorities, such as politeness, reliability, or speed, and that VAQI offers a repeatable way to quantify these trade-offs as voice models evolve.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.