September 2025 Summaries

Latency in speech-to-text (STT) systems is crucial for delivering effective voice experiences, such as interactive agents and live captioning, by ensuring swift and accurate responses. Gladia's approach to measuring latency involves distinguishing between the time to first partial token (TTFB) and the time to final result, achieving sub-300 ms partial and ~700 ms final latency on 3-second utterances. This involves measuring at multiple milestones, such as audio capture start to first partial hypothesis, and controlling factors like frame size, endpointing thresholds, and network jitter to balance latency with stability and accuracy. Real-Time Factor (RTF) is used to assess throughput and capacity planning, ensuring systems can handle live audio without delays. When benchmarking, it's essential to maintain consistent conditions and report metrics like P50, P95, and P99 for each milestone, rather than a single blended value, to capture the nuances of latency across different scenarios.

AI voice agents are revolutionizing global customer support by enabling natural, real-time communication across multiple languages and dialects, overcoming the previous limitations of human agents who could only handle a few languages and struggled with fluid switching between them. For providers, building effective multilingual AI voice agents involves more than basic translation; it requires sophisticated speech-to-text (STT) and natural language understanding (NLU) systems that can dynamically detect languages, handle dialect-specific nuances, and maintain context to accurately capture user intent and tone. This is crucial as simple translations often fail to convey the true meaning, leading to customer frustration and mistrust. Companies like Gladia offer advanced AI platforms that support over 100 languages, providing features like code-switching, context-aware translation, and tone control, essential for delivering seamless, culturally sensitive customer interactions. As businesses increasingly operate in a global market, robust multilingual capabilities in voice agents are becoming foundational, not just advantageous, for maintaining customer satisfaction and expanding market reach.

Attention, a New York-based AI startup founded in 2021, enhances sales workflows through AI-powered tools by transforming sales interactions into structured and actionable data. Central to its operations is high-quality transcription, which is crucial for features like CRM autofill, conversation analytics, and automated follow-ups. To ensure accuracy and scalability, Attention uses Gladia's speech-to-text API, which offers high precision, multilingual support, and speaker diarization. This collaboration has enabled Attention to improve pilot project outcomes, increase retention rates, and support global teams effectively. Gladia's API provides a reliable transcription layer that ensures accurate keyword and competitor detection, essential for maintaining robust downstream analytics and consistent sales processes.

In the realm of real-time speech-to-text (STT) systems, partial transcripts play a crucial role in balancing latency and accuracy during voice interactions. These interim results, generated before final transcripts are confirmed, allow voice agents to preload content, display live captions, and respond more naturally without waiting for complete silence. While partials can enhance responsiveness, they also pose challenges due to their inherent instability, which can lead to incorrect actions if acted upon prematurely. Effective management of partial transcripts involves using confidence scores, time-based delays, and debounce logic to mitigate risks and ensure reliability. Strategic approaches such as warming up language models, optimizing retrieval operations, and incorporating confirmation steps for high-risk actions are recommended to leverage partials effectively. By treating partial transcripts as a core architectural decision, voice systems can achieve both speed and accuracy, maintaining flexibility to adapt to various use cases and user preferences.

Gladia has introduced partial transcripts to its real-time API for Voice AI agents, a feature designed to enhance response times by streaming word-by-word transcripts rather than waiting for complete outputs. This innovation allows agents to grasp user intent more quickly and respond with less delay, thereby improving the fluidity and naturalness of conversations. Partials are emitted exceptionally fast, particularly the initial words of an utterance, facilitating ultra-low latency in the Speech-to-Text (STT) phase and allowing developers to leverage Large Language Models (LLMs) for formulating responses. Although partials may be less accurate than final transcripts, LLMs can interpret them effectively to maintain the quality of interactions. To optimize performance, Gladia recommends specifying the target language to avoid potential issues in language detection. The new feature is available to all Gladia users but requires activation through the API's configuration settings.