June 2026 Summaries

Batch Diarization V2 is a significant upgrade to speaker attribution technology, enhancing the accuracy of speaker labeling in pre-recorded audio across various domains such as contact centers, healthcare, and voice AI applications. The new model introduces expanded training data, an improved speaker embedding model, and better segmentation and clustering, leading to more precise speaker attribution and reduced labeling errors. Evaluations showed that Diarization V2 consistently outperformed its predecessor, Diarization V1, with human evaluators preferring the new version 3.3 times more often due to improved performance, particularly in challenging audio scenarios. The update is available via the new diarize_model parameter, allowing users to select between model versions without any changes in pricing or existing integrations, and is compatible with Deepgram's batch Speech-to-Text offerings.

Integrating voice agents with Salesforce involves choosing between using the native Agentforce Voice or external voice AI APIs, each offering different levels of control and complexity. Agentforce Voice, leveraging OpenAI Whisper for speech-to-text (STT), handles calls over PSTN and SIP trunking, supports customer interruption, and logs conversations, while integrating external APIs allows using dedicated STT providers for domain-specific terminology. The choice of STT provider is crucial, as transcription errors can significantly impact CRM data quality, leading to costly rework and unreliable records. For effective deployment, businesses should evaluate STT providers based on real call audio, consider latency requirements for real-time interactions, and model costs across all layers, including Salesforce's Flex Credits and telephony charges. Businesses are advised to start with a single use case, validate STT accuracy, and track CRM metrics to ensure that voice agents improve data quality and operational efficiency.

Enterprise restaurant brands face unique challenges when integrating voice AI technology due to the diverse nature of their menus and point-of-sale systems, as illustrated by two contrasting Deepgram clients: a fried chicken chain and a pastry brand. These brands leverage custom AI models developed by Deepgram, enabling them to streamline operations through tailored speech-to-text, text-to-speech, and speech-to-speech functionalities, which are integrated into various facets of their business, from order-taking to staff interactions. The approach emphasizes the importance of brands building their own AI solutions for greater control and innovation, supported by direct collaboration with AI research teams. Deepgram's strategy facilitates this by providing enterprise clients with fine-tuned models and dedicated engineers, allowing them to own the AI models and achieve better operational efficiency and customer engagement.

Named Entity Recognition (NER) on voice transcripts faces significant challenges compared to traditional text due to the inherent differences in Automatic Speech Recognition (ASR) output, which often lacks capitalization and punctuation, leading to a loss of crucial formatting cues that models rely on. Despite advancements in architectures like pipeline approaches, LLM-based extraction, and joint audio-to-entity models, issues such as ASR error propagation, particularly in domain-specific entities, persistently degrade accuracy. To improve NER outcomes, enhancing the quality of transcripts through better Speech-to-Text (STT) accuracy, smart formatting, and keyterm prompting is essential. For real-time applications, the ASR-then-NER pipeline remains the most feasible, while batch processing benefits from LLM-based approaches for higher accuracy on rare entities. Understanding the nuances of entity-level Word Error Rates (WER) rather than just aggregate WER is critical, as transcription errors concentrated in entity spans have a disproportionately negative impact on NER performance.

The choice between a bundled voice agent API and an assembled stack significantly impacts not just costs but also architectural decisions, integration workload, and system performance. Bundled APIs consolidate STT, LLM orchestration, and TTS into a single endpoint, simplifying operations by reducing vendor sprawl and integration complexity, but at the cost of reduced customization and control over individual components. Conversely, assembled stacks, while potentially cheaper and offering greater customization, demand more complex management and integration work, particularly in handling separate billing units and ensuring efficient latency management. The decision to opt for either approach depends on factors such as volume, need for custom-trained models, compliance requirements, and existing infrastructure capabilities. Bundled APIs are often recommended for teams seeking ease of setup and predictability, while assembled stacks are preferable for those needing deep control and optimization for specific requirements.

Deploying AI call center voice agents at scale requires careful orchestration to manage latency, failure modes, cost, and monitoring. The process involves integrating speech-to-text (STT), large language models (LLM), and text-to-speech (TTS) into a seamless pipeline, where each component contributes to the overall latency, with LLM being a significant factor. Ensuring the reliability of these systems under real-world conditions is crucial, as demonstrated by real incidents where background noise and inaccurate confidence scoring led to failures. The choice between bundled and build-your-own (BYO) stacks involves trade-offs between integration simplicity and control over individual components. Effective monitoring should focus on conversation-level metrics to catch issues that standard API health checks might miss. Cost modeling is essential, as pricing structures can vary significantly at high volumes, influenced by factors like concurrency fees and billing during silent periods. Compliance and latency requirements also drive the selection of stack components and their deployment, particularly for regulated industries.

Accent variability in Automatic Speech Recognition (ASR) systems poses significant challenges for multi-region restaurant voice ordering, as demonstrated by McDonald's terminated voice ordering pilot, which saw accuracy drop due to accent misinterpretations. The ASR layer is crucial because any transcript errors from this stage impact the entire ordering process, leading to incorrect orders and costly customization mistakes. To address these issues, strategies such as increasing speaker diversity in training data, using Keyterm Prompting for real-time vocabulary corrections, and selecting locale-specific models are recommended to maintain accuracy across different accents. Real-world testing with market-specific audio and conditions is essential to evaluate operational accuracy versus benchmark accuracy, as the latter often fails to account for the complex, noisy environments of drive-thrus. Implementing these techniques can help restaurant chains achieve stable Word Error Rates (WER) across regions, ensuring that voice AI systems function effectively in diverse linguistic settings.

As healthcare technology evolves, the intersection of HIPAA regulations and voice AI technology necessitates careful compliance with business associate agreements (BAAs). The anticipated updates to the HIPAA Security Rule by 2026 aim to increase vendor accountability, mandating encryption and technical safeguard certifications, which may require amendments to existing BAAs to accommodate voice AI deployments. When voice AI systems handle clinical audio, they qualify as electronic protected health information (ePHI) processors, triggering the need for a BAA amendment that addresses data flow, subcontractor disclosure, breach notifications, and data retention. The intricacies of voice AI involve multiple layers of processing, often with third-party providers, which complicates compliance and necessitates precise contractual language to ensure data protection. It is crucial for healthcare technology leaders to audit current agreements before implementing voice AI solutions, ensuring that all subcontractor relationships and data handling processes are explicitly covered. Engaging legal and compliance teams early in the procurement process can mitigate risks associated with inadequate BAAs, helping organizations align with both current and forthcoming regulatory standards.

Deepgram, a real-time AI infrastructure company, has partnered with Fortanix and NVIDIA to deliver a secure, on-premises voice AI solution for regulated industries, ensuring sensitive data remains private during processing. This collaboration leverages Fortanix Confidential AI and NVIDIA Confidential Computing to protect data and proprietary model weights from theft or misuse by running them in hardware-isolated environments. The solution provides enhanced security for enterprises handling sensitive information, such as patient conversations and financial transactions, by maintaining data confidentiality during active use. The partnership allows enterprises to deploy voice AI applications securely without compromising performance, meeting strict regulatory requirements like HIPAA and GDPR. By integrating Deepgram's advanced voice AI models with Fortanix's security platform and NVIDIA's GPUs, organizations can confidently use voice AI to enhance efficiency and security across various applications, from healthcare to IT operations.