May 2023 Summaries

Speechmatics has developed a speech-to-text and translation API that enables real-time translation in 34 languages, facilitating communication across borders. The technology is designed for both batch and real-time scenarios, overcoming challenges such as latency, linguistic variations, and word ordering complexities. With its unified API, users can easily integrate transcription and translation services into their applications, streamlining the setup process and reducing costs. This innovation has significant implications for media organizations, educational institutions, contact centers, and businesses looking to expand globally, enabling them to reach a wider audience without additional effort, save time, and increase revenue. The technology also promotes diversity and inclusion in education by providing real-time translation for global accessibility and engagement.

Speech recognition technology has revolutionized numerous aspects of modern life, extending beyond voice assistants to various industries and applications. In healthcare, it enables doctors to save time by converting notes from speech to text, potentially reducing wait times and improving patient care. Autonomous bank deposits utilize speech-to-text software to provide better customer service, while personalized ads are tailored to individual behavior through speech recognition. Speech-to-text technology also simplifies home lives with voice-activated search, playlist shuffling, and productivity management tools. Furthermore, it enhances road safety in cars by allowing hands-free interactions, supports remote work with transcription services, and even aids military pilots in reducing workload during missions. With applications ranging from medical assistants to fighter jets, speech recognition technology has become an indispensable component of our daily lives, offering numerous benefits despite its limitations.

The company Speechmatics has moved its transcription models to use GPUs, which significantly improves accuracy but also increases costs. To ensure efficient processing and maintain cost-effectiveness, they use the Real Time Factor (RTF) ratio as a guideline for performance. However, running on GPU hardware introduces challenges such as shared resources and unpredictability in traffic demand. To address this, Speechmatics uses Kubernetes Event-Drive Autoscaling (KEDA) with Prometheus integration to scale their GPUs based on metrics provided by the Triton Server's `/metrics` endpoint. KEDA allows them to scale out based on specific metrics, including inference queue duration and count, which provides a more accurate representation of performance issues. Additionally, they implement deallocation on scale-down mode in AKS to accelerate node scaling and reduce pending time, resulting in improved cost efficiency and reliability.

Real-Time ASR Systems have two common modes of operation: batch and real-time. In batch mode, audio is provided in complete files with a single transcript output, allowing higher accuracy. Real-time systems provide an audio stream, returning short segments of transcription back at regular intervals, where the trade-off between latency and accuracy comes into play. Evaluating batch versus real-time ASR, Ursa outperforms competitors in accuracy even when prioritizing speed over accuracy, achieving near-batch levels of accuracy with low latency settings. Latency is controlled through `max_delay` and `max_delay_mode`, allowing for a balance between timeliness and accuracy. Ursa's latest release demonstrates outstanding performance, reducing to zero relative difference in WER as latency increases to 10s, outperforming major vendors such as Amazon, Microsoft, and Google.