April 2023 Summaries

The Quantization of Transformer models is a critical challenge for deploying state-of-the-art models in a cost-effective way. One popular technique for doing so is quantization, which aims to increase throughput and decrease memory footprint by reducing numerical precision of network parameters and activations. However, this can harm model accuracy if not done carefully. To address this, various techniques such as calibration, quantization-aware training, and operator fusion are explored in the paper. The authors investigate the nuances around achieving peak performance on GPU for INT8 GEMM operations, which are crucial for deploying transformer models efficiently. They also discuss the future of 8-bit quantization with the advent of Nvidia's Hopper/Lovelace architectures, which support a new floating point datatype - FP8. This has both accuracy and performance benefits, and can potentially remove/reduce the need for Quantization-Aware Training (QAT).

The text discusses the release of real-time translation in a SaaS offering, which provides translation of speech to and from English for 34 languages through its high-accuracy transcription system. The translation builds on top of the state-of-the-art speech-to-text system and benefits from the Ursa generation models, but it cannot recover from breakdowns in transcription, as seen in examples where small mistakes can have a large impact on the resulting translation. The text also compares the performance of different systems using metrics such as Word Error Rates (WER) and BLEU scores, which are limited in their ability to measure translation quality. It highlights the importance of capitalization and punctuation in real-time translation and notes that delivering high-quality real-time translation poses several challenges beyond translation quality, including minimizing delay between word recognition and translation, and striking a balance between gathering enough context for high-quality translation and minimizing delay. The system is expected to improve in line with its transcription accuracy, and more APIs are planned to be rolled out in the coming months.

This offering integrates real-time translation with its industry-leading speech recognition technology, Speechmatics, to provide a comprehensive API for businesses to overcome language barriers and automatically translate live content from multiple regions. The combined offering enables customers to use the world's most accurate speech-to-text engine and translate speech for 69 language pairs, breaking down language barriers and unlocking new opportunities for businesses to reach a wider geographical audience across multiple industries. With real-time translation, businesses can provide highly accurate captions globally for TV shows, broadcast, sports, events, podcasts, game streaming, YouTube and social media videos, while also streamlining processes and speeding up workflows. The new offering is expected to be launched later this month, with demos available at NAB Show in April 2023.

We have demonstrated that scaling self-supervised learning significantly improves the sample efficiency of automatic speech recognition (ASR) models. By leveraging large amounts of unlabeled data, these models can learn rich representations of input features and improve performance with fewer samples of labeled data. This approach is particularly effective in low-resource settings where training with high sample efficiency is crucial. Our experiments show that scaling self-supervised learning leads to greater sample efficiency and generally better performance, even when reducing the amount of labeled training data by several orders of magnitude. The results have significant implications for ASR systems, enabling them to achieve excellence with a fraction of the hours of labeled data typically required.

Ursa is a new generation model for Speechmatics' Automatic Speech Recognition (ASR) system that achieves market-leading accuracy in speech transcription and demonstrates impressive performance in specialized domains through domain generalization. The model's success is driven by significant scaling up of self-supervised learning models and neural language models, which enables better understanding of every voice and expanded coverage of domain-specific vocabulary.

The recent explosion of unstructured speech data in Contact Centers has been a transformational moment for Contact Center solutions. With several high-profile customers, providers are under pressure to tap into this change and reduce costs across the board for customers - be that in automation, agent retention or simply in avoiding compliance fines. To add value, solution providers can embed Speechmatics' best-in-class speech recognition into their offerings, enabling them to accelerate competitive differentiation and forward plan the speech recognition capabilities of tomorrow. ISVs are doing this by offering a true difference, prioritizing inclusivity, making the workforce work smarter, showing cost savings, and setting out a roadmap to growth. By leveraging Speechmatics' technology, solution providers can differentiate their Contact Center offerings, improve customer experience, reduce costs, and drive business growth.