September 2022 Summaries

Aloware, a Contact Center Software as a Service provider, has leveraged AssemblyAI's speech recognition and audio intelligence APIs to enhance its product offerings. By integrating AI models into their platform, Aloware has been able to ship smart transcription capabilities in just 6 weeks, automating most QA tasks and transcribing calls with near human-level accuracy. This development not only saves customers' time but also offers unforeseen insights and performance tracking for call reviews. Moreover, AssemblyAI's AI models have helped Aloware win more clients by offering an automated call QA feature. In the future, Aloware plans to use these AI capabilities for in-moment notifications for poor call quality and aggregated reporting on agent call performance.

AssemblyAI has been recognized as a G2 High Performer and Momentum Leader in Voice Recognition Software for the fifth time, with previous recognitions in Summer 2022, Spring 2022, Winter 2022, and Fall 2021. The company maintains high customer satisfaction scores, including an average rating of 9.9 for Ease of Use and 9.6 for Quality of Support. Reviewers praise AssemblyAI's accuracy, support quality, commitment to AI research, and innovative API platform.

The Micro Machines example was transcribed with Whisper on both CPU and GPU at each model size, and the inference times are reported below. First, we see the results for CPU (i5-11300H) ``` Tiny: 0.02 sec Base: 0.06 sec Small: 0.14 sec Medium: 0.39 sec Large: 1.47 sec ``` Next, we have the results on GPU (high RAM GPU Colab environment) ``` Tiny: 0.01 sec Base: 0.02 sec Small: 0.05 sec Medium: 0.13 sec Large: 0.46 sec ``` Here are the same results side-by-side ``` CPU GPU Tiny: 0.02 sec 0.01 sec Base: 0.06 sec 0.02 sec Small: 0.14 sec 0.05 sec Medium: 0.39 sec 0.13 sec Large: 1.47 sec 0.46 sec ``` The cost to run Whisper is as follows, using different batch sizes (values of which can be found in the legend): ``` Tiny: 0.03 USD/h Base: 0.09 USD/h Small: 0.21 USD/h Medium: 0.57 USD/h Large: 2.28 USD/h ```

The paper "Multistream CNN For Robust Acoustic Modeling" explores the concept of Multi-resolution optimization through a Multistream Convolutional Neural Network (CNN). It leverages different dilation rates across layers, allowing them to learn various feature views at multiple resolutions. The authors also decompose the convolution matrix in TDNN-F with an orthonormal constraint for improved performance. Furthermore, they propose a Multi-resolution optimization approach that could be used with other modeling techniques. They emphasize the benefits of TDNN-F layers compared to standard Conv1d layers due to their mathematical structure.

The Audio Intelligence Dashboard is a project that utilizes Gradio and OpenAI to provide an interactive interface for exploring the contents of audio files. It uses OpenAI's text-to-speech model GPT-3 to generate transcriptions from uploaded audio, then extracts topics, entities, and speaker labels from these transcripts using BART and DistilBert models. The dashboard is built with Python and Flask, and can be run locally or deployed to Hugging Face Spaces for sharing. It features several interactive components such as topic analysis, entity detection, and speaker profiling. Users can upload audio files in .wav format and the dashboard will display transcriptions of these files along with relevant extracted information. Overall, the Audio Intelligence Dashboard is a useful tool for anyone interested in analyzing the contents of audio recordings. It demonstrates how powerful pre-trained models like GPT-3 can be when combined with user-friendly visual interfaces.

AssemblyAI has launched its Creators program, aiming to foster a community of AI creators who help one another and contribute back to the developer community. This initiative is aligned with AssemblyAI's mission to provide state-of-the-art AI models accessible through an API. The Creators will be supported with various resources and benefits, such as exclusive swag, free credits to APIs, special recognition on social media, and networking opportunities. The program targets developers who are passionate about creating content and interested in the AssemblyAI platform. Examples of projects from existing Creators include a speech-to-text app and transcribing a talk with minimal coding.

The paper "Towards Contextual Spelling Correction for Customization of End-to-end Speech Recognition Systems" proposes a general ASR biasing solution that is domain-insensitive and can be adopted in various scenarios. A Seq2Seq model corrects the spelling of rare words or proper nouns by considering both ASR hypotheses and external context words/phrases. Combining Shallow Fusion with Contextual Spelling Correction reduces Word Error Rate (WER). The model is efficient for large context phrase lists during training and inference. It works well on high OOV rate test sets, indicating that it learns error patterns at the subword level rather than word-level. ASR biasing post-processing can improve proper noun detection in end-to-end ASR compared to encoder biasing methods like Contextual RNN-T or CLAS. Non-AutoRegressive (NAR) models are faster for inference, speeding it up by 2.1 times compared to AutoRegressive solutions.

AssemblyAI has received its annual SOC 2 Type 2 certification, which highlights the company's ongoing commitment to maintaining secure systems and protecting customer data. The report covers five trust service principles (TSPs): security, availability, processing integrity, confidentiality, and privacy. Examples of security controls in place at AssemblyAI include encryption of all data at rest and in transit, secure backup of data with regular restoration tests, a security reporting process, mandatory employee training on security awareness, strict access control to production environment using SSO and MFA, source code change peer reviews following a secure development policy, and regular reviewing of vendors' security practices. While the report serves as proof of the company's efforts in maintaining robust security measures, AssemblyAI will continue to evaluate its systems to ensure constant vigilance against ever-increasing threats.

AssemblyAI has introduced two new enterprise offerings - AutoTune Early Access and Premier Support, along with a security certification. The company aims to improve transcription accuracy through its AI technology, making it easier for businesses in industries like Video Platforms, Call Tracking, Podcasts, etc., to innovate and differentiate themselves. AutoTune helps detect potential errors in the transcription process by identifying areas where AI models may be underperforming, while Premier Support offers specialized technical assistance for large-scale deployment users. Furthermore, AssemblyAI has achieved its SOC 2 Type 2 certification, ensuring stringent security controls for customers' audio data. These developments aim to provide more reliable and seamless building experiences with the AssemblyAI API platform.

The files contained several sensitive pieces of data such as names, addresses, dates of birth, phone numbers, email addresses, credit card numbers, medical records, and financial information. The PII Redaction feature successfully detected and redacted all of this sensitive information in the transcriptions generated by the AssemblyAI Speech-to-Text API. The Content Safety Detection model flagged some potentially risky content such as hate speech and weapons references. The sentiment analysis results indicated that there were more positive sentiments than negative ones in the recordings. The Summarization feature automatically segmented the files into chapters and generated brief summaries for each chapter. Finally, the Entity Detection model accurately identified various entities present in the transcriptions such as people's names, organizations, locations, dates, times, quantities, percentages, currencies, email addresses, URLs, phone numbers, social security numbers, credit card numbers, medical conditions, and treatments.

AssemblyAI has announced upcoming model updates and new services for fall 2022, following a recent $30M Series B announcement. These include AutoTune for improving transcription accuracy, Premier Support for dedicated assistance, Summarization Model for automatically summarizing audio/video files, upgrades to Real-time Transcription, and support for additional languages. The company is committed to continuous improvement and expansion of its AI offerings to better serve customer needs.