September 2021 Summaries

AssemblyAI has released an update to its Topic Detection feature, which now accurately predicts the topics spoken in audio/video files. This improvement is particularly useful for customers handling podcasts or other media content where understanding the discussed topics can enhance advertising, recommendation, and search capabilities. The updated feature uses a more advanced deep learning neural network and the IAB Content Taxonomy to identify 698 different topics. It demonstrates improved accuracy in clustering these topics compared to its previous version. This technology is especially beneficial for customers building contextual targeting solutions for advertisers.

AssemblyAI has been recognized as a G2 High Performer and Momentum Leader for Fall 2021 due to its outstanding customer reviews on G2.com. This is the ninth consecutive recognition by G2, with an average rating of 4.8 out of 5 stars based on customer feedback. The company's API has been praised for its ease of use and quality of support, making it the top-rated API for Speech-to-Text recognition. AssemblyAI focuses on applying advanced neural networks and machine learning technology to become an industry leader in this field.

The paper "Text-Free Prosody-Aware Generative Spoken Language Modeling" introduces a novel approach to generative spoken language modeling by incorporating prosody as a feature. Previously, text has been the intermediate representation between speech inputs and NLP analyses, but this work suggests that it is suboptimal due to being a lossy medium for capturing speech. By directly modeling in the spoken language domain without cascading through text, the authors aim for a more optimal representation. They leverage self-supervised acoustic units representing phonetic content and quantized, speaker-mean normalized log F0 bins together with unit durations as input streams, which are modeled jointly with a transformer language model. The paper's findings show that prosodic input features improve both content and prosody modeling. This research direction is promising but still exploratory, indicating the potential for spoken language modeling to move towards end-to-end approaches in the future.

Recent advancements in deep learning technology have significantly improved AI's ability to recognize speech. While human transcription remains the gold standard, Automatic Speech Recognition (ASR) models are now able to transcribe lyrics in songs with a surprising level of accuracy. In tests conducted on 15 songs from three different genres, ASR achieved word error rates (WERs) ranging from 0.473 to 0.878. Female voices were generally recognized better than male ones across all genres. The most accurately transcribed song was "Hotline Bling" by Drake, with a WER of 0.473, while the least accurate transcription was for Michael Jackson's "Thriller," with a WER of 0.878. Overall, ASR models were able to transcribe about 20-30% of lyrics in songs accurately.

This tutorial demonstrates how to convert an MP3 file to text using AssemblyAI's free, fast, and simple-to-use Speech-to-Text API. The process involves uploading the MP3 file to the API, starting a transcription job, and retrieving the result of the transcription job. With advances in Deep Learning, speech recognition technology is now nearly as accurate as human transcription. This tutorial provides step-by-step instructions on how to use AssemblyAI's API for this purpose.

Python offers a variety of options for implementing Automatic Speech Recognition (ASR), categorized mainly into open-source and cloud-based solutions. Open-source libraries like wav2letter, SpeechRecognition, and DeepSpeech provide flexibility and customization, allowing developers to modify the source code, but they often require significant computational resources and expertise to manage dependencies and installations. Wav2letter, originally developed by Facebook, uses convolutional neural networks, while SpeechRecognition serves as a wrapper for various speech recognition services, and DeepSpeech, maintained by Mozilla, offers on-device offline capabilities. In contrast, cloud-based solutions like AssemblyAI's Speech-to-Text API offer higher accuracy, ease of use, and features such as speaker diarization and custom vocabulary without the need to manage local resources, although they may involve costs. Developers must consider factors such as accuracy, cost, and implementation ease when choosing between open-source and cloud-based ASR solutions for their Python projects.

On August 31st, AWS experienced an outage in their us-west-2 region, impacting a single availability zone (usw2-az2). This resulted in increased API error responses and slowdowns in transcription turnaround time for users. The issue was caused by a component within the subsystem responsible for processing network packets becoming impaired, affecting health checks and causing packet loss within the region. During this incident, there were also massive spikes in DynamoDB query times and some SQS queues started to back up. AWS' incident updates mentioned impacts to many key AWS services, including Kinesis. The company has learned from this experience and plans to make use of all four availability zones in the us-west-2 region and move production workloads out of the default VPC to follow best practices.

The study explores whether negative events in news podcasts can predict the stock market. It uses AssemblyAI's automatic speech to text transcription API to transcribe audio files of two prominent news podcasts, The Daily and Up First, and then calculates their negativity ratings. The results suggest that on days with exceptionally bad news (negativity rating over 0.7), a dip in the stock market can be expected within the next 1-3 days.

Zoom hosts 3.3 trillion meeting minutes annually, a 3,300% increase from the same quarter last year. The platform's integration with various software companies has enriched virtual conversations by enabling features like automatic transcription and highlighting key moments. Automated Speech Recognition (ASR) is crucial for these platforms, and AssemblyAI, Google Cloud Speech-to-Text, and AWS Transcribe are evaluated for their accuracy in transcribing Zoom recordings across various use cases. The benchmark report also explores the capabilities of AssemblyAI's enrichment models for Topic Detection, Keyword Detection, PII Redaction, and Content Safety Detection.