October 2021 Summaries

The research paper "Deep Shallow Fusion for RNN-T Personalization" discusses methods to improve the accuracy of proper nouns and rare words in end-to-end deep learning models, which are typically hard to personalize. Two key techniques mentioned include subword regularization and grapheme-2-grapheme (G2G) augmentation. Subword regularization involves sampling from a list of n-best outputs during training instead of using the highest probable prediction, reducing overfitting on high-frequency words. G2G can generate alternative spellings with similar pronunciations, improving recognition of rare names when used for decoding. These techniques help enhance the model's ability to predict low-frequency words like proper nouns.

There are numerous data science podcasts available for those interested in learning more about artificial intelligence, machine learning, and big data. These nine podcasts cater to different levels of expertise and cover a wide range of topics within the field. Practical AI focuses on making complex concepts accessible while Data Science at Home discusses trends and interviews industry leaders. Towards Data Science is an online journal with weekly podcasts, and O'Reilly Data Show Podcast examines conversations driving data science, AI, and big data. Data Skeptic applies critical thinking to evaluate trending topics in the field, while Talking Machines features discussions on current data science topics with experts. Data Science Imposters discusses real-world applications of data science, AI, and machine learning, and Not So Standard Deviations explores trends in academia and industry. Banana Data Podcast covers the latest news and conversations surrounding all things data science.

CallRail, a leading SaaS solution for businesses and marketing agencies, sought a modern Speech-to-Text API to enhance its Conversation Intelligence transcription accuracy and quality. The company partnered with AssemblyAI, which provided high-accuracy transcriptions and advanced features such as Automatic Transcript Highlights and PII Redaction. With this partnership, CallRail has seen call transcription accuracy improve by up to 23% and doubled the number of customers using its product. The collaboration between the two companies allows for continuous improvement in analytics and customer experience.

When using Speech-to-Text APIs, it's crucial for developers to evaluate the data privacy and security measures in place to protect sensitive information. Key considerations include whether the API retains copies of audio, video, or transcription data, and if this data is used for training models, optimizing products, or shared with third parties. The presence of Personal Identifiable Information (PII) in files raises additional concerns, especially in the event of a data breach. Best practices include choosing APIs that do not store raw files post-transcription, encrypt transcription data, allow for data deletion upon request, and only use customer data with explicit consent. Transparency in data security practices is essential for ensuring trust and confidence in the API provider.

AssemblyAI's team relies on various Machine Learning and Deep Learning research sources to improve their Speech-to-Text API. They have compiled a list of top five blogs they reference, including Distill, Machine Learning Mastery, ML CMU, Neptune Blog, and Hacker News. These blogs offer foundational content for beginners as well as advanced topics for experienced developers. Each blog has its unique focus: Distill uses interactive media to distill the latest findings in Machine Learning; Machine Learning Mastery offers practical articles aimed at developers with a series of eBooks and guides; ML CMU provides accessible content from Carnegie Mellon University's dedicated Machine Learning blog; Neptune Blog focuses on research and production teams running frequent experiments, offering insights into experiment tracking, model management, MLOps, tools, and more; Hacker News is a social news website with an abundance of user-generated or shareworthy Machine Learning content.

Over 300 participants took part in Hack The Valley, an annual student-run hackathon at the University of Toronto, where they built over 70 projects within 36 hours. Among these projects were three that utilized AssemblyAI's Speech-to-Text API: ShopAdvisr, a mobile shopping app with smart recommendations and product recognition; AcadeME, an educational tool for note taking and summarization during live lectures; and Speed Reader, an application that enables users to read text at faster speeds by streaming words on the screen.

Deep Learning is a crucial component of many everyday technologies, such as machine translation, show recommendations, and facial recognition for device login. The video explores the concept of Deep Learning, its position within the broader field of Artificial Intelligence, and the reasons behind its successful integration into daily life. It also compares Deep Learning with traditional Machine Learning algorithms, highlighting its distinct advantages and applications.

In the field of Automatic Speech Recognition (ASR), custom models are rarely more accurate than general models due to their extensive training on diverse datasets. General models can handle most audio data, and custom models may only be necessary for unique characteristics like children's speech. Adding custom vocabulary to a general model is often sufficient for improving accuracy in specific use cases. Maintaining custom models is expensive and time-consuming compared to updating general models with the latest research. Companies should consider their unique data needs, budget, and ability to maintain models before investing in custom ASR solutions.

The paper "DialogueGCN: A Graph Convolutional Neural Network for Emotion Recognition in Conversation" presents an innovative solution for emotion recognition in conversations with multiple speakers. It leverages the Relational Graph Convolutional Network to model dependencies between utterances, overcoming limitations of existing RNNs-based approaches that ignore speaker-level dependency. The proposed DialogueGCN model uses bidirectional GRUs to embed sequential based utterances and construct a relational graph with learnable weights for edge relations. This approach effectively captures both temporal and speaker dependencies, demonstrating superiority in recognizing emotions in multi-party conversations with long time-frame inputs.

AssemblyAI has released its most accurate Speech Recognition model to date, version 8 (v8), which delivers significant accuracy improvements across various types of audio and video data. The v8 model also introduces a major improvement in proper noun recognition. The company's research team, comprising AI researchers and engineers from leading technology companies, constantly researches and improves the models that power its Speech-to-Text API and other features like Topic Detection. By the end of 2022, AssemblyAI aims to develop speech recognition models approaching human level accuracy for challenging audio and video files with heavy accents and background noise. The v8 model's improvements include enhanced use of Transformers, interleaving Convolution Neural Network layers between Transformer layers, improved regularization via Layer Norm, jointly trained Language Model, and the use of word pieces instead of individual characters for predictions.

The paper "Pretraining Representations for Data-Efficient Reinforcement Learning" introduces a technique called SGI that decouples representation learning from reinforcement learning, making RL more data efficient. This is achieved by pretraining the encoder of the RL agent in an unsupervised manner using observed trajectories and two prediction tasks: predicting the next state based on the current state and action, and predicting the action responsible for state transitions. The paper demonstrates that this approach enables the RL agent to achieve greater performance under limited training data and utilize bigger encoders effectively. This work contributes to building more generalized AI agents by leveraging prior knowledge to solve new tasks.

DeepSpeech is a neural network architecture first published by Baidu's research team. Mozilla created an open-source implementation of this paper, known as "Mozilla DeepSpeech". The original DeepSpeech paper from Baidu popularized the concept of "end-to-end" speech recognition models. These models directly output characters or words from audio input, unlike traditional models that predict phonemes and then convert them to words in a separate process. The goal of end-to-end models like DeepSpeech is to simplify the speech recognition pipeline into a single model. Additionally, the theory introduced by Baidu's research paper suggests that training large deep learning models on large amounts of data can yield better performance than classical speech recognition models. Mozilla DeepSpeech offers pre-trained speech recognition models and tools for users to train their own DeepSpeech models. Users can also contribute to DeepSpeech's public training dataset through the Common Voice project. In this tutorial, we covered how to install and transcribe audio files with the Mozilla DeepSpeech library. We discussed the basic DeepSpeech example and real-time speech recognition example using Python.

AssemblyAI has released version 4 (v4) of its Topic Detection feature, which accurately predicts the topics spoken in audio/video files. The latest version is powered by an advanced deep learning neural network and demonstrates a significant 8.37% increase in relative accuracy compared to v3. This feature helps customers better understand transcribed content for various use cases such as content recommendations, internal data analysis, and advertising. The model uses the IAB Content Taxonomy with 698 common language topics when assigning topics to transcribed content.

AssemblyAI sponsored Sunhacks, a student-run hackathon at Arizona State University, where close to 300 participants built over 60 projects in under 36 hours. The top three projects using the AssemblyAI API were In-flight Closed Captioning, Speech Box, and Discord Clip. These projects aimed to solve accessibility and communication challenges faced by individuals with hearing impairments or those seeking alternative ways of communication.

OpenAI's GPT-3 is a large deep learning model with 175 billion parameters, which requires significant computational resources and time for training. Training such models on a single GPU would take hundreds of years. However, OpenAI utilized Microsoft's high-bandwidth cluster of NVIDIA V100 GPUs to train GPT-3 in weeks instead of years. The cost of setting up a similar cluster with 1,024x NVIDIA A100 GPUs is estimated at almost $10 million, not including electricity and hardware maintenance costs. Training large models is expensive and slow, which poses challenges for startups that need to iterate quickly. AssemblyAI, a startup building large Automatic Speech Recognition (ASR) models, has learned several lessons about training large models efficiently. They recommend using more GPUs, improving GPU performance, and reducing precision during training to improve iteration speed. To reduce costs, they suggest buying your own hardware or renting dedicated servers from smaller hosting providers like Cirrascale instead of relying on public clouds like AWS or Google Cloud.

Speaker Diarization is a technique used in Automatic Speech Recognition (ASR) to identify the number of speakers in an audio file and assign words spoken by each speaker accurately. It involves breaking down the audio file into utterances, converting them into embeddings using deep learning models, clustering these embeddings based on similarity, and finally labeling each word with a speaker label. This technology is useful for making transcriptions more readable and meaningful, as well as for analytical purposes such as identifying patterns or trends among individual speakers. However, current limitations include the inability to work in real-time and decreased accuracy when dealing with short speaker talk times, energetic conversations, or significant background noise.

AssemblyAI has updated its real-time Speech Recognition system, improving accuracy while maintaining the same model. The upgrades include improved training methods and vocabulary tokenization. The new system uses intermediate CTC loss and bidirectional loss for AED models, which increases the quality of lower level text representations and introduces a better understanding of language to the model. Additionally, the updated model learns the start of words instead of using a "blank" or "separator" token, making it easier to predict sentence structure.