December 2021 Summaries

IronScribe is a fully-fledged transcription service built using AssemblyAI's Speech-to-Text transcription API. The app allows users to upload videos, transcribe them, and edit the captions easily. It also enables video clippings with captions "burned in" for social media use. Inspired by conversations with video content providers seeking a better way to transcribe their content, IronScribe was built with AssemblyAI's API due to its accuracy and features. The app offers a great editing UI, clippings (shorts), and respects user privacy by not exposing files to third parties. A demo of the product is available on their YouTube channel, and users can sign up for a free trial on their website.

GPT-3, a groundbreaking development in artificial intelligence, has captured widespread attention due to its unique capabilities and the excitement it generates for the future of AI. The video explores what makes GPT-3 stand out, delving into its architecture to provide insights into its impressive performance and potential risks.

In this PyTorch tutorial, we explore Torchaudio and its functionality for working with audio data. Key topics covered include loading and saving audio files, transformations, resampling, data augmentation, feature extraction, and using Torchaudio datasets. The tutorial provides a comprehensive guide to effectively handling audio data in machine learning projects.

In 2021, AssemblyAI made significant strides in improving its speech recognition technology and expanding its feature set. The company released v8 of its transcription model, which boasted up to 18.72% better accuracy across all types of audio and video data. They also introduced nine major new features such as real-time transcription, entity detection, auto chapters (summarization), sentiment analysis, filler words, severity scores for content safety, word search, paragraph detection, and usage alerts. AssemblyAI also launched a developer dashboard, public changelog, YouTube channel with weekly machine learning tutorials, and expanded its social media presence. The company's research team published several papers on end-to-end architectures for speech recognition, Transducer models, and various AI topics. Additionally, AssemblyAI was recognized as both a Fall and Winter 2021 High Performer and Momentum Leader on G2.

AssemblyAI has released its latest feature, Sentiment Analysis, which is now part of their Speech-to-Text API. This feature classifies content as positive, negative, or neutral and can be used in various applications such as analyzing customer-agent conversations, virtual meetings, news stories, and more. The sentiment analysis results are returned for each sentence spoken in the audio file along with a confidence score.

In this tutorial video, a Streamlit application is created to summarize podcast episodes. The ListenNotes API is utilized to fetch podcast information, while the AssemblyAI API is employed for transcribing the podcast and generating Auto Chapters.

The paper "JUST - JOINT UNSUPERVISED AND SUPERVISED TRAINING FOR MULTILINGUAL ASR" presents a novel Wav2Vec2-inspired pre-training technique for multilingual automatic speech recognition (ASR). JUST utilizes a five-stage modeling architecture with three stage-level unsupervised and supervised loss functions. The proposed approach achieves a 32% performance increase over the first-stage Wav2Vec2 XLSR network in low-resource language ASR settings. Key findings include the use of contrastive MLM (Masked Language Modelling) and RNN-T losses for joint pre-training on audio-text pairs across multiple languages, leading to more useful information extraction, better generalization, and robust contextualized token prediction. JUST outperforms Wav2Vec2 by using only the MLS dataset for pre-training, demonstrating its effectiveness in multilingual ASR tasks with fewer data requirements.

AssemblyAI has received two G2 awards for Winter 2022 in the Voice Recognition Software category, including Momentum Leader and High Performer. The company holds one of the top spots with an average customer rating of 4.8 out of 5 stars. As a Deep Learning company, AssemblyAI focuses on advanced neural networks and Machine Learning technology to become an industry leader in Speech Recognition. Their API offers powerful features such as Automatic Transcription, Real-time Transcription, Speaker Diarization, Topic Detection, Sentiment Analysis, PII Redaction, Entity Detection, Summarization, and more.

PyTorch and TensorFlow are the leading frameworks for deep learning, each with distinct strengths and evolving capabilities that influence their suitability for various applications. PyTorch has become the preferred choice in the research community due to its rapid adoption and dominance in research publications, facilitated by its user-friendly interface and extensive model availability through platforms like HuggingFace. In contrast, TensorFlow is favored in industry settings for its robust deployment capabilities and comprehensive ecosystem, which includes TensorFlow Serving and TensorFlow Lite for efficient model deployment across various platforms. While PyTorch is closing the gap in deployment with tools like TorchServe, TensorFlow's integration with Google Cloud and its end-to-end machine learning solutions continue to make it a strong choice for industrial applications. Ultimately, the decision between PyTorch and TensorFlow depends on specific use cases, with PyTorch excelling in research and TensorFlow offering advantages for production and deployment.

The debate over which deep learning framework is superior - PyTorch or TensorFlow - remains a complex and nuanced discussion. Both frameworks have matured exponentially since their inceptions, making many of the technical differences between them vestigial at this point. Practical considerations such as model availability, deployment infrastructure, and ecosystems now play a more significant role in determining which framework is best suited for a given domain. In terms of model availability, PyTorch currently dominates the research landscape due to its wide adoption by the community and extensive range of available models on platforms like HuggingFace. However, TensorFlow still holds an advantage when it comes to deployment infrastructure, with robust tools such as TensorFlow Serving and TensorFlow Lite allowing for easy deployment on clouds, servers, mobile devices, and IoT/embedded devices. The ecosystems surrounding each framework also play a crucial role in determining their utility. PyTorch boasts an array of libraries tailored to specific problem domains like Computer Vision (TorchVision), Natural Language Processing (TorchText), and Audio processing (TorchAudio). On the other hand, TensorFlow offers end-to-end platforms for model deployment (TensorFlow Extended) and a comprehensive toolkit for building multimodal, cross-platform applied Machine Learning pipelines (MediaPipe). In 2023, both PyTorch and TensorFlow are very mature frameworks with good documentation, many learning resources, and active communities. While the choice between them may depend on specific use cases and preferences, it is clear that both frameworks have a significant role to play in the future of deep learning research and applications.

AssemblyAI offers a way to extract valuable insights from audio or video files by providing sentiment analysis for company earnings calls through a web application. Users can input recordings of earnings calls, and the app will deliver insights into the sentiment expressed during the call. This process involves transcribing the audio and analyzing the sentiment, enabling users to gain deeper understanding from unstructured data.

Activation functions in neural networks are explored in this video, highlighting their definition, purpose, and various types, such as Step Functions, Sigmoid, TanH, ReLU, Leaky ReLU, and Softmax. The explanation includes how these functions are implemented in code, emphasizing their importance in transforming input signals within neural networks to introduce non-linearity, enabling the models to learn complex patterns.

The paper "VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning" presents a new self-supervised learning technique that does not rely on contrastive samples, unlike previous methods. Instead, it uses three regularization terms to maintain variance and decorrelate features in the learned representations. The method is theoretically sound and empirically performs better than contrastive techniques. It shares similar performance with other non-contrastive techniques but offers greater potential due to its simplicity and theoretical transparency.

PyTorch Lightning is a high-level framework built on top of PyTorch that abstracts away boilerplate code, making it easier for researchers and developers to focus on building great models. It provides benefits such as faster experimentation, scalability, and ease of use by organizing the training process in an object-oriented manner. In this tutorial, we explored the differences between Lightning and ordinary PyTorch, understood the Lightning workflow, and built a GAN model to see Lightning in action. The main benefits of using PyTorch Lightning include automation, lack of hardware references, removal of boilerplate code, and inclusion of best practices from top AI labs.

This benchmark report compares the transcription accuracy of AssemblyAI, Google Cloud Speech-to-Text, and AWS Transcribe on various audio use cases. The dataset includes a wide range of accents, audio quality, number of speakers, and industry-specific vocabularies from product demos, tutorial videos, documentaries, podcasts, sports talk radio, and corporate earnings calls. Accuracy is measured using Word Error Rate (WER), which compares the API-generated transcription to human transcriptions. AssemblyAI also provides additional enrichment models such as Topic Detection, Keyword Detection, PII Redaction, Content Safety Detection, Sentiment Analysis, Auto Chapters, and Entity Detection. The report concludes by offering complimentary benchmark reports for any team seeking a transcription solution.

AssemblyAI has released a new feature for its Speech-to-Text API called Entity Detection, which identifies and categorizes key information in text. This feature can detect entities such as names of people, organizations, addresses, phone numbers, social security numbers, locations, and more. The process involves identifying entities and classifying them into categories like "person_name," "phone_number," or "occupation." Entity Detection is useful for various industries, including telephony and CRM platforms, hiring platforms, virtual meeting platforms, voice bots, and medical applications. By collecting this entity information, companies can perform analytics to improve customer understanding, adjust marketing campaigns, modify products, and more.