June 2021 Summaries

In this tutorial, the process of fine-tuning two different transformer models, BERT and DistilBERT, for two NLP problems - sentiment analysis using Stanford Sentiment Treebank v2 (SST2) dataset and duplicate question detection using Quora Question Pairs (QQP) dataset is demonstrated. The HuggingFace Transformers Repo provides over 60 different network types including pre-trained models like DistilBERT and BERT, which are used in this tutorial. Training these models involves creating PyTorch datasets and dataloaders for the training and validation sets, defining a loss function, and implementing functions for model training/evaluation. The results show that fine-tuning pre-trained Transformers on downstream tasks can save significant time with performance often immediately high out of the box.

This report evaluates the accuracy of Speech Recognition technology from AssemblyAI, AWS Transcribe, and Google Speech-to-Text on 12 audio/video files from various sources. The Word Error Rate (WER) is used to measure the accuracy of each transcription API. Additionally, the results of AssemblyAI's unique Content Safety, Topic Recognition, and Keyword Detection features are reviewed. This report aims to serve as a point of reference for comparing the best Automated Speech Recognition solutions in the market.

Product managers and developers at telephony companies are increasingly utilizing automatic speech recognition (ASR) to enhance their products' core features. Examples of such applications include Interactive Voice Response (IVR), Virtual Voicemail, Call Transcription, Call Tracking, Coaching Enablement, and Conversational Intelligence. The accuracy of a Speech-to-Text system is crucial for these telephony platforms to create high-quality features that users and customers appreciate. This report examines the performance of three ASR providers - AssemblyAI, AWS Transcribe, and Google Speech-to-Text - in transcribing earnings call recordings from five major companies: Twilio, Facebook, Apple, Microsoft, and MongoDB. The evaluation includes not only accuracy but also features like Personal Identifiable Information (PII) Redaction, Topic Recognition, Keyword Detection, and Content Safety. The results showcase the strengths and weaknesses of each provider, providing valuable insights for companies seeking to integrate ASR solutions into their telephony platforms.

AssemblyAI has open-sourced a Drone plugin called drone-deploy-ecs that enables deploying updates to Amazon ECS. The plugin is opinionated and may not fit everyone's needs perfectly. The team chose to use ECS as their deployment controller due to its built-in rollbacks via the deployment circuit breaker. Writing the plugin was simple, with Drone exposing plugin settings as environment variables. The plugin only modifies a single container within a Task Definition and has specific IAM permissions requirements. Currently, only one container in a given Task Definition can be updated at a time, but they are open to adding support for updating multiple containers.

This month, Voice Search added a plugin for Algolia's InstantSearch.js library, enabling developers to easily integrate voice search into their websites or apps. The company also made significant improvements in accuracy and turnaround time for phone calls, videos, and podcasts. Additionally, two new security features were introduced: PII Audio Redaction, which removes personal identifiable information from audio playback, and Profanity Filtering, which replaces curse words with "#" characters in transcripts.