June 2022 Summaries

Natural Language Understanding (NLU) and Natural Language Processing (NLP) tools, such as Topic Detection, can process large amounts of data to extract valuable insights from social media platforms, news sites, podcasts, etc. These tools work by segmenting texts into smaller parts, analyzing each segment against a standardized taxonomy like the IAB Taxonomy, and outputting a summary score. Several APIs for Topic Detection are available on the market, such as AssemblyAI's, Amazon Comprehend's, TextRazor's, Azure Cognitive Services', MeaningCloud's, and uClassify's. The use cases of these tools span across Conversation Intelligence Platforms, Customer Research Platforms, and Revenue Intelligence Platforms to automate data analysis, gain customer insights, and optimize sales coaching among others.

In response to the issue of ad placement next to inappropriate content on video sharing platforms, AI-backed Content Moderation APIs have been developed. These models analyze texts and audio streams to detect sensitive or offensive issues and return probability scores as well as severity ratings. Some of the top APIs for this purpose include AssemblyAI's Content Moderation API, Microsoft’s Azure Content Moderator, Amazon Rekognition, DeepAI’s Content Moderation API, and Hive Moderation. These APIs are used across a range of industries and platforms to protect advertisers, maintain brand reputation, increase user engagement, and ensure community safety.

Imagen is a text-to-image generation model developed by Google Brain that demonstrates state-of-the-art performance in generating high quality and diverse images. It utilizes large language models (T5) for encoding the input text prompts, which are then used to condition a series of diffusion models for image synthesis. These models were trained on two corpora: a large dataset of web-scraped image-text pairs and a smaller dataset containing high quality images. The model achieved impressive results in human evaluations against other state-of-the-art text-to-image generation models such as DALL-E 2, GLIDE, VQGAN+CLIP, and Laten Diffusion. It outperforms all of these models on a comprehensive set of challenging prompts called DrawBench. The key takeaways from the Imagen paper suggest that scaling up the text encoder is very effective, dynamic thresholding is critical, noise conditioning augmentation in the super-resolution models is critical, text conditioning via cross attention is critical, and efficient U-Net design is important for achieving high performance in image generation tasks. Reference(s): [1] Sahariah, A., Steiner, T., Anderson, J., et al. (2022). Imagen: High Resolution Image Synthesis with Text-Guided Diffusion Models. arXiv preprint arXiv:2205.11493. [2] Dhariwal, P., and Nichol, A. (2021). Diffusion Models Beat GANs on Image Synthesis. arXiv preprint arXiv:2106.07748.

This week's Deep Learning Paper Recaps feature two significant research works, namely "Bridging the gap between streaming and non-streaming ASR systems by distilling ensembles of CTC and RNN-T models" and "BRIO: Bringing Order to Abstractive Summarization". The first paper focuses on improving streaming automatic speech recognition (ASR) models using knowledge from non-streaming models, resulting in a significant reduction in Word Error Rate for Spanish, Portuguese, and French. The second paper proposes a novel training method for abstractive summarization that involves assigning probability mass to candidates based on their quality, leading to new state-of-the-art results on several well-known datasets.

The paper "TOXIGEN: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection" presents the creation of a large machine-generated dataset containing 274k toxic and benign statements, making it the largest hate speech detection dataset to date. The authors demonstrate that this dataset can improve fine-tuning performance when used alongside other implicit toxic datasets. Additionally, the paper "Knowledge Distillation Meets Open-Set Semi-Supervised Learning" explores how Knowledge Distillation methods can compress powerful Deep Learning models by using student's representations to learn from teacher's outputs and improve generalization on unseen data. Both papers contribute valuable insights for training Content Moderation models and improving the efficiency of Deep Learning models through knowledge distillation.

During HawkHacks 2022, AssemblyAI sponsored the event by providing its Speech-to-Text and Audio Intelligence APIs to attendees. Three notable projects emerged from this collaboration: Speech2Code, a web application that translates spoken commands into written coding languages; Vid2Text, an app for generating custom transcriptions of audio/video files; and Vivy.ai, an AI for Discord capable of verbal responses and speech-to-text transcription.

This week's Deep Learning Paper Reviews feature two notable papers: Decision Transformer and SPIRAL. Decision Transformer casts Reinforcement Learning (RL) as a sequence modeling task, outperforming behavior cloning and being competitive with State-of-the-Art Temporal Difference learning techniques. It is particularly capable in tasks involving long-term credit assignment. On the other hand, SPIRAL offers an alternative ASR speech pre-training method that claims to reduce training costs by 80% compared to wav2vec2 for the base model and 65% for the large model. Through aggressive downsampling, SPIRAL matches the performance of wav2vec2 while handling noisy inputs better, making it an attractive alternative SSL pre-training method.

Vania King is a professional tennis player who has had an impressive career, having won multiple WTA titles. In this interview, she talks about her journey in the sport and how she first met fellow tennis player Lauren Davis. They also discuss their involvement with the Diversity and Inclusion committee for the USDA and share details about a charity exhibition tournament that they will be participating in next month. Additionally, Vania King mentions an exciting achievement - within her first WTI 1000 doubles tournament at the Italian Open, she was able to win the championship title.

We have successfully transcribed audio files into text using ESPnet's pretrained models for Automatic Speech Recognition (ASR). The process involved converting audio files to .wav format, if not already in that format, and then running them through the speech2text object. Preprocessing was also performed on the resulting transcriptions by removing punctuation and converting all text to uppercase using a helper function named "text_normalizer". The final transcriptions were compared with their corresponding true transcriptions for accuracy.

The use of Speech-to-Text (STT) and Audio Intelligence APIs can significantly enhance the user experience of hybrid events by providing automatic transcription, accessibility support, and improved searchability. These tools leverage advanced Artificial Intelligence (AI), Machine Learning (ML), and Deep Learning techniques to ensure accurate speech recognition and provide additional insights such as entity detection. As a result, companies can create more engaging and accessible virtual and hybrid event experiences for their participants.