June 2023 Summaries

In recent years, the field of generative audio models has seen rapid advancements with several notable models being developed for both music generation and text-to-speech synthesis. We will discuss some of these key developments in this article. Music Generation Models: Text-to-Music Synthesis: A growing trend among AI researchers is the development of text-to-music generative models that can produce music based on natural language descriptions, akin to how text-to-image diffusion models work. One such model, MuLan, is a transformer-based model trained on an extensive dataset consisting of soundtracks from 44 million online music videos alongside their text descriptions. It generates embeddings for the text prompt and a spectrogram of the target audio. Once trained, MuLan can either take a piece of music as input and generate textual descriptions and attributes or it can take textual descriptions as input and outputs a representation of musical elements that align with the text. Music Generation Models: Generative Adversarial Networks (GANs): Another approach to music generation is through the use of GANs, which have been successfully applied in various domains for content generation tasks. For instance, GANSynth is a generative model that uses WaveNet as its discriminator and can generate high-quality audio samples of musical notes based on random noise inputs. Speech Synthesis Models: Text-to-Speech (TTS): In the field of TTS synthesis, several breakthroughs have been made over the past few years with models like VALL-E, NaturalSpeech 2, and Voicebox showcasing exceptional performance in terms of voice cloning and naturalness. These models leverage advanced architectures such as Latent Diffusion Models and Flow-Matching for non-autoregressive audio generation tasks. In summary, generative audio models have made significant strides forward in recent years with various innovative approaches being explored across different subdomains within this field.

In this article, we will delve into the evaluation and comparison of Speech Recognition models. Proper scientific evaluation is vital when it comes to understanding the performance of these models in real-world applications. We will discuss various aspects that contribute to a proper evaluation process, including using consistent datasets, normalizers, and metrics for comparisons. Firstly, we must ensure consistency in the dataset used for evaluation across different Speech Recognition models. This means using the same public datasets or incorporating noise into them to simulate real-world conditions. By doing so, we can eliminate any potential biases that may arise from differences in the testing data itself when comparing multiple models. Secondly, it is crucial to use a consistent normalizer for evaluating different Speech Recognition models. A normalizer is responsible for standardizing transcriptions and ensuring fair comparisons between models. Therefore, using the same open-source normalizer like Whisper's normalizer is essential when comparing multiple models. Thirdly, choosing an appropriate metric for evaluation plays a significant role in determining how well a model performs in real-world applications. While Word Error Rate (WER) provides a good measure of overall performance, it fails to capture the magnitude of errors and instead counts the number of errors. In some cases, such as when proper nouns need accurate transcription, using proper noun WER may not be sufficient either. Instead, Jaro-Winkler distance can serve as an alternative metric that offers a more fine-grained notion of similarity between two strings, thus better aligning with our preferences as humans. In summary, to ensure proper scientific evaluation and comparison of Speech Recognition models, we must utilize consistent datasets, normalizers, and metrics across all evaluations. By following these guidelines, we can more accurately assess the performance of these models in real-world applications and make informed decisions when selecting a model for specific tasks or use cases.

The field of Conversation Intelligence has grown rapidly due to the increasing amount of digital conversational data from various sources. However, only a small percentage of companies analyze this data, resulting in missed strategy and revenue opportunities. AI-powered Conversation Intelligence platforms can help manage and process this data by extracting intelligent insights such as summarizing meeting outcomes or identifying common sentiments and topics. The utility of these tools has increased with the advancement of AI research and innovation. Jiminny, a leading Conversation Intelligence platform, uses AssemblyAI's state-of-the-art speech recognition model to improve their product offerings and enhance customer experience.