November 2024 Summaries

Voice content moderation is becoming increasingly important as platforms handle millions of hours of voice chat daily. Traditional methods such as manual reviews and basic keyword filters are proving inadequate, leading to the rise of AI-powered voice content moderation. This technology uses advanced language processing to understand context, detect threats, and create safer digital spaces. By combining AI's scalability with human oversight, platforms can achieve consistent, scalable moderation that still accounts for nuance. The benefits of using AI for voice content moderation include speed, cost efficiency, consistency, prevention, pattern recognition, accuracy improvement over time, and allowing human moderators to focus on complex cases.

Zoom allows users to record each meeting participant's audio separately, both locally and with cloud recordings. This feature is useful for building applications on top of Zoom recordings using Speech AI. By separating the audio tracks for each participant, it becomes easier to identify who said what in a recording and transcribe the audio files more accurately. In this tutorial, learn how to fetch Zoom participant recordings using the Zoom API and transcribe each recording simultaneously but individually using AssemblyAI's multichannel transcription. The process involves three main steps: fetching participant recordings, combining individual recordings into one recording with separate channels, and transcribing the file with multichannel transcription.

In 2024, integrating AI into business processes and products is a clear goal for many organizations, but the path to implementation can be complex. The primary barriers include technical complexity, team bandwidth, cost, speed of iteration, and data security, privacy, and compliance. Organizations must decide whether to build in-house, use open-source tools, or partner with AI providers. Each option has its pros and cons, such as full control and customization for in-house builds, flexibility and shared innovations for open-source tools, and faster time to market and dedicated support teams for AI providers. The right approach depends on an organization's unique goals, resources, and expertise. Strategic partnerships with AI providers can help businesses overcome challenges and think bigger while moving faster.

AssemblyAI has developed an AI Lakehouse solution to manage and store large volumes of audio data and metadata effectively. The primary goals of this project are to democratize data access while ensuring security and compliance, consolidate datasets across the organization in a high-quality manner, and shift dataset quality responsibility to the requester. The design of their AI Lakehouse is intended to efficiently manage, store, and serve large volumes of data, offering fast access and robust analytics capabilities. They chose Google Cloud Storage (GCS) for blob storage and Bigtable for metadata storage due to its favorable cost-to-performance ratio and compatibility with their needs. The solution they chose for integrating metadata into BigQuery is leveraging BigQuery Scheduled Queries to create a BigQuery native table from the Storage Layer every 24 hours, focusing on essential data and updating it daily. This approach provides a balance between simplicity, performance, and cost-effectiveness while maintaining heavy-duty, detailed tables for higher resolution queries if needed.

Dexa is a platform that transforms podcasts into an interactive knowledge base, making expert insights from hours of audio content instantly accessible and actionable for everyone. By democratizing access to high-level knowledge and expertise previously buried within podcasts, professionals, learners, and decision-makers can quickly access the specific insights they need when they need them. Dexa's platform leverages multiple AI capabilities to make podcast content more accessible and actionable, including automatic chapter detection and direct connections to source material for verifiable and contextual information. The impact of Dexa's platform extends across various use cases, such as professionals accessing industry insights, researchers analyzing expert discussions, learners finding specific answers from trusted sources, content creators referencing expert opinions, and decision-makers accessing relevant expert perspectives to inform their choices.

Dexa is revolutionizing the podcasting landscape by transforming podcasts into an interactive knowledge base, making expert insights instantly accessible and actionable. The platform allows users to engage with podcasts as if they were in a personal AMA session, providing immediate answers to their questions by leveraging advanced AI capabilities. Dexa's use of AssemblyAI's transcription technology enables accurate and scalable processing of vast amounts of audio content, breaking it down into topic-based segments and offering reliable speaker identification. This innovation democratizes access to high-level expertise, allowing professionals, learners, and decision-makers to extract valuable insights without sifting through entire episodes. The seamless implementation and reliable performance of the transcription solution have allowed Dexa to focus on enhancing user experience and expanding its features, setting new standards for how podcast content can be consumed and utilized in the digital age.

This tutorial explains how to implement a voice call bot using Twilio for telecommunication services, AssemblyAI for Speech-to-Text transcription, and GPT-4 for generating responses based on the transcribed text. The integration of these technologies allows for natural, conversational interactions between users and AI systems over the phone. The tutorial covers setting up the development environment, installing necessary libraries and services, configuring Twilio webhooks, handling incoming voice calls, processing voice recordings, integrating GPT-4 for generating responses, and starting the server. By following this tutorial, developers can create applications that can interact with users through voice calls, understand their spoken queries, and respond in a conversational manner.

Co-founders Sabba Keynejad and Tim Mamedov created Veed, a next-generation video editing platform focused on browser-based editing, AI tools for efficiency and creativity, and online collaboration. By leveraging AssemblyAI's language-based model, Veed generates text transcripts of videos, allowing users to edit by simply modifying the text. The platform includes powerful AI-based tools like Magic Cut, which uses the Universal-2 Speech-to-Text model for precise video editing based on transcriptions. Veed has recently surpassed $40 million in ARR.

The Google Cloud Speech-to-Text API is a service that enables developers to convert audio to text using Deep Learning models exposed through an API. It supports various audio formats and languages, offers streaming Speech-to-Text, speaker diarization, automatic punctuation and casing, word-level confidence scores, and has a usage-based pricing model. However, it may have accuracy issues, lacks feature completeness compared to some other providers, and requires strong support from the developer's side. To use Google's Speech-to-Text API in Python, you need to set up a Google Cloud project with Speech-to-Text enabled, create a service account and generate a JSON key file, set the credentials environment variable, and initialize the Speech-to-Text client in your Python code.

The rapid acceleration of Artificial Intelligence (AI) adoption across industries is driven by various factors, including early adopters' competitive advantage and the increasing accessibility of AI technology. Businesses are leveraging AI in areas such as Generative AI, Speech-to-Text (STT), and Streaming Speech-to-Text to improve productivity, streamline communication, and build tailored customer experiences. However, implementing AI requires significant organizational shifts, specialized knowledge, and adherence to strict regulations, which can be challenging for many companies. To stay ahead in the race to integrate AI, businesses need to adopt strategic moves like shifting to multimodal and making informed decisions on whether to build or buy AI solutions.

This article compares the performance of four Speech-to-Text models - Universal-2, Universal-1, Whisper large-v3, and Whisper turbo - in real-world scenarios. The evaluation focuses on proper nouns, alphanumerics, text formatting, and hallucinations. Universal-2 outperforms the other models in most categories, showing significant improvements over its predecessor, Universal-1. It has the best overall accuracy (6.68% WER), superior proper noun handling (13.87% PNER), and best formatting accuracy (10.04% U-WER). Whisper large-v3 shows some notable strengths and limitations, with the best alphanumeric transcription accuracy (3.84% WER) but also a documented propensity for hallucinations. The article concludes that Universal-2 is the leading model in most categories, offering significant improvements over its predecessor and showing a 30% reduction in hallucination rates compared to Whisper large-v3.

This tutorial teaches users how to use AssemblyAI to transcribe videos and automatically generate accurate subtitles in both SRT and VTT formats. It covers setting up the environment, choosing a subtitle format, transcribing the video file, generating subtitles, and customizing the output. The differences between SRT and VTT are also discussed, with SRT being suitable for traditional media players like VLC, while VTT is better suited for web browsers.