June 2024 Summaries

AssemblyAI introduces Conversational Intelligence AI to help businesses navigate the vast amount of digital conversational data generated daily. The company offers various capabilities such as sentiment analysis, topic detection, auto chapters, and key phrases extraction from audio data. In addition, they have added input_tokens and output_tokens keys to LeMUR responses for users to track their usage more effectively. AssemblyAI also provides tutorials on hotword detection with streaming speech-to-text, transcribing YouTube videos with Node.js, speaker diarization libraries and APIs, and building an AI voice translator.

This tutorial guides you through creating a personal AI assistant using hotword detection with the help of AssemblyAI's Streaming Speech-to-Text API and Go programming language. The process involves setting up an environment, recording audio data from the microphone, transcribing it in real time, and implementing hotword detection to trigger the AI. The final application can be used for various purposes such as transcription, analytics, or accessibility solutions.

This guide teaches how to transcribe YouTube videos using Node.js and AssemblyAI. It covers setting up a development environment, retrieving audio from a YouTube video, transcribing the audio with AssemblyAI, saving transcripts and subtitles, and prompting a YouTube video using LeMUR. The process involves installing necessary NPM modules, configuring an AssemblyAI API key, retrieving audio URLs from YouTube videos, transcribing audio files, generating SRT subtitles, and running the script to obtain transcript text and subtitle files. Additionally, it demonstrates how to prompt a YouTube video using LeMUR for tasks such as summarizing the video with bullet points.

AssemblyAI's weekly update highlights the latest improvements in their Speaker Diarization service, which is now up to 13% more accurate and available in 16 languages including Chinese, Hindi, Japanese, Korean, and Vietnamese. The company also introduced billing alerts for better spend management and improved PII and Entity Detection models. Additionally, the Universal-1 model now supports German language transcription. New blog posts cover creating SRT files for videos in Node.js, 10 ways live transcription is being used today, and a tutorial on building an AI chatbot in Java with real-time prompting.

AssemblyAI has recently updated its Speaker diarization service, improving accuracy by up to 13% and adding support for five additional languages. The new Speaker Diarization model demonstrates a 10.1% improvement on Diarization Error Rate (DER) and an 85.4% reduction in speaker count errors. These improvements stem from recent upgrades, including the new Speech Recognition model Universal-1, an improved embedding model, and increased input sampling frequency. The enhanced service is now available for testing via a no-code Playground or by using AssemblyAI's Python SDK with an API key.

This post guides developers on how to deploy a ready-made Speech AI application to the cloud using Encore and the new AssemblyAI starter app for Go. The app allows transcribing local audio files of conversations with Speaker Diarization enabled, enabling users to follow what each speaker said and when. Encore is a development platform that simplifies building and operating APIs and event-driven systems. The AssemblyAI starter app for Encore lets developers bootstrap their own Speech AI app with a single command. The app consists of three components: a frontend built with React and Tailwind CSS, a backend built with Go and the AssemblyAI Go SDK, and a PostgreSQL database to store finished transcripts.

The text provides a detailed guide on how to create WebVTT files for videos using Node.js and the AssemblyAI API. It outlines the steps required, including setting up the development environment, transcribing the video, generating the WebVTT file, and running the script. Additionally, it explains how to customize the maximum number of characters per caption and provides an example of a generated WebVTT subtitle format. The guide also suggests further actions such as configuring the subtitle file in a video player or uploading it to YouTube Studio.

This guide explains how to create SRT files for videos using Node.js and the AssemblyAI API. First, set up your development environment by installing Node.js 18 or higher, creating a project folder, initializing a new Node.js project, and installing the AssemblyAI JavaScript SDK. Next, obtain an AssemblyAI API key and configure it as an environment variable on your machine. Then, transcribe your video file using the AssemblyAI SDK and generate the subtitles in SRT format. Finally, run the script to create the SRT file and customize the maximum number of characters per caption if needed.

This weekly update introduces new features in Universal-1, a multilingual Speech-to-Text (STT) model that now supports German language transcription with high accuracy. Additionally, improved PII Text Redaction and Entity Detection models have been released for English, expanding their language support to include Chinese, Dutch, Japanese, and Georgian. The blog offers tutorials on using these features in Node.js and Python, while YouTube tutorials cover topics such as transcribing large amounts of audio and coding an AI voice bot from scratch.

Streaming Speech-to-Text technology, also known as live transcription, converts real-time audio streams into accurate text. Industries such as financial services, healthcare, customer support, and market research are using this technology to drive engagement, improve accessibility, and generate immediate insights. Live transcription is becoming faster, more reliable, and more accurate, expanding its list of practical use cases. Some examples include live broadcasts, virtual meetings and conferences, customer service and support, education and online learning, legal proceedings, healthcare and telemedicine, financial services, government and public sector, market research and focus groups, and AI-powered live assistants.

This tutorial teaches how to redact Personally Identifiable Information (PII) from audio and video files using the AssemblyAI JavaScript SDK in a Node.js environment. The process involves setting up the development environment, transcribing audio or video files with PII redaction enabled, getting the PII-redacted audio file, and running the script. The final output is a redacted transcript text and an associated redacted audio file where specified PII policies are replaced with hashes or beeps.

This week's update highlights AssemblyAI's integrations with various platforms to enhance speech AI workflows, including LangChain for analyzing audio dialogues, LlamaIndex for building searchable audio archives, Twilio for enhancing call transcription and analysis, Haystack for detailed audio analysis in large language model workflows, and AWS for scaling audio analysis projects. Additionally, the blog features tutorials on summarizing audio with LLMs in Node.js, filtering profanity from audio files using Node.js, content moderation on audio files with Python, and YouTube tutorials on coding an AI voice bot from scratch, testing LLMs with audio data, and building a talking AI with LLAMA 3 (Python tutorial).

In this tutorial, learn how to use Large Language Models (LLMs) with Node.js to summarize audio using LeMUR and AssemblyAI. First, set up the environment by installing Node.js 18 or higher and configuring an AssemblyAI API key. Next, transcribe the audio file using the Universal-1 model from AssemblyAI and prompt LeMUR to summarize it with a specified format. Finally, run the script to see the summary printed in the terminal. The same code can be used for automatic summarization of audio files by passing in an audio file instead of a video URL.