February 2024 Summaries

The article discusses how Speech AI models can serve as foundational building blocks for advanced AI video editing tools and platforms. It explains that AI video editing replaces traditional software to help users create high-quality videos quickly, offering benefits such as eliminating the tedious manual review process of editing videos, discovering important sections of videos to create highlights and summaries, and adding captions to videos for better accessibility and compliance. The article then delves into three main components of Speech AI: AI Speech-to-Text, Audio Intelligence, and Large Language Models (LLMs). It further explores how these components can improve AI video editing by automatically adding captions, supporting video searchability and indexing, and unlocking insights for smarter collaboration.

In this tutorial, you'll learn how to build an application that uses Speech AI to transcribe a phone call and identify scam attempts. The application will use AssemblyAI for real-time transcription and LeMUR for summarizing the transcripts. By the end of this tutorial, you'll be able to transcribe a phone call in real-time using Twilio and AssemblyAI, and assess whether it's a scam attempt or not using LeMUR. The complete code example is provided at the end of the text.

Call tracking solutions are becoming increasingly popular among marketing teams due to their ability to provide higher conversion rates than other forms of engagement. These solutions offer a suite of AI-powered call tracking automation tools that ease the manual tasks associated with mass phone calls, such as note taking and logging CRM data. Key features include Dynamic Number Insertion (DNI) for lead tracking, form tracking, lead valuing, call flows, lead searching/filtering, lead qualifications, custom reporting, and more. The integration of Speech AI models into these tools has significantly improved their accuracy and efficiency, allowing companies to gain a greater competitive advantage in the market.

This week's update includes information about new features for real-time transcription, including custom silence threshold and utterance control. These enhancements are valuable for latency-sensitive applications like voice bots. Additionally, the blog discusses AI trends in 2024, focusing on Graph Neural Networks and their potential impact on productionized AI models. The text also covers AI music generators, such as MusicLM, MusicGen, and Stable Audio, exploring how they work and their technical challenges. Furthermore, there are tutorials for transcribing phone calls in real-time using Node.js with AssemblyAI and Twilio, creating videos from text input using Python, indexing podcasts based on keywords, and understanding the physics behind Generative AI models.

The article discusses various Speech-to-Text APIs, AI models, and open-source engines available in the market today. It highlights three popular Speech-to-Text APIs with a free tier - AssemblyAI, Google, and AWS Transcribe. It also explores several open-source Speech-to-Text libraries such as DeepSpeech, Kaldi, Flashlight ASR, SpeechBrain, Coqui, and Whisper. The article emphasizes the importance of considering factors like accuracy, model design, features, support options, documentation, security, and more while choosing a suitable option for one's project. It concludes by suggesting that the best free Speech-to-Text API, AI model, or open-source engine will depend on the specific needs of the project.

Graph Neural Networks (GNNs) have become increasingly important in the field of AI, with companies like Uber, Google, Alibaba, Pinterest, and Twitter shifting to GNN-based approaches. GNNs are designed for handling graph data, which is present in various systems consisting of entities and their relationships. Graph machine learning has seen a significant rise within the research community, with an exponential growth of published research involving GNNs. Some key applications of GNNs include recommendation systems, traffic predictions, weather forecasting, data mining, materials science, drug discovery, and protein design. The use of GNNs in these diverse domains demonstrates their potential to revolutionize AI in the coming years.

This week's update introduces PII Redaction and Entity Detection in 13 new languages, including Spanish, Finnish, French, German, Hindi, Italian, Korean, Polish, Portuguese, Russian, Turkish, Ukrainian, and Vietnamese. These features automatically find and hide personal details like names and email addresses in transcripts to protect privacy, and spot and categorize important data in audio files. Additionally, new blog posts cover real-time phone call transcription using Go with Twilio and AssemblyAI, RLHF vs RLAIF for language model alignment, and transcribing a phone call in real-time using Python with AssemblyAI and Twilio. The latest YouTube tutorials focus on Graph Neural Networks applications, indexing podcasts with keywords, and the physics of generative AI.

In this tutorial, a Flask application is built that transcribes phone calls in real-time using Twilio and AssemblyAI. The process involves configuring a user's call to be forwarded to the AssemblyAI transcription service, which sends partial transcripts as they are received. These transcripts are then printed to the terminal. The application is set up with an ngrok tunnel that exposes the Flask app to Twilio without requiring cloud machine provisioning or firewall modifications. The code for this tutorial can be found on GitHub.

In this tutorial, you will learn how to build a server application in Go that transcribes an incoming Twilio phone call into text in real-time. You will use a Twilio MediaStream to stream the voice data to your local server application and then pass the voice data to AssemblyAI for transcription. By the end of this tutorial, you will be able to record a phone call in real-time using a Twilio MediaStream connection and transcribe audio in real-time using the AssemblyAI Go SDK. To complete this tutorial, you will need an upgraded AssemblyAI account and Go installed on your system.

This week's update introduces new features to the Python SDK for real-time transcription, including temporary authentication support and Mu-law encoding. Additionally, users can now achieve real-time speech-to-text with microphone input in just a few lines of code using AssemblyAI. The latest blog posts cover how to do Speech-To-Text with Go, get YouTube video transcripts, and extract phone call insights with LLMs in Python. Furthermore, the company has released new tutorials on real-time transcription in Python, developing a talking AI chatbot with Text-to-Speech, and indexing podcasts with keywords like on Huberman's website.

Conversation intelligence, powered by Artificial Intelligence (AI), helps companies analyze conversational data. It uses AI models like Automatic Speech Recognition (ASR), Audio Intelligence, and Large Language Models (LLMs) to transcribe human speech into readable text, infer intent or sentiment, surface named entities, sort conversations into chapters or summaries, identify common topics or themes, and more. Conversation intelligence is important because it helps companies record and analyze all desired customer interactions, replacing tedious manual processes. It generates high ROI for users by providing intelligent, actionable insights based on customer data. Some top use cases of conversation intelligence include automating meeting transcription and analysis, searching and indexing voice conversations at scale, and surfacing actionable insights.

This weekly update provides information on new product features, tutorials, and community updates. It highlights LeMUR, a tool that makes it easy to apply Large Language Models (LLMs) to audio and video data. Users can transcribe audio files using Python code with an API key and then use LeMUR to summarize the transcript, answer questions about the audio, or generate tags, titles, and descriptions. The update also features blog posts on speech-to-text in Go, getting YouTube video transcripts, and extracting phone call insights with LLMs. Additionally, there are two trending YouTube tutorials: indexing podcasts with keywords like on Huberman's website and live speech-to-text with Google Docs using LLMs. The update concludes with a discussion on the physics of Generative AI.