January 2024 Summaries

This week's update introduces the newly launched Go SDK by AssemblyAI, making it easier for Golang developers to use their API. The release joins an existing suite of SDKs which includes Python, TypeScript, JavaScript, and Java. Additionally, new blog posts cover topics such as integrating Speech Recognition into Go applications, obtaining YouTube video transcripts with Python, extracting insights from customer calls using LLMs and Python, creating speech-to-text apps with minimal or no coding, the physics behind generative AI models, and more. Sign up to receive updates on new product features and weekly tutorials & blogs.

Revenue intelligence platforms utilize AI models to automate data collection, provide comprehensive views of customer interactions, offer real-time data for decision making, align decisions with true customer opinions and actions, and connect the entire company through shared data. Speech AI technology enhances these platforms by transcribing audio and video streams, analyzing sentiments, identifying important entities, summarizing text, detecting topics in a transcription text, and generating customized summaries. The top benefits of incorporating Speech AI into revenue intelligence include automatically identifying important sections of calls, increasing efficiency on digital selling tasks, and optimizing coaching with relevant data insights to boost customer engagement and experience. By building with advanced AI models like those provided by Gong, sales teams can improve their win rates, reduce the length of sales cycles, and stay competitive in today's fast-paced market.

Large Language Models (LLMs) are revolutionizing how humans and Artificial Intelligence interact, particularly in the field of Conversation Intelligence. These platforms integrate with virtual meeting providers like Zoom and Google Meet to record calls, transcribe them, and analyze individual and aggregate call data to extract actionable insights from conversational data. LLMs can synthesize vast amounts of conversation data very quickly, making them uniquely suited for the field of Conversation Intelligence. Integrating LLMs into Conversation Intelligence platforms offers several benefits: better user experience, improved customer understanding for end users, personalized analysis tools, cost and time savings, quick modification to tools, and more intelligently informed decision-making for end users. These benefits enable product teams at Conversation Intelligence platforms to offer cutting-edge, competitive solutions to their customers.

The article discusses the integration of Speech Recognition into Go applications using either APIs or open source libraries. Due to limited options in Go, it suggests using a Speech-to-Text API like AssemblyAI's SDK for Golang, which is easy to integrate. It also provides sample code snippets for transcribing audio from local files and URLs. Alternatively, the author mentions an open source library called PocketSphinx for Go as another option but notes that it hasn't been updated in over four years.

In this weekly update, users learn about Anthropic's Claude 2.1 model now available through LeMUR, offering reduced hallucinations, an expanded context window, and improved performance in citations. Additionally, the AssemblyAI Integration for Haystack is announced, allowing users to transcribe, summarize, or extract speakers from audio files. A new AssemblyAI integration for Semantic Kernel .NET is also highlighted, enabling users to utilize transcription models by integrating with TranscribePlugin. Furthermore, resources for building with AI Speech-to-Text are provided, including no-code and low-code ways, SRT file creation, key phrase detection in audio files, and tutorials on YouTube. The update also mentions the emergent abilities of LLMs and a Python tutorial on converting Hindi speech to text. Users can sign up for updates on new product features, tutorials, and blogs.

The AssemblyAI Go SDK allows developers to integrate transcription capabilities into their Go applications. Users can transcribe audio from URLs or local files and utilize LeMUR, a framework for building LLM apps on voice data. The Go client library provides high-level operations for common use cases like polling and real-time transcription. Developers are encouraged to explore the GitHub repository for more examples and information about the SDK.

The text introduces the AssemblyAITranscriber, a component that transcribes audio data to be used with advanced LLM models and other technologies. It is integrated into the Haystack framework, an end-to-end LLM platform. By using this transcriber in a Haystack pipeline, users can transcribe audio files and access additional features such as summarization and speaker labels. The output of the transcriber is a Haystack document with metadata like transcript ID and audio URL. This integration is open source and can be accessed through GitHub or Haystack documentation.

AI applications are projected to contribute $15.7 trillion to the global economy by 2030, with 35% of businesses already utilizing AI technology. Among these applications is AI Speech-to-Text, a crucial component of Speech AI that transcribes and processes speech into readable text. This technology serves as the foundation for other AI-powered applications that process or interact with speech data, such as Audio Intelligence, Generative AI, and Large Language Models. For individuals without coding experience who are interested in building or experimenting with AI Speech-to-Text tools, there are numerous no-code and low-code integrations available today to simplify the process. The article explores nine of these simple no-code and low-code integrations and SDKs for building AI Speech-to-Text applications: AssemblyAI Python SDK, AssemblyAI JavaScript SDK, Zapier, Cloudflare, Recall, Langchain (Python and JavaScript/TypeScript), Semantic Kernel, Rivet, and Haystack. These integrations allow users to transcribe audio files into text with minimal coding knowledge. The adoption of AI Speech-to-Text technology has given rise to various use cases across different industries. Video editing platforms are using it for advanced automatic transcription, subtitling, content distribution on social channels, and more. Telehealth platforms are leveraging this technology to capture patient-doctor conversations and enhance online therapy services. Ad targeting and brand protection platforms are building robust contextual advertising and dynamic ad insertion tools by integrating AI Speech-to-Text. Sales Intelligence Platforms are using the technology for analyzing audio data, summarizing conversations, transcribing real-time calls at scale, and recapping action items from calls. Finally, call analytics platforms are adding this functionality to speed up QA processes, efficiently review calls at scale, enable context-sharing among team members, and reduce manual tasks.

This week, AssemblyAI has significantly improved its API's inference latency, allowing for transcribing an hour-long audio file within just 45 seconds and achieving a Real-Time-Factor (RTF) of up to .008x. Furthermore, they have substantially reduced their pricing for speech-to-text AI services. They have also enhanced their Free Tier Program with increased transcription limit and concurrency upgrade. In addition, the AssemblyAI Java SDK is now available for developers to transcribe audio files asynchronously or in real-time, leverage audio intelligence models, and apply LLMs to their audio data using LeMUR. Several new blog posts and YouTube tutorials have also been released, showcasing various applications of the platform.

AssemblyAI has introduced major improvements in their API's inference latency, making the majority of audio files complete within well under 45 seconds regardless of audio duration and with a Real-Time-Factor (RTF) as low as .008x. These advancements have been implemented without any compromise on accuracy, as evidenced by their Conformer-2 model achieving an industry-leading average Word Error Rate (WER) at approximately 6%. AssemblyAI has achieved this through intelligent mini batching, hardware parallelization and optimized serving infrastructure. This results in reduced pricing for both async ($0.37 per hour) and real-time ($0.47 per hour) speech-to-text models. The company also plans to release more updates over the next few months.

In this tutorial, we built a simple .NET Core application that uses the OpenAI API to demonstrate the Retriever- Augmented Generation (RAG) pattern. We used semantic search to retrieve relevant pieces of text from a podcast transcript and then used a language model to generate answers based on those retrieved pieces of text. The code in this tutorial is for demonstration purposes only and should not be used as-is in production applications. Always ensure that your application complies with the terms of service of any third-party services it uses, including OpenAI's API.

The Semantic Kernel SDK now integrates with AssemblyAI's transcription models through its TranscribePlugin. This enables developers to transcribe audio and video files using the Transcribe function, either directly or within a semantic function. By doing this, the output can be printed as a transcript of the audio file or summarized by an LLM. The plugin supports transcribing files stored on local machines and can also be used with Semantic Kernel planners. A tutorial demonstrates how to store transcripts in a vector database to avoid exceeding LLM token limits.

The AssemblyAI Java SDK is now available, allowing users to transcribe audio files both asynchronously and in real-time, utilize their audio intelligence model, and apply large language models (LLMs) through LeMUR. Key features include transcribing audio from a file or real-time input from a microphone, using LLMs for tasks like summarization on transcripts, and accessing audio intelligence models to analyze sentiment, among other capabilities. The SDK is designed with simplicity in mind, offering comprehensive documentation and usage examples within its GitHub repository.

This week's update includes information about new product features, tutorials, and community involvement for AssemblyAI. Notably, they have introduced a set of integrations with LangChain, LlamaIndex, Semantic Kernel, Zapier, Rivet, and Recall.ai to improve user experience and functionality across different platforms. Additionally, their blog provides insights into how to build an interactive lecture summarization app, transcribe audio files on Cloudflare Workers using TypeScript, and explains the concept of Qwen-Audio. Furthermore, AssemblyAI has also made their presence felt on AWS and Cloudflare. Lastly, they have shared a series of tutorials on YouTube that include understanding emergent abilities in LLMs, key AI terms for 2024, and converting speech to text using Java SDK.

A study by Owl Labs found that only 37% of companies upgraded their virtual meeting technology, with employers focusing more on training employees on effective hybrid meetings instead. Despite this, there is significant room for improvement in user experience with virtual meetings, which can be achieved through incorporating Speech AI. Speech AI includes Automatic Speech Recognition (ASR), Audio Intelligence models, and LLMs that can transcribe conversations at scale, provide meeting notes, and offer insights on those meetings. By leveraging these technologies, virtual meeting companies can enhance users' experience with their meeting software and boost usage and market share. Virtual meeting platforms can leverage Speech AI in several ways: accurately transcribing meetings, creating customized meeting summaries, and providing action items and insights for users.