September 2022 Summaries

OpenAI's Whisper model is an open-source automatic speech recognition (ASR) tool that supports multiple languages and translation into English. It can be easily invoked at the command line, but for better scalability and integration with other software systems, an API is more suitable. This article provides a step-by-step guide on how to build an HTTP API using Python's Flask framework, which allows users to transcribe audio files by sending POST requests containing the file data. The final API can handle multiple submitted files at once and returns JSON responses with transcription results for each file.

Deepgram has released an Enhanced Portuguese Speech-to-Text Language Model (beta) for its customers in Europe and South America. The new model provides exceptional accuracy and supports both Brazilian and Portugal Portuguese pronunciations. It is available for pre-recorded and streaming phone call, meeting, voicemail, and conversational AI use cases. Customers can transcribe on-premises or through the Deepgram Cloud, with options for both Base and Enhanced model tiers. The company continues to invest in research and engineering to provide diverse language models for its customers.

The OpenAI Whisper models are a set of pre-trained deep learning models for speech recognition that can be fine-tuned on custom data. They have been trained on large amounts of diverse and publicly available datasets, making them suitable for various tasks such as transcription, translation, and voice activity detection. The Whisper models are designed to handle different languages and accents, and they support multiple output formats including text, subtitles, and timestamps. They can also be used in a variety of applications, from simple transcription tasks to more complex ones like speech-to-text conversion for video captioning or live streaming. Overall, the OpenAI Whisper models are a powerful tool for anyone working with speech recognition, especially those who need real-time transcription capabilities. However, they may not be suitable for all use cases due to their size and complexity.

Deepgram has launched a new Summarization feature as part of its Speech Understanding Features. The feature uses deep learning techniques and abstractive summarization methods to generate meaningful and relevant summaries from audio data. It is supported for English language and pre-recorded audio, and can be used by developers to create auto-generated meeting notes, podcast summaries, and sales call summaries. The Summarization API works by first preprocessing the text into logical segments, then feeding each segment individually through the model to produce a summarized version of the segment's text.

In this project, an IVR system was built using Python and Twilio, a speech recognition provider. The purpose of the system is to identify the best virtual customer support agent to respond to a call based on detected conversational language. Deepgram's speech-to-text API with language detection capabilities was utilized for this purpose. The code breakdown demonstrates how to transcribe audio MP3 files and return the transcript and detected language, which are then used to route phone calls from customers using an IVR system to the appropriate virtual customer agent who speaks their language.

The article discusses the challenge of determining when a speaker has finished talking in virtual conversations and how it can be addressed using Deepgram's real-time speech-to-text service. Two main mechanisms are provided by Deepgram to help build conversational flows: interim results and endpointing. Interim results are sent back every few seconds, indicating that more audio is being gathered and the transcription may change as additional context is given. Once enough audio has been collected for the best possible prediction, a finalized transcript is sent back. Endpointing detects the end of speech and sends an immediate message when silence is detected. The article provides code examples in Python that demonstrate how to use these mechanisms to determine when someone is done talking and respond accordingly.

OpenAI has released Whisper, a general-purpose speech recognition model, which marks a significant milestone for voice intelligence. Deepgram, a company that has been working on end-to-end deep learning for speech for over four years, congratulates OpenAI and shares the belief that voice intelligence is one of the most important areas of research in AI. To support this vision, Deepgram now offers Whisper as an additional model hosted by their API, making it available to anyone without requiring a sign-up. This collaboration between Deepgram and OpenAI highlights the growing importance of voice intelligence and its potential impact on various industries.

This tutorial demonstrates an approach for improving Interactive Voice Response (IVR) systems by gathering data on failed attempts and inferring their meaning based on successful attempts. The process involves using Deepgram with JavaScript and browser live transcription to set up live transcription in the browser, creating intents and triggers, matching user speech to intents, saving intent matches, and prompting the user to speak. The final report contains potential improvements that can be made to conversational AI intents.

Deepgram has released its Automatic Language Detection feature for speech recognition API. This feature can detect the dominant language in an audio file and transcribe it accordingly, supporting over 16 languages including Spanish, Hindi, French, German, Japanese, and Polish. The automatic language detection enhances customer experience by providing targeted services and optimizing resources. It also helps developers derive meaningful insights from audio files based on identified dominant languages in conversations. Deepgram's Automatic Language Detection feature is available in batch mode and can be used with all model tiers (base, enhanced, and custom trained).

In this project, the author demonstrates how to build a web scraper using Python and voice commands with the help of Deepgram's speech-to-text API. The process involves installing necessary Python libraries such as BeautifulSoup4 and requests, defining functions for web scraping and processing audio data, and rendering the extracted links on an HTML page. The final result is a voice-controlled web scraper that extracts links from a specified URL when the word "scrape" is spoken into the computer's microphone.

The August release of Deepgram's on-prem ASR solution includes improved diarization with enhanced accuracy and language agnosticism, as well as support for generally available language models. Users can now invoke specific models via their UUID, which is visible in the model_info JSON object or by sending an HTTP GET request to the /v2/models endpoint. Additionally, troubleshooting streaming callback responses has been made easier with the inspect_response=true parameter in the ASR request. The changelog provides a full list of changes included in this release.

This tutorial demonstrates how to track brand mentions across podcast episodes using Python and the Deepgram API. The script takes a podcast feed URL, start/end dates, and a brand name as input, then generates a report of all detected mentions. To use this script, you need to have Python installed on your machine and obtain a Deepgram API Key. The final code is provided for reference. This project can be extended by creating more complex or graphical reports, allowing multiple brands to be searched in one request, or building a user interface around the logic.

This tutorial demonstrates how to automatically transcribe podcast episodes using Pipedream and Python. The process involves creating a new workflow in Pipedream, setting up a trigger for when new podcast episodes are released via RSS feeds, and then using Python code to send the URL of the episode to Deepgram's API for transcription. The transcribed text is then sent as an email to the user. This project aims to make it easier for users to consume podcast content by providing a written transcript that can be read at their convenience.

Machine Learning (ML) has traditionally been the domain of Python engineers and data scientists, but with TensorFlow.js, front-end developers can now create machine learning models for desktop, web, mobile, and cloud applications. These models can be used for tasks such as augmented reality, sound recognition, sentiment analysis, web page optimization, and accessibility improvements. TensorFlow.js allows developers to import existing pre-trained models, retrain imported ML models using transfer learning, or create new models from scratch. By leveraging this technology, front-end developers can expand the capabilities of their applications and contribute to the development of machine learning in the web environment.

Deepgram has released a beta Base Polish Speech-to-Text Language Model for its platform, aiming to support the sixth most spoken language in the European Union (EU). The model is designed to transcribe Polish speakers accurately and can be used with various use cases such as Phone Calls, Meetings, Voicemails, and Conversational AI. Developers can leverage this model for tasks like transcribing recordings from call centers or increasing the productivity of their sales teams. Deepgram also offers high accuracy in transcription and supports both pre-recorded and streaming audio formats. The company plans to expand its EU language capabilities further in the coming months to cater to a growing global customer base.