October 2022 Summaries

The text discusses a solution for transcribing only the necessary audio data using Python and a speech-to-text API like Deepgram. It suggests using a buffer or queue to store audio data before the WebSocket connection is opened, allowing continuous transcription even after the connection is made. The implementation involves installing required packages, defining callback functions, and creating context managers for connecting to the Deepgram WebSocket server. The code also includes functions for sending audio data to the WebSocket and receiving transcriptions from it.

Deepgram has introduced a new feature that enables real-time numeral formatting adjustments during an audio stream using JSON Configure messages. This allows users to enable or disable numeral formatting at any point in the stream by sending a configuration message with "numerals": true or false, respectively. Currently, this is the only feature that can be turned on and off mid-stream, but Deepgram plans to add more flexibility to its real-time API in the future.

In this project, we analyzed several episodes of Lex Fridman’s podcast using Automatic Speech Recognition (ASR) and Natural Language Processing (NLP). We started by downloading the audio files from YouTube using youtube_dl. Then, we transcribed these audio files into text format using Deepgram's ASR API. After obtaining the transcriptions, we performed several analyses on them. Firstly, we analyzed the time spent speaking and words spoken by Lex Fridman and his guests. We found that Lex speaks for less time but says a similar number of words as his guests. Next, we used The Text API to identify common phrases mentioned in each podcast episode. This analysis revealed that Lex frequently uses adjectives such as "beautiful", "poetic", "fascinating", and "loving". We also analyzed the subjects discussed in these podcasts. Finally, we used The Text API's Named Entity Recognition (NER) feature to extract named entities from the transcribed podcasts. This helped us understand who or what was being talked about during each episode. Additionally, we generated summaries of each podcast episode using The Text API's AI text summarizer. Overall, this project demonstrated how ASR and NLP can be used together to analyze podcast transcripts effectively.

In this tutorial, a virtual presentation coaching application is built using Recall API and Node.js. The application allows users to add a Recall bot into a Zoom call, remove it, and get insights once the call is over. The key steps include setting up an express application, creating a reusable recall() helper method, adding bots to a Zoom call, making bots leave a Zoom call, showing data from calls, getting all speaker usernames, displaying each speaker's turn count, and calculating each speaker's speaking time. The application can be further enhanced by performing additional analysis using the data returned by Recall and Deepgram.

Deepgram has announced an update to its streaming API for enhanced usability. The new feature allows users to close streaming connections by sending a JSON message containing the type "CloseStream". This ensures that all transcripts are received and prevents unnecessary charges for untranscribed audio. Previously, closing connections was done using empty bytes, which is now deprecated. Deepgram will release additional JSON messages in the coming months to unlock more features in its real-time API.

Automatic Speech Recognition (ASR) and Natural Language Understanding (NLU) are being used by social media platforms to transcribe audio and video content, providing valuable information about user interactions. This technology is used in various ways such as closed captioning, add-on analytics, improved ad targeting, enhanced search functionality, insights & automation, among others. ASR solutions help save money, generate new revenue, and increase productivity for social media companies by unlocking the potential of audio and video content.

OpenAI's Whisper is a new automatic speech recognition (ASR) model based on transformers and trained on 680,000 hours of audio data. Deepgram researchers have found that Whisper performs best with GPU access. Google Colab provides an ideal environment for testing Whisper without needing to configure it for GPU usage. A simple notebook has been set up in Google Colab to install Whisper, download audio from YouTube, transcribe the audio, playback segments for evaluation, and compute Word Error Rate (WER) for transcription accuracy. Users can easily upload their own files to test Whisper's performance.

Keywords are a way of specifying that certain words are expected to appear in a conversation, which can help increase the accuracy of speech recognition models without having to train them on all possible audio scenarios and special vocabulary. Deepgram's deep learning platform utilizes keyword information to make more accurate predictions. Customers who want better performance on product names or industry-specific vocabulary can use keywords to increase the chances that Deepgram's model will predict those words instead of more common ones. Keyword boosting is useful for improving speech recognition accuracy in specific contexts, but a custom model trained on representative data will always give the best performance.

OpenAI has released Whisper, a general-purpose speech recognition model designed for AI researchers studying robustness, generalization, capabilities, biases and constraints of the current model. While Whisper's larger models show impressive accuracy, Deepgram's speech-to-text API is built for software developers to create highly scalable, production-quality products using voice. For companies that build with Whisper, they must essentially commit to rebuilding the wheel that speech processing companies like Deepgram have been refining for years as their sole mission. Deepgram allows users to avoid these disruptions and go straight to production with reliable, accurate, cost-efficient, fast, and feature-rich speech processing tools.

OpenAI has released a new open-source ASR model called Whisper, along with a repository of tools that make it easy to try out. Users can experiment with various options for inference and observe how they impact results. The Whisper paper describes its complex decoding strategy, which includes several heuristics aimed at making transcription more reliable. These strategies are currently implemented in the code, resulting in slightly improved test results but slowing down inference by up to six times. By default, the Whisper CLI tool runs inference and decoding up to six times with different decoding strategies. Users can adjust these settings to improve performance for their specific data. The model also struggles with periods of non-speech, which could be addressed using a voice activity detection algorithm in parallel with Whisper or by adjusting the compression ratio threshold.

Deepgram has launched a Topic Detection feature as part of its speech understanding offerings. The unsupervised topic modeling technique helps developers and customers detect important topics discussed in conversations. This feature is useful for analyzing audio data, improving search capabilities, automating workflows, and enhancing user experience. It supports over 350 topics and can be enabled using the detect_topics=true parameter in API calls. The output includes a list of detected topics with confidence scores and word positions. Developers can use this information to build downstream workflows, generate tags based on topics, power analytics tools, or integrate with other applications.

Deepgram and Recall.ai have partnered to streamline the process of extracting data from virtual meetings for developers. The partnership aims to make it easier for developers to gain insights from voice data and automate tedious workflows, using minimal lines of code. Recall.ai provides a single Developer API to access real-time meeting data from various platforms such as Zoom, Google Meet, Microsoft Teams, and more. The partnership enables efficient feedback flows for developers, providing users with unparalleled accuracy, speed, and scale for their meeting platforms at a fraction of the cost.

Code-switching is a communicative behavior where two or more speakers use words and phrases from different languages while conversing. This phenomenon can lead to language change, even the creation of new languages. Examples include Spanglish (a mix of Spanish and English), Hinglish (Hindi and English), and Taglish (Tagalog and English). Code-switching is not simply mixing two languages but rather using words, phrases, grammar, and other cultural elements from different languages to communicate more fully. It often occurs in multilingual communities where speakers have knowledge of multiple languages or dialects. AI researchers need to consider code-switching when building language models as it can significantly impact the accuracy of speech recognition systems.

Hacktoberfest 2022 is a month-long event where participants contribute to open-source projects on platforms like GitHub or GitLab. Deepgram's DevRel team has opted into Hacktoberfest with several Open Source projects, including SDKs for various programming languages and a Chrome extension for live transcription and translation. Community members are also participating in the event by creating their own Deepgram-related projects. Events related to Hacktoberfest are planned, and participants can share their open source projects on Deepgram's Open Source Project discussion. Feedback is welcome through GitHub discussions.

In this tutorial, learn how to use Twilio Functions and Deepgram's summarize feature to automatically transcribe, summarize, and send phone call summaries via SMS once a conversation has ended. The process involves setting up Twilio Functions, recording and forwarding inbound calls, transcribing and summarizing the call using Deepgram's Node.js SDK, and sending summary messages to both parties involved in the call.

The article provides a step-by-step guide on how to run the OpenAI Whisper tool on your machine using the command line. It starts with setting up a virtual environment and installing necessary dependencies such as Python, torch, numpy, transformers, tqdm, more-itertools, and ffmpeg-python. The author then demonstrates how to use the tool by transcribing an audio file of Nicholas Tesla's vision of a wireless future. Additionally, it introduces Deepgram's Whisper API endpoint as an alternative for those who want to avoid the complexities of setting up and running OpenAI Whisper locally. The article also highlights that transcript results may vary between runs due to non-deterministic nature of these models.

This tutorial demonstrates how to use the retext libraries and Node.js to detect potentially problematic language in text or markdown files. The process involves installing necessary dependencies, setting up a Deepgram API key, generating a transcript with Deepgram, and creating a reusable function to check for non-inclusive language using retextProfanities and retextEquality plugins. The results are then logged to the console as suggestions for improvement. This method can be used to identify and change usage patterns of non-inclusive language in various contexts.