June 2023 Summaries

Sci-fi enthusiast Jose Nicholas Francisco shares his experience in building a Universal Translator inspired by Star Trek using AI and LLMs. He outlines the process of creating an audio-to-audio translator, which involves converting spoken words into text, translating that text into another language, and then converting it back to speech. The author uses Deepgram's Speech-to-Text API for transcribing audio input and Google Translate API for translation purposes. While the current technology doesn't support real-time audio output, this project demonstrates how existing AI tools can be combined to create innovative solutions.

Researchers are exploring the possibility of incorporating Theory-of-Mind (ToM) into artificial intelligence (AI), which could make machines more useful by enabling them to better anticipate and understand human behavior. ToM is a human capability that allows us to infer other people's mental states, such as their beliefs, desires, intentions, and emotions. While some AI researchers argue that large language models (LLMs) have already developed "ToM-like" capabilities, others remain skeptical and believe more comprehensive testing is needed before attributing ToM to machines. The debate highlights the challenges of understanding and replicating human cognition in AI systems.

Vonage, a global cloud communications leader, has partnered with Deepgram, an automatic speech recognition (ASR) and language understanding technology provider. This partnership aims to integrate Deepgram's cutting-edge ASR technology with Vonage's Communications Platform, allowing developers to create innovative voice solutions tailored to their customers' unique needs. The collaboration will enable the rapid development of voice-enabled solutions such as virtual agents, personalized virtual assistants, call center analytics, and sales and support enablement tools. With this partnership, developers can leverage low-code/no-code options and swift deployment of rich omnichannel experiences to meet evolving customer needs and industry trends.

Deepgram offers a fully managed Whisper API that is faster, more reliable, and cheaper than OpenAI's. However, even with these improvements, Deepgram Nova is faster, cheaper, and more accurate than any Whisper model in the market (including their own). Beyond performance and cost, Whisper models lack critical features and functionality that can impede successful productization. Deepgram Smart Formatting is now available and delivers superior entity formatting results to OpenAI Whisper.

This blog post introduces four Python Notebooks that allow users to utilize Deepgram's speech-to-text capabilities without writing any code. The notebooks cover transcribing pre-recorded audio, live audio streams in real-time, creating translated subtitles for any audio stream, and labeling every speaker in transcripts (diarization). Instructions are provided within the notebooks, with accompanying video tutorials available for some of them. The author also mentions Deepgram's API Playground as an alternative way to explore the platform without coding.

Stream KeepAlive is a feature that allows developers to keep DG streams open without sending any data. This can be useful in various scenarios, such as opening a websocket before it's needed or stopping the transmission of unwanted transcribed data. The system previously closed connections if no audio data was sent for a couple of seconds, which could be frustrating for developers. To address this issue, users can now send a KeepAlive message to keep the stream open for an additional 60 seconds. This feature aims to improve the development experience and provide more flexibility in managing websocket connections.

Fountain, a podcast platform, has partnered with Deepgram to integrate AI-powered automatic speech recognition (ASR) into its service, offering highly accurate transcripts that increase podcast reach and drive audience engagement. Podcast transcripts are essential for accessibility, allowing hearing-impaired individuals or those who speak English as a second language to engage with content. They also improve discoverability by enabling search engines to index podcast content, enhance user experience through better search capabilities, and provide more options for sharing content on social media platforms. Deepgram's integration with Fountain allows users to search through transcripts and find specific points to start playing from, while podcasters can repurpose their content into different formats like blogs or newsletters. The partnership has the potential to drive significant positive changes for both podcasters and listeners alike by offering high-quality transcripts quickly and affordably, transforming the listening experience and enabling Fountain to stand out among other platforms.