April 2022 Summaries

The team behind AirNote developed a collaborative note-taking application using Augmented Reality (AR). Inspired by Apple Worldwide Developer Conference, they decided to experiment with AR and built an interactive shared workspace. They integrated Deepgram's Speech Recognition API for live transcriptions and faced challenges in learning native iOS app development, 3D modeling, and collaborating on XCode projects. The team plans to add more interaction features like drawings, file sharing, and sharing any AR object in the session. The code is available on GitHub, and feedback is welcome.

Voice technology has significantly impacted the enterprise sector by enabling contact centers to analyze customer interactions and creating meeting transcripts automatically. With 99% of respondents in the State of Voice Tech 2022 report indicating that voice technologies will be a significant part of their future plans, it's clear that this technology is here to stay. One often overlooked aspect of voice technology is its ability to improve accessibility and information access for everyone, particularly those who may have difficulty using computers or smartphones. In this blog post, we outline the top 7 ways in which voice tech supports accessibility and inclusion today. 1. Text-to-Speech: This feature turns written words into spoken language, helping people with vision problems read text on screens more easily. Ebook apps often include features that read books aloud to users, making it possible for those who are blind or have low vision to enjoy these works even if an audio version of the book doesn't exist. 2. Voice Interface Systems: Voice interfaces like Alexa and Google Home provide alternative ways to interact with technology by allowing users to ask questions in spoken language and receive audible responses. This can be particularly helpful for those who have difficulty using visual interfaces due to visual impairments, cognitive issues, or age-related challenges. 3. Smart Home Technology: Voice-controlled smart home systems enable people with mobility impairments to easily control their homes from a distance. These systems often work in conjunction with voice interface systems like Alexa and Google Home, allowing users to perform tasks such as turning lights on and off, answering the doorbell without needing to get up, opening and closing blinds, and calling for help in an emergency. 4. Real-Time Captioning and Transcription: Voice technology can be used to provide real-time captioning and transcription services, making it easier for people with hearing impairments or those who prefer written communication to follow along during conversations or presentations. This technology has improved significantly in recent years, moving towards end-to-end deep learning systems that offer nearly limitless options for implementation. 5. Voice Recognition and Control: Voice recognition technology allows users to interact with devices using spoken commands instead of relying on touchscreens or keyboards. This can be particularly helpful for individuals with physical disabilities who may have difficulty using traditional input methods. 6. Language Translation: Voice-enabled language translation services make it possible for people who speak different languages to communicate more easily, breaking down barriers and fostering greater understanding between cultures. 7. Accessibility Features in Mobile Apps: Many mobile apps now include accessibility features such as voice commands, text-to-speech options, and compatibility with screen readers, making them more accessible to users with disabilities. Overall, these seven use cases demonstrate how voice technology is creating a more inclusive and accessible world by providing alternative ways for people to interact with devices, consume information, and communicate with others.

Creating accessible audio and visual content is crucial for reaching a wider audience, including those with hearing or visual impairments, cognitive disabilities, limited dexterity, or any other reason preventing them from consuming your content. To ensure accessibility, follow these guidelines: For video content: 1. Include closed captions or transcriptions. 2. Use a clear font for captions and ensure proper color contrast. 3. Synchronize the captions with the audio. 4. Identify who is speaking. 5. Provide a clear point of focus. 6. If there's essential visual information, provide a description. 7. Avoid flashing images that can cause seizures. 8. Practice standards for high-quality visuals when possible. For audio content: 1. Provide transcripts and captions (where applicable). 2. Synchronize the captions with the audio, ensuring proper background contrast and clear font and color. 3. Identify who is speaking. 4. Reduce background noise and follow standards for recording clear audio. 5. Provide an audio description if necessary. By following these guidelines, you can create more equitable content that allows your audience to engage with and learn from it.

The text discusses six challenges in training a Dutch automatic speech recognition (ASR) model due to the language's diverse dialects, data, and dictionaries. These include inflection, compound words, different dialects, vocabulary size issues, pronunciation variations, and potential biases introduced by standardizing data. The article emphasizes that these challenges make it difficult for ASR models to accurately transcribe speech in various varieties of Dutch.

DevRel, or Developer Relations, is an interdisciplinary field that focuses on creating and nurturing relationships between a company or product and developers or developer teams. The approach to DevRel can vary depending on the team's department and leadership. Deepgram's DevRel team takes a three-pillar approach: Community, Education, and Experience. They collaborate with existing communities, create educational content, maintain their website and tools, and provide support for developers in various spaces. Their goal is to help developers become better developers by providing valuable resources and experiences.

In this article, the author discusses their experience with building a To-Do List application using Pinia and Vue 3. They first encountered Pinia while working on a full-stack live streaming web app project where they initially used Vuex for state management but later decided to explore Pinia due to its official status as the recommended state management library for Vue. The author begins by setting up their Vue project and installing Pinia, then proceeds to create three components - TodoApp.vue (the parent component), TodoForm.vue (a child component), and TodoList.vue (another child component). They explain the concept of a Pinia store as a place to hold global state properties that can be accessed by any component in the project, making it easier to track changes to that global state. Next, they demonstrate how to create a store and set up their global state with Pinia. Unlike Vuex, there is no longer the need for mutations, which simplifies the process of changing state properties. They also explain how actions in Pinia follow the same mental model as methods, data, and computed properties in Vue 2. The author then proceeds to create an initial state with a todoList property, add actions to add and delete items from the list, and set up input forms to interact with these actions. They also discuss how to access Pinia store from a component and use reactive properties in Pinia. Finally, they demonstrate how to show completed items on the to-do list by adding styles based on whether an item is marked as complete or not, and how to delete items from the list using actions. They also provide a brief overview of how to add Typescript support to their project for better error catching and type safety. Overall, the author expresses enthusiasm about Pinia due to its simplicity and compatibility with Vue 3 and Typescript.

PyCon is the largest Python conference for developers worldwide and will be held in person at Salt Lake City, Utah this year. Deepgram, an automated voice-to-text company, is one of the sponsors for PyCon US 2022. Deepgram offers real-time transcription and pre-recorded speech-to-text services with faster, more accurate transcription through AI Speech Recognition. Some use cases for Deepgram's voice-to-text technology include medical transcription, police bodycam analysis, and accessibility improvements in customer service and education.

Transfer Learning is a technique in natural language processing where a model trained on one task is used for another task by changing its training data. It started with word vectors, which are static vectors representing the relationship between words in a large abstract space. Contextual word vectors like ELMo and transformers improved this process by considering complex characteristics of word use and how they vary across linguistic contexts. Transfer learning is valuable as it allows for better representation of corpus words, improving language-related tasks such as speech recognition. Deepgram highly values transfer learning and uses it to train models for new languages by leveraging knowledge from similar languages' models.

In this tutorial, we learn how to transcribe voice calls using Twilio and Python by combining it with the Deepgram Speech Recognition API. We first set up a virtual environment for our project and install necessary dependencies such as deepgram-sdk, twilio, python-dotenv, Flask, pysondb, and 'flask[async]'. Then we create a Flask application that runs without errors and set up an endpoint to handle incoming calls. We also store our environment variables in a .env file. Next, we implement the /recordings endpoint which uses Deepgram's speech-to-text feature to transcribe the audio of our call. We then add the utterances to the calls database. Finally, we create a /transcribe route and an index.html file that displays the phone speech-to-text transcript. After making a phone call and engaging in a conversation with the provided phone number, we can hang up and see the transcript appear in our browser. This project demonstrates how to build a speech-to-text Python application using Twilio and Deepgram.

In this tutorial, learn how to transcribe messages in real-time using Flutter applications and Deepgram's Speech Recognition API. The audio is converted into data and live-streamed over WebSocket to Deepgram's servers, which then transcribe the audio and return it in JSON format back through the WebSocket. To set up this application, you will need a Deepgram API Key and follow instructions for installing Flutter onto your machine. The tutorial guides you through adding device-specific permissions, creating a user interface with three components (text area, start button, stop button), handling text state, installing dependencies, and handling audio input. Finally, test the application by running flutter run on your mobile device.

Speech recognition and understanding technologies are crucial for the deaf and hard-of-hearing (HOH) community. Automatic speech recognition can help address challenges faced by these communities, such as difficulties in communication due to masking requirements during the pandemic. Deep neural networks can convert audio inputs of spoken language into written text, which is useful for people with hearing loss and those struggling to understand speech through masks or in noisy environments. However, challenges remain, including language barriers between deaf people and hearing people, access to education and employment, and social isolation. Efforts are ongoing to develop technology that can assist the deaf and HOH community, such as converting sign language into written text.

This tutorial guides users through building an integration that allows multiple client subscribers to watch live transcripts from ongoing Twilio calls using Deepgram, a great automatic speech recognition (ASR) solution. The code for this tutorial is located here. Users will need a Twilio account with a Twilio number and a Deepgram API Key. They will also need ngrok to let Twilio access a local server. The integration involves setting up a TwiML Bin, creating a proxy server, and handling concurrent connections using Python's asyncio library.

Spelling Hero is a spelling bee simulator designed to replicate the experience of participating in a spelling competition. Users can select their desired difficulty level, and the browser will pronounce a word aloud for them to spell correctly. The game's data was manually curated, with each word including its definition, origin, type, and a sample sentence. Spelling Hero utilizes Deepgram Speech Recognition API for voice input understanding and P5.js for its user interface. The code is available on GitHub, and feedback is welcome through the project's GitHub discussions.

The article discusses optimizing technical writing content creation by following a structured lifecycle process. It emphasizes the importance of brainstorming and answering three key questions: what's your process, where do you post, and what type of content are you comfortable creating? The author suggests using various platforms to increase visibility and repurposing content in different formats such as livestreams, YouTube videos, podcasts, TikTok/Instagram reels, or wiki posts. Tools like Canva and Deepgram can assist with editing images, video transcripts, and creating content paths. The article encourages readers to share their content creation lifecycle on Twitter and provide feedback through GitHub discussions.

Sentiment Analysis and Emotion Recognition are two distinct yet related concepts in speech understanding. While both involve determining an individual's opinion or emotional state, sentiment analysis typically focuses on text-based classification tasks to determine whether the author has a positive or negative viewpoint. Common applications include analyzing customer feedback, monitoring social media conversations, and tracking brand reputation. However, challenges such as detecting sarcasm can limit its effectiveness.

In this tutorial, Sandra Rodgers demonstrates how to use Vue 3 composables to power a text-captions component that integrates with Deepgram's speech-to-text API. The post covers using async and await to write a composable that fetches a temporary API key from Deepgram, using Vue 3's watch method to react to data that is updating in real-time as Deepgram sends a text transcription back through a browser WebSocket, and writing logic that is sensitive to the order things occur - i.e., asynchronous logic that flows between the component and the composable. The tutorial assumes some knowledge of Vue 3, in particular Vue composables.