April 2022 Summaries

Gmail's intent analysis magic behind separating emails into different mailboxes relies on natural language processing (NLP) techniques, which can also be applied to various other applications such as product success exploration, real-time transcription, and contextual insights. NLU is used for more dynamic voice interactions, creating a structure from unstructured data to define content. Implementing NLU using Python with tools like JohnSnowLabs' nlu and scikit-learn can help analyze text data from sources like Reddit reviews. Intent analysis has practical use cases such as client classification, intent marketing, spam detection, and modern uses in various industries, but also has limitations including incorrect results due to unstructured data and human behavior nuances. Tools like Symbl.ai offer custom trackers and APIs for building pipelines and gathering insights from voice and video data.

Extracting raw audio streams from Zoom meetings can be beneficial for various applications such as real-time sentiment analysis, transcription, and capturing action items. To achieve this, developers need to use the Symbl.ai SDK, which provides a JavaScript library to connect with Zoom's PSTN mode. The process involves initializing the SDK, setting up the DTMF sequence, and establishing a connection with the meeting. Once connected, the developer can subscribe to the live events on the connection to receive real-time transcripts, which can be used for various purposes such as sentiment analysis or action item capture. By using raw audio streams, developers can work with unaltered data, providing better opportunities for reading and analyzing the audio in its original form.

This application demonstrates how to build and use a multi-party video-conferencing application using Twillio and Symbl.ai's Real-time APIs, combining the features of both platforms to create a more inclusive and intelligent video conferencing experience. The application integrates with Symbl.ai's Real-time APIs for out-of-the-box conversational intelligence features such as Live Closed Captioning, Real-time Transcription, Video conferencing with real-time video and audio, Enable/Disable camera, Mute/unmute mic, Screen sharing, Dominant Speaker indicator, and Network Quality Indicator. The application is currently supported only on Google Chrome and can be run locally using a set of instructions that include cloning the repository, installing dependencies, storing Symbl and Twilio credentials in environment files, and running the app using a command.

Symbl.ai's Web SDK simplifies the process of handling WebSocket connections and managing audio from multiple devices in web applications. The SDK takes care of the bi-directional handshake and network-related issues, allowing developers to focus on core application functionality. With the Web SDK, developers can easily manage multiple WebSocket connections by creating multiple instances of the Symbl() object and handle device changes with automatic device handling mechanism that updates codec and sample rates appropriately. Additionally, the SDK supports a wide range of audio codecs and sample rates, and allows developers to input custom audio streams with only a few lines of code. By using the Web SDK, developers can build applications quickly and efficiently.

Symbl.ai has partnered with Agora.io to provide developers with new ways to enhance their Real-Time Engagements through Conversation Intelligence (CI) features. Symbl.ai's Extension enables developers to add CI features directly within their Android mobile applications, providing a seamless integration with Agora.io's Video SDK. To get started, developers need to sign up for a free developer account, set up their project, and activate the extension in the Extensions Marketplace. Once activated, they can clone or download the sample Android app, update its configuration, build the app, and deploy it to an Android device. The app provides features such as live captioning, transcription, action items, questions, follow-ups, topics, trackers, and more, allowing developers to create a comprehensive Real-Time Engagement solution. Symbl.ai also invites developers to join their community, participate in hackathons, and access resources on how to set up the app for production.

Agora.io is a platform that allows developers to embed real-time video and audio streaming into their apps. It provides APIs compatible with various platforms, enabling the addition of real-time engagement (RTE) to apps. Agora.io's Web Voice SDK offers features like voice recording and voice calls, making it suitable for web applications. The platform has partnered with Symbl.ai to provide AI-powered services such as speech recognition, conversation analytics, and contextual insights. By utilizing Agora.io's SDKs and services, developers can easily add real-time audio and video capabilities to their apps, enhancing user engagement without the need for extensive development efforts.