October 2024 Summaries

Agora Middleware Service is an open-source project that simplifies integrating Agora's cloud services into existing real-time voice and video applications. It provides a set of RESTful APIs for token generation, cloud recording management, and real-time transcription, allowing developers to focus on delivering great user experiences while offloading complex server-side builds. The service exposes endpoints for Token Generation, Cloud Recording, Real-Time Transcription, and RTMP services, enabling developers to manage these features in their applications. To get started with the middleware service, developers can clone the repository, set up their environment, and run the service using Go or Docker and Make. With the service running, developers can test its APIs using cURL commands or tools like Bruno or Postman, and follow best practices for optimal performance and troubleshooting.

This guide outlines the implementation of Real-Time Speech-To-Text functionality in a backend built with Astro, using Agora's RESTful API to transcribe video calls in real-time and store them for later use. The backend generates tokens to secure video calls, records video calls, and transcribes them using Agora's speech-to-text service, allowing developers to build applications that can utilize this feature without exposing sensitive data directly to the application. The implementation includes generating a credential, creating a resource, and making requests to the Agora API to start and stop transcription sessions, as well as handling errors and responses from the API.

In today’s digital-first world, developers need a reliable way to integrate real-time chat capabilities into their applications. Agora's Chat SDK provides a powerful solution that supports multiple platforms and offers advanced features such as end-to-end encryption, translation, content moderation, and more. With its user-friendly interface and straightforward initialization process, developers can quickly get started with integrating chat functionality into their apps, regardless of the platform they're developing for. Agora’s Chat SDK also includes a range of tools to help manage and secure conversations, such as token management and support resources.

Agora has released its Conversational AI SDK for OpenAI, integrating with the Realtime API to enable natural voice interaction with AI, allowing for lifelike conversations and understanding human emotion. This integration combines Agora's real-time audio capabilities with conversational intelligence, enabling developers to create lifelike voice-driven conversational AI experiences. The direct integration with OpenAI makes it easier for developers to build AI voice agents and experiences, while also addressing challenges such as latency and wireless last mile issues with its Software-Defined Real-Time Network (SD-RTNTM). This network has been powering real-time voice interaction for over 10 years and serves users in 200+ countries and regions. The new SDK enables a wide range of use cases, including customer support, language learning, mental health and wellness coaching, interactive AI-powered tutoring and learning sessions, intuitive hands-free interactions, and engaging real-time voice conversations on social media and in games.