January 2020 Summaries

Video content dominates digital outreach, with platforms like YouTube and Facebook incorporating video chat or interactivity features to boost usage and engagement. One example is Momo, the top social and dating application in China, which used Agora SDK to explore a new use case for real-time audio and video communication: "KTV Together." This online karaoke feature allows users to sing along with recorded music using their smartphones. However, challenges such as song synchronization control and high sound quality and image quality need to be addressed. Agora SDK provides solutions like high-quality audio and video experience, lower latency, better synchronization, and support for voice control. Developers can use this technology to create or enhance karaoke functions in their applications.

This blog discusses the implementation of object detection in real-time communications applications using Agora's SDKs. Object detection is a crucial component of deep learning applications and has various use cases, including football (soccer) monitoring and video call interfaces. The author demonstrates how to set up an object detection model over Agora's video call interface by utilizing the YOLO algorithm for real-time multi-object mapping. The process involves creating a new project on Agora's developer dashboard, setting up the video call with relevant app ID and channel name, and processing frames from the video call for object detection using the COCO dataset.

This tutorial demonstrates how to create an iOS application using Agora Real-Time Messaging (RTM) SDK, where users can connect to a lobby and join video chat breakout channels. The Agora RTM SDK is used as an orchestration layer for connecting users through small channels. The project setup includes creating an Xcode project for iOS and integrating the SDK. Users can create new channels or join existing ones, with real-time messaging updates on channel data. When a target number of users has joined a breakout room, they can start a video call using Agora Real-time Communication (RTC) engine. The application can be used for various purposes such as remote classrooms, virtual conferences, and social gaming.

The debate between building an AR experience using the web or a native app has been fueled by recent advancements in web technologies. JavaScript (JS) plays a significant role in both native apps and WebAR, with JS being natively interpreted by iOS and Android devices. WebAR solutions range from using sensors to more complex solutions like AR.js, TensorFlowJS, and USDZ. While WebAR has its limitations, such as lack of direct access to the ARCamera object and browser compatibility issues, it offers a frictionless experience compared to native apps. The best solution will depend on the project's specific requirements, with both web-based and native app approaches having their advantages and disadvantages.