April 2022 Summaries

Plushcap here! We introduced virtual backgrounds to our Daily Prebuilt Electron demo, allowing participants to set their own custom background images during video calls. We created a new feature that enables users to select from pre-loaded background images or upload their own custom ones. The application uses two renderer processes: one for the main call window and another for the background selection window, which communicate through Electron's contextBridge. When the Daily call object is fully set up and ready to receive background updates, we enable the background feature by creating a background selection window and setting it to "try-enable-backgrounds". We then handle events sent from the main process to the renderer process, including setting and resetting backgrounds using Daily's updateInputSettings() API method. With this setup, users can set custom virtual backgrounds for their video calls, providing more flexibility and personalization options.

The Daily API provides methods to stream calls to external services like Amazon IVS or YouTube, and also records calls in Amazon S3. To achieve this, participants send audio and video streams to each other directly or via an intermediate infrastructure. The backend combines these streams into a single output, multiplexes them, and then streams them to the chosen service. There are two main approaches: consuming streams in a headless browser for scalability, and using a custom application with GStreamer for flexibility and control. Daily's implementation uses GStreamer to perform tasks like decoding, compositing, encoding, and streaming, offering a media pipeline architecture that can be easily extended for new features.

At Daily, they offer a video call UI called Daily Prebuilt that can be easily integrated into any web application with just a couple of lines of code. They also introduced the ability to add live captions to their domains through their startTranscription() instance method in partnership with Deepgram. This feature allows users to transcribe audio from video calls, providing wider accessibility options and creating a great way to generate meeting notes. The tutorial focuses on pairing transcription with Daily Prebuilt and covers setting up with Daily and Deepgram, embedding Daily Prebuilt in a Next.js app, creating a transcription component, adding buttons to start and stop transcription methods, loading the live captions, optimizing the app for large amounts of text, downloading the transcript, and more. The tutorial also discusses how to optimize the app's performance by using react-window and react-virtualized-auto-sizer libraries to load only the most recent messages.

Daily provides extensive resources for developers looking to build virtual event platforms using their APIs, offering guidance on whether to use Daily Prebuilt or build custom applications with daily-js's core APIs. Virtual events, encompassing conferences, webinars, and more, typically involve features such as main stage events, audience interaction, and the capacity to host large numbers of participants, with various participant roles like hosts, speakers, and audience members being managed through meeting tokens and broadcast modes. Developers are advised to consider factors like participant count limits and device restrictions to optimize call quality and accommodate large audiences, potentially reaching up to 15,000 attendees through live streaming options. Daily supports both live and pre-recorded event functionalities, with tools for enhancing user experience, such as transcription APIs and virtual backgrounds, and offers solutions for mobile and desktop applications, including React Native options. The documentation and tutorials provided by Daily aim to facilitate the development of scalable, efficient virtual event applications, with additional support available through their guides and tutorials.

At Daily, we've developed tools to seamlessly integrate video calling into any app, prioritizing mobile web support and introducing React Native support in 2020. We're now excited to announce the release of our new iOS and Android SDKs, which serve developers who want to add video or audio calls to their native apps. These libraries are built on a shared Rust core, providing benefits such as equal testing and support prioritization, synchronized release cycles, and consistent method usage patterns across platforms. With these SDKs, customers can integrate Daily into their native app with ease, and we're committed to continuing to improve the developer experience through feedback and new features, including 1:1 calls, group calls, interactive broadcasts, and more.

Kumospace is a video calling platform designed to provide an immersive and human experience for communities, teams, and friends. The platform was developed at the beginning of the pandemic as a response to the need for more engaging virtual events. Kumospace provides fun customization options, including templates for virtual offices or bars, and allows multiple conversations to happen in the same space, creating a sense of shared context. After scaling issues with their initial video API provider, Twilio, Kumospace switched to Daily, finding better pricing without sacrificing flexibility and ease of use. The switch has allowed the team to focus on building new features rather than optimizing AV code, resulting in an ideal AV experience for users.

We've been providing user-friendly WebRTC APIs to help customers build video and audio call functionality into their applications, but we haven't had true native development support until now. We're using Rust to create a central, cross-platform library called daily-core that sits behind lightweight, platform-specific API wrappers. This allows us to keep our core business logic consistent across platforms while taking full advantage of the various native platforms' capabilities. With Rust, we can write thin, "smart" types around necessary C++ types, ensuring safe memory management and handling different pointer lifetimes across all levels of the stack. We chose Rust for its multi-platform cross-compilation, memory management advantages with Rust's borrow checker, a robust type system, concurrency model, performance on-par with C++, and rich package ecosystem. These features suit our vision for daily-core well, enabling us to bring consistent core business logic, APIs that feel native to each platform, and type-safety benefits to the Web. Our long-term plans include improving the Rust codebase, building on a solid foundation to create lightweight, easy-to-use platform libraries for Daily's customers.

Daily has adopted Rust for developing a cross-platform WebRTC API platform, introducing a core library called daily-core to unify business logic across multiple platforms while providing native-feeling APIs. Rust was chosen over C++ due to its benefits, including cross-platform compatibility, robust memory management with the borrow checker, a strong type system, and performance that rivals C++. The decision to use Rust allows Daily to wrap WebRTC's C++ API with safer, richer types and to leverage Rust's concurrency model, which is beneficial for both web and mobile platforms. By compiling Rust code to WebAssembly, Daily ensures that the same type safety and performance are extended to web applications, while also benefiting from Rust's rich crate ecosystem and robust tooling for maintaining code quality. This strategic move aims to provide consistent library performance, improve platform-specific wrappers, and enhance the overall API experience for Daily's customers.

In this post, the developers at Daily showcase how their WebRTC APIs can be used to build a purpose-built e-learning application. They demonstrate several features such as moderation, transcription, whiteboarding, raising hands, and polls, which are commonly found in educational contexts. The demo is built using NextJS and showcases how these features can be integrated into a virtual class application. The developers also provide guidance on how to set up the demo locally and provide links to other resources that might come in handy for an e-learning application. Additionally, they mention their fitness app as another example of building e-learning features with Daily. They encourage readers to contact them if they have any questions about building e-learning applications with Daily.