May 2024 Summaries

Mux has built Truckload, an open-source video migration framework designed to simplify heavy video migration tasks between hosting platforms. The project aims to address the challenges of long-running synchronous tasks that can be expensive, resource-intensive, and prone to errors. Truckload uses Inngest to fetch, process, and transfer massive video collections, making it easier for users to migrate their entire video collection over to Mux or other platforms. The framework supports multiple sources where video collections may be stored and allows the user to choose from different services.

Vercel's waitUntil utility method is now available for all serverless functions, allowing developers to ensure that an async function completes before the serverless function shuts down. This tool is useful when performing non-critical asynchronous work in the background without blocking the user's response. However, it does not guarantee success and should not be used for critical business logic. It is also not designed for running background jobs, which require a system that handles queueing, retries, and logs failures. Inngest can handle background jobs reliably but may have networking issues when used with waitUntil.

Inngest is an event-driven system that allows developers to build functions capable of filtering over millions of events. It offers the flexibility of filtering and matching event-driven systems like Kafka without the need for managing or optimizing the system itself. The challenge lies in creating ephemeral event matchers that can be quickly added, evaluated, and removed due to their short-lived nature. Inngest's solution involves normalizing expressions and parsing them into abstract syntax trees (ASTs), then using in-memory indexes like hashmaps and B-trees for string equality matching within expressions. This allows the system to evaluate billions of expressions in real-time, reducing latency and improving performance.

banger.show is a video app maker that heavily relies on background data processing. It's built with Inngest, which powers the video rendering pipeline for musicians, DJs, and labels. The app allows music industry people to create stunning visual assets for their music in just a few minutes without needing complex software. The video rendering pipeline includes updating render status, cropping audio files, rendering videos using remote workers, invalidating CDN cache, creating video thumbnails, and handling failures. Inngest simplifies the process by managing background jobs, orchestrating tasks, observing each step, and abstracting away lower-level processes.

banger.show is a video app maker that heavily relies on background data processing. It's built with Inngest, which powers the video rendering pipeline for musicians, DJs, and labels. The app allows music industry people to create stunning visual assets for their music in just a few minutes without needing complex software. The video rendering pipeline includes updating render status, cropping audio files, rendering videos using remote workers, invalidating CDN cache, creating video thumbnails, and handling failures. Inngest simplifies the process by managing background jobs, orchestrating tasks, observing each step, and allowing for abstraction of lower-level processes.