Multimodal models need video. We froze time to give it to them

Replay Vision was developed to automate analysis of vast numbers of session recordings that are typically stored and never watched, using multimodal AI to identify user actions and problems at scale. Because session replays are not videos but rrweb event streams that reconstruct browser DOM changes, they were too large and structurally opaque for language models to interpret directly, requiring a rasterization system that converts replays into MP4 video. The team built a service using headless Chromium, Puppeteer, Temporal, and ffmpeg, but initially encountered skipped frames, freezes, and nondeterministic video lengths when rendering and capture processes competed for CPU resources. They resolved this by using Chrome’s HeadlessExperimental.beginFrame mode to synchronize rendering and capture one frame at a time, while virtualizing JavaScript time so the rrweb player advances at a fixed frame interval regardless of real-world processing delays. In production since March 2026, the deterministic rasterizer runs hundreds of concurrent jobs, has converted more than 370 years of recordings into about 3.5 million videos, and enables Replay Vision and related Session Replay features to provide multimodal AI with viewable recordings.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.