What we learned from our first 400k Replay Vision scans

Replay Vision is PostHog’s AI layer for Session Replay, using configurable scanners to watch individual recordings and produce observations such as labels, scores, summaries, or yes-or-no findings with cited evidence. After analyzing more than 400,000 recordings internally, PostHog found that useful scanners require a narrow, observable question, a carefully targeted set of relevant sessions, and the ability to return “no” or “inconclusive” rather than generate speculative findings. The guidance emphasizes that humans must define product-relevant questions and validate results, while queries, prompts, calibration on small batches, and model selection determine the quality and cost of automated analysis. Examples include identifying users who abandon error investigation for AI help, detecting product contradictions and dead ends, finding feature opportunities, analyzing experiment behavior, reviewing sessions before low-NPS responses, and understanding high-value users’ first sessions. Digests and Scouts are intended to synthesize patterns across many observations, while individual scanners remain focused on evidence from one recording at a time.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.