MotionGraphicsBench: Evaluating Motion Graphics Generation with Programmatic Verification
Blog post from Hugging Face
MotionGraphicsBench is an expert-curated benchmark designed to isolate and evaluate AI models’ ability to animate existing vector graphics rather than generate graphics and motion from a single text prompt. Its 100 tasks provide a complete static SVG and a natural-language motion-editing prompt covering 17 types of spatial, temporal, appearance, and visibility requirements, while task-specific deterministic Python verification programs assess rendered SVG frames through human-readable checks. Across 32 proprietary and open-weight language models from 12 providers, pass@1 scores ranged from 6.0% to 67.7%, with GPT-6 Astra at maximum reasoning effort leading the evaluation and only four models surpassing 50%; the strongest open-weight models remained below 40%. Repeated attempts substantially improved results, with pass@32 ranging from 38.0% to 95.0%, but analyses identified frequent failures in matching requested poses, following motion trajectories, and coordinating related events. Built from real-world Lottie animations and supported by human-reviewed prompts, outputs, and labels, the benchmark’s verification programs achieved 93.3% agreement with human judgments on 180 held-out animations, outperforming an evaluated vision-language-model judge at 83.3%, and the authors have released the tasks, code, verification tools, model outputs, and annotations publicly.
| Trend | Post Mentions | Total Month Mentions | Posts | Companies | MoM |
|---|---|---|---|---|---|
| LLM | 7 | 3,355 | 742 | 191 | -49% |
| GPT-6 Astra | 5 | 173 | 46 | 27 | -48% |
| Multi-agent systems | 2 | 153 | 84 | 47 | -53% |
| Gemini 3.8 Flash | 1 | 281 | 36 | 21 | +37% |
Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.