Content Deep Dive

MotionGraphicsBench: Evaluating Motion Graphics Generation with Programmatic Verification

Blog post from Hugging Face

Post Details
Company
Date Published
Author
Jiaju Ma, Ashvini Kumar Jindal, Chuhua Wang, and Sumithra Bhakthavatsalam
Word Count
4,771
Company Posts That Month
60
Language
-
Hacker News Points
-
Post removed?
No
Summary

MotionGraphicsBench is an expert-curated benchmark designed to isolate and evaluate AI models’ ability to animate existing vector graphics rather than generate graphics and motion from a single text prompt. Its 100 tasks provide a complete static SVG and a natural-language motion-editing prompt covering 17 types of spatial, temporal, appearance, and visibility requirements, while task-specific deterministic Python verification programs assess rendered SVG frames through human-readable checks. Across 32 proprietary and open-weight language models from 12 providers, pass@1 scores ranged from 6.0% to 67.7%, with GPT-6 Astra at maximum reasoning effort leading the evaluation and only four models surpassing 50%; the strongest open-weight models remained below 40%. Repeated attempts substantially improved results, with pass@32 ranging from 38.0% to 95.0%, but analyses identified frequent failures in matching requested poses, following motion trajectories, and coordinating related events. Built from real-world Lottie animations and supported by human-reviewed prompts, outputs, and labels, the benchmark’s verification programs achieved 93.3% agreement with human judgments on 180 held-out animations, outperforming an evaluated vision-language-model judge at 83.3%, and the authors have released the tasks, code, verification tools, model outputs, and annotations publicly.

Trends Found in this Post
Trend Post Mentions Total Month Mentions Posts Companies MoM
LLM 7 3,355 742 191 -49%
GPT-6 Astra 5 173 46 27 -48%
Multi-agent systems 2 153 84 47 -53%
Gemini 3.8 Flash 1 281 36 21 +37%
Use This Data

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.