ToyBlocks: A controlled world for robot learning and evaluation

Ai2 introduces ToyBlocks, the first in a proposed series of “FoundationWorlds”: controlled simulated environments designed to make robot-learning experiments reproducible, scalable, and easier to diagnose. Centered on a robot arm manipulating labeled tabletop blocks, ToyBlocks combines realistic simulation, automatically generated tasks and demonstrations, reinforcement-learned and motion-planned skills, and detailed evaluation metrics that extend beyond simple success rates. The framework supports research on low-level vision-language-action models, high-level planning, frontier-model tool use, action-conditioned world models, and sim-to-real transfer, with initial results finding that popular action models can generalize poorly across spatial configurations while hierarchical systems can perform multistep reasoning tasks but remain vulnerable to perceptual and orientation errors. In tests with Claude-based agents, automatically learned skill descriptions improved task success beyond both default and human-written descriptions, while world-model experiments suggested that more accurate predicted rollouts can produce substantially better training data. Models trained solely in ToyBlocks achieved comparable atomic-skill performance in simulation and on a physical Franka robot, and a hierarchical Molmo 2–ManiFlow system completed half of real-world block-spelling trials. Ai2 is releasing the simulation assets, data generators, demonstrations, trained models, code, and validation sets, with plans for expanded datasets, integrated reinforcement learning, bimanual manipulation, dexterous hands, and unstructured block-pile interactions.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.