Rebuilding the Data Foundation for Embodied AI with Lance: From Long Videos to Random-Access-Friendly Multimodal Samples

Folding clothes, a simple task for humans, presents significant challenges in embodied AI due to the amorphous and deformable nature of garments, requiring advanced coordination, perception, and recovery capabilities from robotic systems. A closed-loop data iteration pipeline is crucial for training AI to handle such tasks, involving data collection, training, deployment, and trajectory sampling, supported by a unified data infrastructure. The LeRobot format, popular in robotics, faces issues with data organization and access due to its reliance on Parquet files, which are not optimized for frequent modifications and random access. Lance offers a solution by reorganizing data into GOP-scale blobs stored in S3, allowing direct read and write operations without duplicating data across devices, which significantly reduces the cost of modifications and improves read/write performance. This approach aligns data at the highest frequency of robot control, preserving high-frequency action data while maintaining video compression efficiency. Experiments show that Lance's design, particularly with a GOP size around 8, provides a favorable balance between storage efficiency and random read throughput, offering a performance improvement of 1.7 to 6 times over traditional methods like LeRobot. The China Merchants Lion Rock AI Lab is pioneering research in embodied AI, focusing on integrating robots with large models to enhance perception, reasoning, and execution capabilities.

No tracked trend matches for this post yet.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.