Open X-Embodiment: Curate Robot Trajectory Data in FiftyOne

Open X-Embodiment (OXE) is an open-source real-robot dataset combining more than one million trajectories from 60 datasets produced by 34 laboratories, covering 22 robot types, 527 skills, and about 160,000 language-annotated tasks. Its diversity is intended to support more generalizable robotics policies, but differing camera configurations, sensor and action formats, video properties, and metadata quality across contributing datasets make inspection and curation important before training. The article describes how Voxel51’s open-source FiftyOne toolkit can convert OXE’s Parquet and video shards into browser-based, searchable grouped datasets, representing multi-camera episodes as synchronized views, trajectory data as queryable frame-level fields, and task descriptions as searchable sample fields. A demonstration using the berkeley_fanuc_manipulation subset loads 415 two-camera episodes and 32 tasks, enabling playback, filtering, visual-similarity search, and CLIP embedding projections with UMAP to identify clusters, duplicates, outliers, scenes, tasks, and trajectory phases. It also suggests extending the workflow across OXE robot embodiments or applying robotics and perception models to locate failure cases.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.