The First Healthcare Robotics Dataset and Foundational Physical AI Models for Healthcare Robotics

Open-H-Embodiment is a groundbreaking community-driven initiative that has launched the first open dataset specifically for healthcare robotics, addressing the need for physical AI in the field. Developed by a steering committee and involving 35 organizations worldwide, the dataset comprises 778 hours of training data for surgical robotics, ultrasound, and colonoscopy tasks, and is used to train and evaluate AI autonomy and world foundation models. Two permissively open-source models, GR00T-H and Cosmos-H-Surgical-Simulator, have been developed using this data, focusing on surgical robotics. GR00T-H is a Vision-Language-Action model designed for surgical tasks and incorporates unique design choices to handle hardware challenges, while Cosmos-H-Surgical-Simulator is a world foundation model that addresses the complexities of sim-to-real challenges in surgical robotics simulation. Future plans for the Open-H-Embodiment project aim to enhance reasoning capabilities in surgical robotics, encouraging community involvement to develop reasoning-ready data that captures intents and outcomes.