Introducing NVIDIA Cosmos Policy for Advanced Robot Control

NVIDIA has introduced Cosmos Policy, a cutting-edge robot control policy that enhances the Cosmos Predict-2 world foundation model for manipulation tasks. This policy marks a significant advancement in robot control by integrating robot actions and future states directly into the model, achieving state-of-the-art performance on benchmarks like LIBERO and RoboCasa. Cosmos Policy leverages the pretrained Cosmos Predict model, which is adept at predicting scene evolution over time, allowing for efficient and accurate robot control without the need for separate neural networks for perception and control. This unified approach supports visuomotor control, future observation predictions, and planning, making it highly effective across diverse tasks. Cosmos Policy's efficacy is demonstrated by outperforming existing diffusion-based and vision-language-action models in various multi-task and long-horizon robotic scenarios. The Cosmos Cookoff, an open hackathon, invites developers to explore these innovations further, offering opportunities to engage with Cosmos models and win prizes by building applications and workflows in robotics and video analytics.