VLAs and World Models Need Web-Scale Data. Just Not the Same Data

A recent event at Bright Data’s Web Data Loft in San Francisco brought together engineers from leading robotics and AI companies to explore the transition from language models to real-world robotic applications. The discussion, moderated by Adam of HackerSquad and the Builders Collective, highlighted the significance of training corpora in developing Vision-Language-Action (VLA) models, emphasizing that model architecture is not the sole bottleneck. VLAs begin as vision-language models trained on large-scale internet data before being fine-tuned with robotic data, allowing for better generalization. The conversation also covered the integration of vision, language, and action into a unified token space, the distinct data needs of VLAs versus world models, and the hierarchy of data sources for training. A consensus emerged that broad web-scale data offers foundational world understanding, while robot-specific data is crucial for execution. Participants noted the absence of reliable scaling laws for robotics akin to those in LLMs, underscoring the importance of rapid data curation and discovery for effective model training.