Building Superb AI’s Synthetic Data Pipeline with NVIDIA Isaac Sim

Superb AI is developing a Multi-Target Multi-Camera (MTMC) 3D tracking system to track multiple objects across numerous cameras in large-scale environments, leveraging synthetic data to overcome the challenges of collecting and labeling extensive multi-camera video datasets. Utilizing NVIDIA's Isaac Sim ecosystem, the team has created a synthetic data generation pipeline that produces large-scale training datasets, ensuring the automatic generation of ground-truth labels and addressing the Sim-to-Real gap. The team uses advanced techniques like 3D Gaussian Splatting and a two-pass rendering workaround to optimize rendering quality and labeling accuracy independently. They have also extended the Omniverse Replicator pipeline with a custom annotator that mimics human perception to improve data quality, while a script-based domain randomization pipeline introduces diverse training scenarios by varying environmental variables. Superb AI's innovations push beyond the existing capabilities of simulation ecosystems, offering a transformative approach to managing and delivering high-quality training data for machine learning teams.