October 2024 Summaries

The October '24 AI, Machine Learning, and Computer Vision Meetup featured presentations on accelerating machine learning research and development for autonomy, utilizing 2D image representations in applied robotics, and PostgreSQL for innovative vector search. Key discussion points included hardware used in autonomous vehicles, embedding techniques for images, tools for labeling, challenges with sensor data expansion, and the use of depth estimation methods for stationary objects. The next Meetup is scheduled for November 14th, featuring speakers on human-in-the-loop AI systems, strategies for optimizing visual AI datasets, and deploying ML models on edge devices using Qualcomm AI Hub.

CoTracker3 is a state-of-the-art point tracking model that achieves impressive performance with significantly less training data compared to previous models. It introduces architectural simplifications, a novel semi-supervised training pipeline, and improved efficiency. By leveraging multiple pre-trained trackers as teachers to generate pseudo-labels for real-world videos, CoTracker3 reduces the dependency on synthetic data and achieves even better accuracy with a thousandfold reduction in real data requirements. The model highlights the power of considering dependencies between tracked points and utilizing context to enhance tracking accuracy.

CoTracker3 is a point tracking model designed to predict individual points' trajectories throughout a video sequence, even when they are occluded or out of the camera's view. It stands out due to its ability to effectively leverage real-world videos during training, resulting in state-of-the-art performance on the point tracking task. CoTracker3 overcomes limitations faced by other SOTA point trackers that rely heavily on synthetic datasets for training by employing a semi-supervised training approach incorporating unlabeled real-world videos. This is achieved by using multiple existing point trackers (trained on synthetic data) as "teachers" to generate pseudo-labels for the unlabeled videos, allowing CoTracker3 to bridge the gap between synthetic and real-world data distributions. The model comes in two flavors: online and offline, with the latter offering better performance, especially for tracking occluded points by interpolating trajectories through occlusions using the entire video context. To run inference with CoTracker3 and parse the output into FiftyOne format, one can follow these steps: 1. Install the required libraries (FiftyOne and imageio[ffmpeg]). 2. Download a dataset from Hugging Face or another source. 3. Explore the dataset using the FiftyOne app. 4. Choose between online and offline modes for CoTracker3, with the latter being used in this tutorial. 5. Download the model from the Torch hub. 6. Prepare a video for inference by converting it to a tensor. 7. Load the video as a tensor and move it to the GPU if available. 8. Set the grid_size parameter, which determines the number of points tracked within a video frame. 9. Run inference using the model and obtain two tensors: pred_tracks (point coordinates) and pred_visibility (point visibility). 10. Visualize the model output using the CoTracker repository's visualization code. 11. Parse the model output into FiftyOne format by converting it to keypoint objects for each frame, adding them to the sample at once, and updating the sample with a "tracked_keypoints" field containing an fo.Keypoints object for each frame. 12. Run inference on the whole dataset using the appropriate batch size and grid_size parameters. 13. Set configurations for the FiftyOne app to visualize the output, such as coloring by instance and looping videos. By following these steps, one can effectively use CoTracker3 with FiftyOne to track points in video sequences and visualize the results using the FiftyOne app.

The text introduces a free course on Coursera called "Hands-on Data Centric Visual AI" by Harpreet Sahota from Voxel51. This course covers various aspects of computer vision, including labeling approaches, annotation quality, bounding box quality, advanced tools like FiftyOne and CVAT, complex challenges, and data augmentation techniques. The text also mentions an upcoming hackathon in NYC on October 27th for ML enthusiasts and college students to tackle real-world challenges in the field of computer vision and data-centric AI. Tech talks, hands-on workshops, engaging challenges, networking opportunities, and exciting prizes are expected at the event.

The October '24 AI, Machine Learning and Computer Vision Meetup featured presentations on various topics such as how Renault leveraged machine learning to scale electric vehicle sales, RGB-X model development for multimodal machine learning, and using Elasticsearch vector search for computer vision use cases. The meetup also included a Q&A session with the speakers and provided information about upcoming events and ways to get involved in the Computer Vision Meetups.

FiftyOne 1.0 has been released, marking a significant milestone for developers working with visual data. The open-source platform now includes new features such as fully open-source ML techniques, modal panels, change tracking, remote zoo models and datasets, and more. Model updates and community contributions have also been made to enhance the overall functionality of FiftyOne. Additionally, an upcoming hackathon in NYC on October 27th will provide opportunities for participants to tackle real-world challenges in AI, network with industry experts, and potentially win cash prizes.

Voxel51, creators of the software refinery for machine learning scientists and engineers, has made its AI/ML software, called FiftyOne, fully open source. The company believes that open source AI greatly outweighs the risks, offering benefits such as collaboration and innovation, safety through transparency and trust, customization and adaptability, and no vendor lock-in. By making their AI work open source, Voxel51 aims to foster a more diverse range of applications across industries and domains, while also enabling public oversight, community-based governance, and collaborative standards for safety, privacy, and fairness in AI systems.

Version 1.0 of FiftyOne open source has been released, marking a significant milestone in providing an open-source foundation for building highly performant visual AI systems. The platform has over 2 million installs and is trusted by hundreds of thousands of AI builders. It supports various data formats, types, and sources used in visual AI projects, including the recent addition of 3D scenes. The release also reaffirms the commitment to full-stack open-source AI.

Voxel51 has announced the release of version 1.0 of its open-source tool, FiftyOne, which is designed to enhance the development of visual AI applications by offering an integrated solution for handling diverse visual data. This milestone release aims to address challenges faced by AI builders, such as the lack of tools capable of managing large volumes and various types of visual data, which often result in delays and failures in AI projects. FiftyOne provides a flexible, extensible framework that supports the development of visual AI models by integrating features such as native vector search with Elasticsearch and open-source machine learning techniques for assessing data uniqueness and similarity. The new version underscores Voxel51's commitment to open-source AI, promoting transparency and collaboration within the AI ecosystem. FiftyOne is already being used by numerous enterprises, including LG Electronics and Berkshire Grey, to enhance model accuracy and team productivity, illustrating its growing influence in the computer vision community.