August 2024 Summaries

FiftyOne 0.25 has been released with several game-changing features, including Python Panels, Custom Dashboards, SAM 2 integration, and Elasticsearch Integration, which enhance visual AI development capabilities. The latest release includes built-in components for information display, tutorials, interactive graphs, triggering operations, and more. Additionally, FiftyOne is hosting an upcoming hackathon at the University of South Carolina, Northwestern University, and NYU, offering tech talks, hands-on workshops, engaging challenges, networking opportunities, and exciting prizes. Participants will have to curate a subset of the provided dataset that achieves two contradictory goals: reducing the overall size while maintaining or improving performance of a YOLOv8m object detection model.

NVIDIA has released a new 8 billion parameter language model called Mistral-NeMo-Minitron-8B, which demonstrates exceptional performance across various natural language processing tasks and is more efficient than training from scratch. Microsoft's Phi-3.5 models have also been released, showcasing independent AI development capabilities with open-source licensing. The UniBench framework provides a comprehensive evaluation framework for vision-language models (VLMs), addressing the fragmented landscape of VLM benchmarks. FiftyOne has updated to version 0.25.0, featuring new features such as Python Panels and Custom Dashboards. Eugene Yan has released a blog post discussing LLM-evaluators and their potential in evaluating the quality of responses generated by large language models. The "AI Scientist" system has been developed for fully automated scientific discovery using foundation models like large language models, automating the entire research process from generating ideas to writing papers and conducting peer reviews.

FiftyOne 0.25 has been released with significant upgrades to its representational power and extensibility, including Python Panels for building custom App panels via a simple Python interface, Custom Dashboards for displaying domain-specific analyses, SAM 2 for zero-shot segmentation in images and videos, and Elasticsearch Integration for text and image similarity queries. The new release provides more ways to customize the FiftyOne environment and surface key insights into data and models, advancing the platform's mission to accelerate and automate visual AI development.

Segment Anything 2 (SAM 2) is a major leap forward in segmentation technology, offering cutting-edge performance in both images and videos. It builds on the foundation of the original Segment Anything, which Meta released in April 2023, enhancing image segmentation while introducing advanced video capabilities. SAM 2 enables precise segmentation and tracking in video sequences using simple prompts like bounding boxes or points from a single frame, opening up exciting possibilities for various video applications. FiftyOne makes it easy to work with visual data by seamlessly generating segmentation labels and visualizing them on datasets. Users can download SAM 2 models and run inference on their FiftyOne datasets directly from the FiftyOne Model Zoo using simple commands. The tutorial demonstrates how to load a dataset, provide bounding box prompts to generate segmentations, use keypoints instead of bounding boxes, and even automatically generate masks for whole images without any prompts. For videos, SAM 2's segmentation and tracking capabilities can be used to propagate masks from one frame to another seamlessly. The tutorial concludes with the available models in the FiftyOne Model Zoo and suggestions for getting started with FiftyOne and its Visual AI features.

Voxel51 is celebrating its 4th anniversary of Open Source FiftyOne, a project that has garnered 8,000 GitHub stars, 100+ open source contributors, 3,000 community Slack members, and 26,000+ members in AI + CV Meetups. The company's tools are accelerating the transformation of industries with Visual AI, enabling AI systems to perceive, reason, and make informed decisions based on visual data. Voxel51 has also published two blogs: one by Brian Moore on What is Visual AI?, which explains how it goes beyond traditional computer vision, and another by Robert Wright and Allen Lee on How I Built an In-Cabin Perception Dataset, which showcases the ease of building a dataset and loading it into FiftyOne. Additionally, Voxel51 has launched a data-centric AI competition on Hugging Face, where participants are challenged to curate a subset of a provided dataset while maintaining or improving object detection model performance. The company is also hosting various events and competitions, including a Visual AI challenge with prizes for the winners.

The August '24 AI, Machine Learning and Computer Vision Meetup featured presentations on SyntStereo2Real, a lightweight I2I translation framework for stereo pairs, and Elevating Security with Data-Centric AI: A Comprehensive Approach to Surveillance. The meetup also included a Q&A session where attendees asked questions about the use of GANs in remote sensing image translation and the development of data science projects with open source tools. The event aimed to bring together communities of data scientists, machine learning engineers, and open source enthusiasts who want to share and expand their knowledge of AI and complementary technologies. Upcoming Meetups include talks on reducing hallucinations in chatGPT-like AI systems and a data-centric AI competition on Hugging Face Spaces.

The open source tool FiftyOne has celebrated its fourth anniversary and has experienced tremendous growth in its community since its launch four years ago. The FiftyOne community has grown to over 8,000 GitHub stars, with more than 100 contributors working on the project, and over 3,000 members of the Slack community. Additionally, the FiftyOne community has reached 26,000+ members of AI + CV Meetups, and there are over 20 success stories from users around the world about how FiftyOne helps accelerate their Visual AI projects. The tool has had 15 new releases in the past year, including features such as native support for full 3D scenes, custom workspaces, and integrations with other ML tools. FiftyOne was also named a winner in the 2024 Artificial Intelligence Excellence Awards and has raised $30M in series B funding to continue accelerating its roadmap.

The latest trends in AI, machine learning, and computer vision are being showcased at the 2024 Olympic Games in Paris, where cutting-edge technology is transforming various aspects of the event. Artificial intelligence is also being applied to athlete support, refereeing, and enhancing the viewer experience. Additionally, OpenAI co-founder John Schulman has left the company to join rival AI firm Anthropic, highlighting ongoing controversies surrounding OpenAI. The article discusses whether we are in a bubble for AI investments, suggesting that while similarities with past tech bubbles exist, there are crucial differences that suggest this time might be different. It also highlights innovative approaches to tackling high-dimensional data challenges and introduces the concept of "tail performance" in AI systems, which is critical for real performance. Furthermore, the article touches on recent releases from Meta, including SAM 2's data engine, which offers a masterclass in data engineering for computer vision.

The meetup was held in August 2024, where attendees could help guide a $200 donation to charitable causes by voting for their favorite charity. The event featured presentations and Q&A sessions on various topics including GenAI for video, evaluating RAG models for LLMs, and end-to-end LLM evaluations with in-house data. The speakers discussed recent advancements and challenges in these fields, highlighting the potential impact of GenAI on video editing and generation, and the importance of evaluating LLM performance using custom data. The meetup also covered strategies for integrating and performing evaluations of Large Language Models within a CI/CD pipeline, and explored ways to fine-tune in-house LLMs as judges using custom data. The event aimed to bring together communities of data scientists, machine learning engineers, and open source enthusiasts to share and expand their knowledge of AI and complementary technologies. Upcoming meetups were announced, including speakers on edge-aware GAN for remote sensing image translation, elevating security with data-centric AI, and developing data science projects with open source tools.

Visual AI is a subset of AI that processes visual data to provide humans, physical systems, or software with insights for informed decision-making and action. It utilizes machine learning models and high-quality data to effectively reason and act, extracting meaning from visual inputs, understanding content, and taking appropriate action. Visual AI is important because it brings new capabilities and augments human vision in remarkable ways, including efficiency, scalability, enhanced capabilities, safety, automation, decision support, and transforming life as we know it. It works by combining models with high-quality data to reason and act, and its applications include autonomous robots, assistive apps, medical systems, apps that generate new images or videos, and more.

The Voxel51 Data-Centric AI competition aims to challenge the status quo in AI development by focusing on data curation. The competition encourages participants to optimize data rather than just model architecture, with the goal of reducing dataset size while maintaining or improving object detection model performance. Participants will work with a large dataset and an open-source tool called FiftyOne for dataset curation and analysis. The competition offers prizes for top performers, including cash rewards and community swag packages, as well as opportunities for skill development, real-world impact, and recognition in the AI community.

The inaugural Data-Centric AI competition is now live on Hugging Face Spaces, where participants can curate a subset of a dataset that achieves two contradictory goals: reducing size while maintaining or improving object detection model performance. The competition offers prizes for first, second, and third places, as well as integrations with FiftyOne and Hugging Face's transformers library, which streamlines computer vision workflows and enables the application of transformer models to datasets. Additionally, FiftyOne integrates with the Hugging Face Hub, simplifying dataset sharing and access to popular computer vision datasets.