June 2023 Summaries

ControlNet has emerged as a successful project in machine learning (ML) in 2023. It provides an easy and interpretable way to influence the outputs of diffusion models through input maps, such as Canny edge maps, segmentation masks, pose keypoints, and scribbles. One of its key features is accessibility; ControlNet models are only 1.45GB in size, making them trainable at home on a single GPU in just 600 GPU hours. Despite the success of ControlNet 1.0, it suffered from bugs that were resolved in version 1.1. The major change was improving data quality, emphasizing the importance of high-quality data for state-of-the-art performance. This blog post provides a step-by-step guide on how to clean and curate high-quality data for training ControlNet models from scratch.

The text provides advice for making the most of CVPR, a prestigious computer vision conference. It suggests four strategies: being selective in choosing which papers to focus on; being observant of macro trends at the conference; being friendly and engaging with others; and being active by recounting experiences after the event. The author shares personal experiences from their first conference and emphasizes the importance of networking and building relationships within the community.

The Voxel51 team has made datasets from CVPR 2023 publications available on their platform, FiftyOne Teams. Out of 2,359 accepted papers at the conference, 59 were associated with new datasets. These datasets can be browsed through the Voxel51 booth and at cvpr.fiftyone.ai. The team manually searched for these datasets by looking into the abstracts of each paper to determine if they involved creating or using a new dataset. They then added 9 compatible datasets to their platform, which are now available for exploration.

Voxel51 has released two new products: FiftyOne 0.21 and VoxelGPT. The former is an open-source machine learning toolset that helps data science teams improve computer vision models, while the latter is a plugin that converts natural language queries into Python code for filtering, sorting, and analyzing images and videos in datasets without coding. In a live webinar, Voxel51 Co-Founder and CTO Brian Moore demonstrated these new capabilities, including VoxelGPT's chat-based interface for querying datasets, the Plugin framework enhancements in FiftyOne 0.21, and how to build custom workflows using operators.

This week's FiftyOne tips and tricks blog covers various topics such as understanding the compute_hardness function, custom annotation color schemes in the FiftyOne App, using Dynamic Groups with a video dataset, resolving MongoDB backend port conflicts, and support for multiple users in the FiftyOne App. The community is encouraged to join the FiftyOne Slack channel and explore its various features.

The article discusses the evolution of handling unstructured data in AI systems over the past two decades. It highlights the challenges faced by researchers due to the increasing size of datasets, which makes it difficult to manually analyze and understand patterns within them. The author emphasizes the importance of a closed-loop approach in building highly performant AI systems. FiftyOne is an open-source toolset designed to help AI scientists and engineers close the loop while working with unstructured data. It enables users to semantically search and explore datasets, visually analyze their distribution, compare multiple model runs, and more. The author compares FiftyOne to a debugger for the AI stack, emphasizing its significance in the field of computer science. Recently, try.fiftyone.ai was launched, allowing users to try out FiftyOne directly in their browser without any local installation. This platform also offers a glimpse into FiftyOne Teams, an Enterprise version of FiftyOne that supports collaboration and security features such as single sign-on, role-based access control, sharing of datasets and models, and more. The author concludes by encouraging readers to explore try.fiftyone.ai and experience the benefits of using FiftyOne in their AI workflows.

The June 8th Computer Vision Meetup featured presentations on Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translation, Re-Annotating MS COCO, and Redefining State-of-the-Art with YOLOv5 and YOLOv8. The event raised $200 for Global Empowerment Mission (GEM) through a voting system. Attendees had the opportunity to ask questions during Q&A sessions following each presentation. The next Computer Vision Meetup is scheduled for July 13th, focusing on vector search applications in computer vision.

CVPR is an upcoming event where computer vision researchers and engineers from around the world will gather in Vancouver, Canada. Voxel51, a company specializing in computer vision tools, will be participating and encourages attendees to visit their booth (#1618). They have several exciting offerings at the event, including: 1. The launch of VoxelGPT, an AI assistant for computer vision that enables users to search images or videos without writing code. 2. A demonstration of FiftyOne, an open-source computer vision toolset designed to improve dataset quality and gain insights about models. 3. Opportunities to meet with the Voxel51 team and other computer vision enthusiasts for discussions on various topics related to the field. 4. Giveaways of FiftyOne swag for those who engage with their open-source community in various ways, such as following them on social media or joining their mailing list. 5. A secret surprise that attendees should not miss at the Voxel51 booth (#1618).

Voxel51 has introduced VoxelGPT, an AI-powered extension of the FiftyOne computer vision toolkit, designed to simplify and accelerate the process of building and deploying high-quality computer vision models. VoxelGPT integrates the capabilities of GPT-3.5 with FiftyOne's query language, enabling users to generate Python code for data analysis through natural language queries without requiring coding skills. This innovation aims to streamline the development of AI systems by allowing users to efficiently navigate large datasets and identify insights, ultimately facilitating faster progression from concept to production. VoxelGPT is available as a free, open-source tool under the Apache 2.0 license and can be used instantly in a browser or installed as a plugin in the FiftyOne App, providing flexibility and ease of use for developers and organizations working with data-centric AI solutions.

VoxelGPT is an open-source application that combines large language models with FiftyOne's computer vision query language, allowing users to semantically slice their datasets and build better machine learning models without writing code. It translates English queries into Python code for dataset filtering and can answer specific FiftyOne usage questions and general computer vision inquiries. VoxelGPT is currently limited to certain types of queries but offers a chat-like interface for interacting with computer vision datasets.

This week's FiftyOne tips and tricks blog covers various topics such as making fields from duplicates_view available in exported annotations, deleting label fields, loading datasets on a headless server, efficiently modifying and deleting fields in views, and accessing samples from Amazon S3. The blog also provides solutions to these issues and directs readers to the FiftyOne Docs for more information. Additionally, it encourages users to join the FiftyOne community and highlights its growing popularity among engineers and data scientists.

Voxel51 and the FiftyOne community have released general availability of FiftyOne 0.21, an open source machine learning toolset that helps data science teams improve computer vision models by curating high quality datasets, evaluating models, finding mistakes, visualizing embeddings, and getting to production faster. The release includes new features such as Operators, dynamic groups, custom color schemes, custom field visibility, multiple point cloud overlays, and more.

The FiftyOne blog series highlights recent developments in the FiftyOne community and celebrates significant milestones. In this edition, METU (Middle East Technical University) shares how they use FiftyOne for medical research on inflammatory bowel disease. They utilize various features of FiftyOne, including CVAT integration, comparison of doctors' annotations with model predictions, and visualization tools like t-SNE. Additionally, the blog announces the release of FiftyOne 0.21 and Teams 1.3, which include upgrades to the plugin framework and new collaboration-minded features for commercial users. The community is encouraged to share their success stories using FiftyOne and contribute to its open-source project on GitHub.