August 2023 Summaries

The healthcare industry is leveraging computer vision, machine learning, and artificial intelligence to drive innovation. With the global healthcare market projected to reach $11 trillion by 2025, these technologies are helping address key challenges such as rising costs, shortage of physicians, and time-intensive electronic health records (EHR). Computer vision applications in healthcare include computer-aided detection and diagnosis, monitoring disease progression, preoperative surgical planning, intraoperative surgical guidance, and assisting people with vision loss. Companies at the forefront of these advancements include .lumen, Future Fertility, Hologic, Iterative Health, Pixee Medical, and SafelyYou.

This week's FiftyOne Plugin is the Visual Question Answering (VQA) Plugin. It allows users to ask open-ended questions about their images within the FiftyOne App, without writing any code. The plugin supports two models: a Vision-Language Transformer and BLIP2 from Salesforce. Users can install the plugin using pip and download it from GitHub. The VQA Plugin is a Python Plugin consisting of four files and doesn't write anything onto the samples themselves, but returns the model's answer as output.

The healthcare industry is leveraging computer vision and machine learning to drive innovation in various areas. Key challenges include rising healthcare costs, a shortage of physicians, and time-intensive electronic health records (EHR) systems. Despite these challenges, computer vision is being used to enhance detection and diagnosis capabilities in medical imaging, monitor disease progression, and improve preoperative surgical planning. Additionally, AI-powered glasses are being developed to assist people with vision loss, while companies like Hologic, Iterative Health, Pixee Medical, SafelyYou, and others are working on various projects using computer vision and machine learning to transform healthcare. The National Institutes of Health is creating and maintaining publicly available health-related datasets, including machine learning and computer vision datasets such as DeepLesion, OASIS, and ChestX-ray8.

The August 24, 2023 Computer Vision Meetup featured presentations on automatic background removal from images using user language input and optimizing deep learning models for edge devices. Jizhizi Li discussed her research on image matting and shared insights into the differences between semantic and matting branches, noise suppression filters, encoder architecture optimization, and more. Raz Petel of SightX explored techniques for deploying AI on edge devices, focusing on hardware-aware optimization of deep learning models. Dan Gural from Voxel51 covered drone applications in computer vision, challenges faced with drone data, and emerging trends like Generative AI. The next Computer Vision Meetup is scheduled for September 7, featuring speakers discussing monitoring large language models (LLMs) in production, neural radiance fields for free-viewpoint videos, and a dataset for long-form video understanding.

This week's FiftyOne tips and tricks blog explores the Command Line Interface (CLI) of the open source machine learning toolset, which can streamline workflows for data science teams. The CLI offers commands to list available datasets, delete old ones, view dataset information, export samples with drawn-on labels, and manage plugins. By leveraging these powerful options, users can improve their computer vision projects' efficiency and performance.

The AI Art Gallery plugin allows users to generate images from text prompts using DALL-E2 or Stable Diffusion models within the FiftyOne App, and add them directly to their dataset. The plugin supports three models out of the box, with options for configuring image size and inference steps. Users can filter their "art gallery" by text prompt, model name, date and time generated, or any other attributes used in the generation process. The plugin also allows users to add notes to specific pieces of artwork and delete unwanted images. To use the plugin, users need an account with at least one of the supported services (OpenAI or Replicate) and have their API info in environment variables.

The OpenCV AI Competition is an annual event that challenges machine learning engineers worldwide to create innovative solutions using the OpenCV Library. Participants are encouraged to explore any task or prompt, with submissions ranging from robotics and agriculture to education and health. The competition fosters growth in the field of computer vision and encourages participants to push their limits. Winners have used their winning projects to start new businesses or develop innovative solutions for real-world problems. Open source FiftyOne can help streamline OpenCV projects by providing powerful data visualization tools, enabling users to extract valuable insights from their datasets.

FiftyOne is an open source toolkit designed to streamline the process of building high-quality datasets and computer vision models. It offers numerous benefits, including efficient data curation, simplified model evaluation, and easy identification of mistakes in your data. With its powerful data ingestors, you can import datasets in over 28 different formats, allowing for quick conversions from one format to another. Additionally, FiftyOne's built-in Brain technology enables the inspection of entire datasets for troublesome samples, making it easier to identify and correct issues such as duplicates or detection mistakes. By using FiftyOne, novice ML engineers can save hours, if not days, within their computer vision workflow.

FiftyOne is an open source machine learning toolset that helps data science teams improve the performance of their computer vision models. It enables users to curate high quality datasets, evaluate models, find mistakes, visualize embeddings, and get to production faster. This week's tips and tricks cover finding and removing duplicate images, classification mistakes, and detection mistakes in datasets using FiftyOne. The toolset helps users identify and correct these issues, improving the overall performance of their computer vision models.

Last week, Voxel51 celebrated three milestones for their open source project FiftyOne. They released FiftyOne 0.21.5 and 0.21.6 with new features such as Segment Anything and DINOv2 models in the Model Zoo, support for controlling field visibility, filtering by label tags, and more. Additionally, they crossed 4000 stars on GitHub and celebrated their third anniversary since launching open source FiftyOne. The project has grown significantly over the past three years, with thousands of users and contributors.

The August 10, 2023 Computer Vision Meetup featured presentations on Neural Congealing, AI-enabled computer vision for personalized medicine and radiotherapy, and a practical approach to deep learning for computer vision with Tensorflow 2. Attendees also had the opportunity to vote for their favorite charity, with Coalition for Rainforest Nations receiving the highest number of votes. The next Meetup is scheduled for August 24th, featuring speakers on background removal, self-supervised representative learning for action recognition in videos, and AI at the edge: optimizing deep learning models for real-world applications.

This week's FiftyOne tips and tricks blog focuses on managing fields in datasets. Fields can include metadata, scalar fields, labels, or tags within a single sample. Users can add predictions to their samples, allowing for visualization of predictions next to ground truths and evaluation of data scores such as accuracy or mAP (mean average precision). Additionally, strings or scalars can be added to samples to keep track of data origins, contributors, or purposes. Another useful feature is mapping original labels to super categories by cloning the sample field and remapping category names to supercategories in the new field. For more information on fields, samples, and other FiftyOne features, refer to the User Guide.

Researchers from MIT, Weizmann Institute, and Adobe have developed a new perceptual similarity metric called DreamSim that bridges the gap between human and machine perceptual similarity. The team created a novel benchmark dataset called NIGHTS (Novel Image Generations with Human-Tested Similarity) to train DreamSim. Unlike previous metrics, DreamSim captures both low-level features such as texture and color, as well as high-level semantic information. This allows for more accurate representation of human perceptual similarity judgments. The development of DreamSim is a significant step forward in the field of computer vision and artificial intelligence, enabling machines to better understand and interpret visual content in ways that align with human perception.

This week's FiftyOne tips and tricks blog covers various questions and answers from the community. Topics include adding new label fields and mapping them to super categories, choosing a dataset type for instance segmentation, reducing the number of images returned, adding keypoint skeletons based on sample attributes, merging datasets and renaming labels, and more. The FiftyOne community is open to all, encouraging members to ask questions and provide answers.

The FiftyOne community has been thriving with new use cases and integrations being developed. Fast Code AI uses FiftyOne for managing their data lifecycle, while two new vector search integrations have been introduced - Milvus and LanceDB. Additionally, the GitHub repository for FiftyOne continues to grow in popularity, and the community Slack channel has welcomed numerous new members. The Computer Vision Meetups sponsored by Voxel51 are also gaining traction, with interesting topics being discussed each month.