March 2023 Summaries

This week's FiftyOne tips and tricks blog covers embeddings in computer vision. Embeddings are numerical vectors that represent visual features of images or videos, capturing underlying characteristics for tasks such as image classification, object detection, and image retrieval. In FiftyOne, the Brain library helps manage and utilize embeddings to analyze and visualize datasets, accelerating machine learning workflows. Tips include computing sample properties like uniqueness and similarity from embeddings, visualizing embeddings using dimensionality reduction techniques, using custom embedding models, computing embeddings for object patches, and performing zero-shot classification with CLIP.

This week's FiftyOne tips and tricks blog covers various topics such as pre-populating panels in the FiftyOne App, accessing a FiftyOne session from a new Python process, filtering samples by patch embeddings, regex on strings in FiftyOne, and updating file extensions in FiftyOne. The blog also encourages readers to join the FiftyOne community and provides information about its growing popularity among engineers and data scientists.

FiftyOne Teams 1.2 offers a GSuite-like experience for managing computer vision datasets and workflows, with added support for secure, real-time multi-user collaboration. It combines the features of open source FiftyOne with additional capabilities such as SSO, role-based access control, and world-class customer support. The latest release introduces enhancements to enterprise security controls, new dataset page capabilities, and a large-scale deployment option.

FiftyOne .20 release introduces native vector search capabilities for computer vision data. Users can now search through their datasets with images or text, and these new querying functionalities seamlessly integrate with existing semantic slicing operations in FiftyOne. The update includes native integration with Pinecone vector search database and Qdrant vector search engine, SDK support for querying by sample ID, query vector, or text prompt, and UI support for querying with images extended to use vector search engines.

The latest release of FiftyOne computer vision toolset (version 0.20) introduces enhanced point cloud support, allowing users to visualize and manipulate their 3D data more effectively. Key features include orthographic projections, support for point cloud-only datasets in the FiftyOne App, multiple point cloud slices in grouped datasets, enhanced rendering and customization, and evaluation of 3D object detection predictions. These additions expand upon FiftyOne's existing 3D capabilities for working with and visualizing point clouds.

Voxel51 and the FiftyOne community have released version 0.20 of their open source machine learning toolset, which includes new features for indexing and searching datasets. The release introduces natural language search capabilities, significant upgrades to the similarity API, Qdrant and Pinecone integrations, and support for point cloud-only datasets. These enhancements enable users to build workflows that co-develop their datasets and machine learning models more effectively.

Voxel51 is hosting a series of educational workshops on using FiftyOne, an open source toolkit for improving computer vision workflows. The first workshop, "Getting Started with FiftyOne," will take place virtually on March 29 at 10 AM Pacific. Led by Jacob Marks, the workshop aims to help participants gain insights into their computer vision datasets and models. The 90-minute session is divided into a lecture portion covering FiftyOne basics and workflows, followed by hands-on lab exercises. Attendees will learn how to install FiftyOne, use the Python SDK and App, and perform basic tasks like importing datasets and drawing insights from data and models. Prior knowledge of Python and computer vision concepts is recommended for maximum benefit.

This week's FiftyOne tips and tricks blog covers various ways to accelerate computer vision workflows using the open source machine learning toolset, FiftyOne. Tips include loading data into FiftyOne faster by leveraging PyTorch DataLoader class, perusing through samples faster with thumbnail images, filtering samples faster by setting sidebar mode to "fast", starting with most unique samples for visual inspection, and pre-annotating with embeddings. These tips aim to help users improve the performance of their computer vision models and get to production faster.

The Cityscapes dataset is a large-scale collection of street scene sequences from 50 cities, containing high quality pixel-level annotations for 5,000 frames and additional weakly annotated frames. It is designed to support research in semantic urban scene understanding tasks such as pixel-level, instance-level, and panoptic semantic labeling. The dataset can be downloaded and loaded into the FiftyOne toolset, which helps data science teams improve computer vision models by curating high quality datasets, evaluating models, visualizing embeddings, and getting to production faster.

Voxel51 hosted the March 2023 Computer Vision Meetup featuring presentations on recycling max pooling modules for 3D point cloud analysis, low light image enhancement in the deep learning era, and taking computer vision models to production. The event also included a Q&A session and information about upcoming meetups and events. Attendees were given the opportunity to vote for their favorite charity, with The Foundation Fighting Blindness receiving the highest number of votes this month.

This week's FiftyOne tips and tricks blog covers adding metadata to a dataset, changing tags when loading CVAT annotations into FiftyOne, previewing video frames in FiftyOne, frame-level aggregations, deleting duplicate samples, and more. The blog also highlights the growing community of engineers and data scientists using FiftyOne for computer vision tasks.

The manufacturing industry is undergoing a massive transformation driven by computer vision, artificial intelligence, robotics, and the industrial internet of things (IIoT). Computer vision plays a central role in this transformation, with companies using machine vision techniques to automate strenuous tasks, identify issues in both product and machinery, and improve safety conditions for workers. Key applications include bin picking, palletizing and depalletizing, machine tending, defect detection, and predictive maintenance. Companies at the forefront of computer vision in manufacturing include Mech-Mind Robotics, Instrumental, Protex AI, Cognex, RIOS Intelligent Machines, PreML GmBH, Prophesee, Datalogic, Stemmer Imaging, and Pickit 3D.

Computer vision is increasingly being used in manufacturing to drive innovation and improve efficiency. The industry, which accounts for 16% of global GDP and 14% of employment, is undergoing a fourth industrial revolution that includes the adoption of computer vision, artificial intelligence, robotics, and the industrial internet of things (IIoT). Computer vision is already playing a central role in this transformation, with companies using machine vision techniques to automate strenuous tasks, identify issues in both product and machinery, and improve safety conditions for workers. Applications of computer vision in manufacturing include bin picking, palletizing and depalletizing, machine tending, defect detection, predictive maintenance, and various other tasks that require precise navigation and interaction with objects of different shapes, sizes, and materials. Companies such as Mech-Mind Robotics, Instrumental, Protex AI, Cognex, RIOS Intelligent Machines, PreML GmBH, Prophesee, Datalogic, Stemmer Imaging, Pickit 3D, and Matroid are at the cutting edge of computer vision in manufacturing, offering integrated hardware and software solutions that enable high accuracy and efficiency.

Google's Open Images dataset has been upgraded, adding 22.6 million point labels spanning 4171 classes. This upgrade introduces a new paradigm for semantic segmentation called "pointillism," which involves sparse annotations rather than densely labeling every pixel in an image. The addition of point labels reflects the ongoing shift in computer vision towards data-centric approaches. The latest version, Open Images V7, can be accessed using FiftyOne, an open source computer vision library recommended by Google's team.

This week's FiftyOne tips and tricks blog covers configuring FiftyOne and the FiftyOne App to tailor your experience working with computer vision data. The text discusses how to access, modify, and leverage configs for various purposes such as setting default ML backend, enabling plugins, and configuring app behavior on a per-dataset basis. It also highlights the flexibility and customization options available in FiftyOne, making it suitable for different computer vision workflows.

FiftyOne 0.19 has been released with new features that enhance visualization of computer vision datasets and improve machine learning model performance. These features include Spaces, which allows users to organize panels of information within the FiftyOne App; In-App Embeddings Visualization, enabling direct visualization of embeddings in the App through a panel made possible by Spaces; Saved views, allowing users to save views under a name and recall them later; On-disk segmentations, which lets users store semantic segmentation masks and heatmaps on disk; and new UI filtering options. These features aim to streamline annotation workflows, identify and correct labeling mistakes, and analyze model performance.