April 2023 Summaries

The text discusses the creation of movement based on written descriptions, which has various applications in industries such as gaming, filmmaking, robotics animation, and the future of the metaverse. It presents a novel framework called T2M-GPT that utilizes a Vector Quantised Variational AutoEncoder (VQ-VAE) and a Generative Pretrained Transformer (GPT) to generate human motion capture from textual descriptions. The T2M-GPT framework achieves superior performance compared to other state-of-the-art approaches, including recent diffusion-based methods. Human motion synthesis is also discussed, which involves developing algorithms and models that can simulate human motion using motion capture data, physical laws, and artificial intelligence techniques.

The April 27, 2023 Computer Vision Meetup featured presentations on leveraging attention for improved accuracy and robustness in neural networks, as well as breaking the bottleneck of AI deployment at the edge with OpenVINO. Attendees had the opportunity to vote for their favorite charity, with Wildlife AI receiving the highest number of votes. The next Computer Vision Meetup will take place on May 11 and will feature talks on the role of symmetry in human and computer vision, as well as machine learning for fast, motion-robust MRI.

Jacob Marks, PhD and Machine Learning Engineer at Voxel51, presented the Getting Started with FiftyOne Workshop, an educational event covering the open source computer vision toolset. The workshop included a lecture on FiftyOne basics, useful workflows, and how it represents unstructured data. It also featured a hands-on lab where participants learned to install FiftyOne, load datasets and models, evaluate model predictions, and more. Q&A covered topics such as segmentation support, 3D point cloud datasets, integrations, model evaluations, backend databases, and dataset transformations. Upcoming events include Computer Vision Meetups in May and June '23, and additional Getting Started with FiftyOne Workshops on May 31 and June 28.

Voxel51 recently released FiftyOne 0.20, an update packed with new features for organizing, visualizing, searching, and exploring computer vision datasets. The release includes natural language search capabilities, Qdrant and Pinecone integrations, enhanced similarity API, and upgrades to point cloud support. These updates aim to streamline data annotation workflows, identify and correct labeling mistakes, analyze model performance, and enable various other computer vision workflows.

This week's FiftyOne tips and tricks blog covers three interesting questions from the community. Firstly, it explains how to perform basic operations on datasets using FiftyOne, such as removing duplicates and showing distributions. Secondly, it demonstrates how to concatenate generated patch views in FiftyOne. Lastly, it discusses exporting FiftyOne datasets with custom paths for filenames. The blog also encourages readers to join the growing FiftyOne community on Slack, GitHub, Meetup, and other platforms.

In this blog post, we explore Google Research's Image Matching Challenge 2023 – Reconstruct 3D Scenes from 2D Images competition using the open-source FiftyOne computer vision toolset. The challenge involves reconstructing 3D scenes from multiple views and discovering the best methods for building accurate 3D models, which can be applied to photography, cultural heritage preservation, and more. We discuss how to download the dataset, install FiftyOne, import the dataset into the App, visualize it, and use embeddings to reveal patterns and clusters in the data. The post also covers tips and tricks for working with the dataset and mentions the sponsors and individuals responsible for putting together the competition.

Voxel51 hosted the April 13, 2023 Computer Vision Meetup featuring presentations on various topics such as using computer vision to understand biological vision, emergence of maps in the memories of blind navigation agents, and generating diverse and natural 3D human motions from texts. The event also included Q&A sessions with each speaker. Voxel51 has expanded its Computer Vision Meetup network to include 13 locations worldwide, aiming to bring together communities of data scientists, machine learning engineers, and open source enthusiasts who want to share and expand their knowledge of computer vision and complementary technologies. Upcoming events will feature speakers discussing topics like leveraging attention for improved accuracy and robustness in AI deployment at the edge with OpenVINO.

Researchers have developed a new method called GLIGEN (Grounded Language to Image Generation) that allows users more control over AI-generated imagery. Unlike existing text-to-image diffusion models, GLIGEN can be conditioned on input from other modalities such as bounding boxes and keypoints. The approach involves adding new learnable parameters to adapt and modify intermediate features in existing models without changing their weights. This enables users to generate images with specific conditions, such as controlling the position of objects or the pose of generated subjects. GLIGEN has been trained on a combination of grounding data, detection data, and caption data, making it capable of handling various tasks like object detection and causal inference.

This week's FiftyOne tips and tricks blog covers various topics such as customizing the visualization of object embeddings, using FiftyOne to copy predictions over to ground truth for CVAT labels, adding detections to a video dataset, splitting data into train, validation, and test sets in FiftyOne, and keeping a persistent view of dataset for multiple sessions. The blog also encourages readers to join the FiftyOne community on Slack, GitHub, Stack Overflow, and Reddit.

The FiftyOne community has been thriving with various computer vision projects and workflows being implemented using the platform. Companies like LiveReach, Protex AI, and G42 have successfully integrated FiftyOne into their systems to improve video intelligence, monitor worker safety, and streamline data processing. Community members are encouraged to share their success stories and claim rewards for their contributions. The latest product release, FiftyOne 0.20, introduces new features such as natural language queries in the app and integrations with Qdrant and Pinecone for native text and image searches on datasets. Additionally, the community has reached over a million downloads, and various events like Computer Vision Meetups and workshops are being organized to help users get started with FiftyOne.

Voxel51's "Getting Started with FiftyOne" workshop, led by Jacob Marks, PhD and Machine Learning Engineer at Voxel51, introduced the open source machine learning toolkit, which helps data engineers and scientists visualize datasets and interpret models faster and more effectively. The workshop covered topics such as curating, understanding, and evaluating data using FiftyOne's flexibility and customizability features. Participants learned how to install FiftyOne, work with the Python SDK and App, import datasets, create views, and draw insights from their data and models. Upcoming workshops were announced for April 26, May 31, and June 28.