May 2024 Summaries

FiftyOne 0.24 has been released, introducing significant updates that enhance its representational power and extensibility. The update includes support for full 3D scenes with meshes (GLTF, OBJ, PLY, STL, FBX), point clouds, and arbitrary 3D geometries composed of boxes, spheres, cylinders, and planes. Additionally, the release introduces Custom Workspaces, allowing users to create custom panel layouts in the FiftyOne App and persist them to their dataset for later use. These features aim to simplify and automate how users explore, visualize, and curate visual data, enabling higher quality datasets, faster iteration, and more flexible integration into workflows.

Diffusion models have revolutionized generative AI in recent years, with applications ranging from image synthesis to weather prediction. One of the most significant advancements in this field is Shap-E, a model that can generate 3D objects based on text or images. The journey towards creating Shap-E involved numerous developments and improvements in diffusion models, such as Classifier Guided Diffusion, Classifier-Free Guided Diffusion, and Latent Diffusion. These advancements allowed for the creation of more complex and diverse samples, ultimately leading to the development of 3D generation with Shap-E. The potential applications of diffusion models are vast, and their continued evolution promises even greater breakthroughs in various fields.

Voxel51 has secured $30 million in Series B funding led by Bessemer Venture Partners to make visual AI a reality. The company plans to use the funds to scale its team, support open-source community, build a research team, and accelerate product roadmap for new data modalities and deeper integrations with the AI stack. Hugging Face is investing $10 million in free shared GPUs through ZeroGPU program to help developers, academics, and startups create new AI technologies. California's Senate passed SB 1047, a bill focused on ensuring the development of powerful AI is safe and responsible. The bill includes strict rules for large AI models, whistleblower protection, fair pricing, CalCompute public cloud system, and open-source support. Lumina-T2X is an open-source model that transforms text into various modalities, resolutions, and durations using flow-based large diffusion transformers (Flag-DiT). It can generate images, videos, 3D objects, and audio from text descriptions.

Bessemer Venture Partners, a well-established Silicon Valley venture firm known for backing successful startups such as Twilio, Pinterest, and Shopify, has led Voxel51's $30 million Series B funding round. The investment is significant due to Bessemer's long-term perspective in the markets where they invest, including AI & ML. Lindsey Li, a partner at Bessemer Venture Partners, shared insights on the firm's focus and approach, emphasizing their commitment to partnering with audacious founders as early as possible in their key investment areas. The partnership between Voxel51 and Bessemer is expected to accelerate the adoption of visual AI technology and contribute to the expansion of Voxel51's platform.

Voxel51 has secured $30 million in Series B funding led by Bessemer Venture Partners, with participation from new investor Tru Arrow Partners and existing investors Drive Capital, Top Harvest Capital, Shasta Ventures, and ID Ventures. The company is focused on making visual AI a reality, as image and video data make up the majority of all data traffic. Voxel51's open-source platform FiftyOne has been downloaded millions of times and used by thousands of teams to manage, collaborate, and refine their data and models. The new funding will enable the company to scale its team, support its open source community, build a research team, and accelerate product development for new data modalities and deeper integrations with other AI tools.

Voxel51, a leader in visual AI, has secured $30 million in Series B funding led by Bessemer Venture Partners, with participation from Tru Arrow Partners and existing investors, to expand its market operations and accelerate its product roadmap. Founded by experts in machine learning and computer vision, Voxel51 aims to address the high failure rate of AI projects, which is often due to the complexities of utilizing visual data for AI applications. The company provides tools such as the open-source FiftyOne and enterprise FiftyOne Teams, which are designed to streamline workflows for AI engineers by simplifying data exploration, visualization, curation, and testing, ultimately improving productivity and model accuracy. The funding will support Voxel51's mission to enable the development of more accurate and robust visual AI applications, with current users including companies like LG Electronics and RIOS Intelligent Machines. Recent innovations include integrations with NVIDIA Omniverse and native vector search capabilities, along with the introduction of VoxelGPT, a natural-language interface that enhances user interaction with visual data.

Voxel51's bi-weekly digest highlights various advancements in the field of artificial intelligence (AI), machine learning, and computer vision. OpenAI's Sora was commissioned to create a music video for The Hardest Part by Washed Out, marking the first AI-generated music video. A mysterious new AI chatbot named "gpt2-chatbot" appeared on LMSYS Chatbot Arena, sparking intense speculation among AI experts. MIT's Jonathan Ragan-Kelley is developing specialized programming languages for efficient visual AI applications like graphics and image processing. StoryDiffusion can generate consistent, long-range image sequences and videos using consistent self-attention to maintain character styles and attires across multiple frames. Simone Scardapane's "Alice's Adventures in a Differentiable Wonderland: A Primer on Designing Neural Networks" delves into the core concepts and components of neural networks, offering a balanced blend of theory and practical application. Voxel51 co-founders Brian Moore and Prof. Jason Corso share their journey from professor and student to co-founders in a special episode featuring them on the How I Met My Co-founder podcast. MM-LLMs: Recent Advances in MultiModal Large Language Models survey paper covers the general model architecture of MM-LLMs, consisting of five components: the Modality Encoder, Input Projector, LLM Backbone, Output Projector, and Modality Generator. The open source FiftyOne computer vision toolkit has crossed 2 Million downloads.

Secury360, a commercial and home security company, has successfully automated its data pipeline using FiftyOne Teams, an open-source dataset management solution. The automation process involves seamless integration with Azure cloud storage, CVAT, and vector databases, allowing the ML team to efficiently filter, query, visualize datasets, and evaluate model performance. This automation not only saves valuable engineering time but also enhances model performance by tracking false positives and enabling continuous improvement. Secury360's security solution, which includes an edge device equipped with a detection model and a cloud AI for detailed analysis, benefits from the seamless integration of FiftyOne Teams with Azure cloud storage, ensuring efficient data management and accessibility.

The May '24 AI, Machine Learning, and Data Science Meetup featured presentations on Anomalib, anomaly detection with FiftyOne, hazy oracles in human+AI collaboration, learning robot perception and control using vision with action, and more. Attendees had the opportunity to vote for their favorite charity, with AI4ALL receiving a $200 donation from the event. The next Meetup is scheduled for May 30th, featuring five speakers discussing topics such as multi-model visual question answering, fine-tuning Llama2 for autonomous agents, and enhancing the adoption of open source libraries.

The tutorial offers a detailed guide on performing anomaly detection in visual data using FiftyOne and Anomalib from the OpenVINO toolkit, focusing on the MVTec AD anomaly dataset. It discusses the setup process, including the installation of necessary tools and dependencies in a Python virtual environment. The tutorial demonstrates training an anomaly detection model using two algorithms, PaDiM and PatchCore, for segmenting anomalies in images by predicting defective regions and creating masks. The evaluation process involves assessing model performance through metrics like precision, recall, and F1 score, and visually comparing models using generated masks and heatmaps. The guide also emphasizes the potential of using multiple models to enhance anomaly detection and suggests exploring additional algorithms, backbones, hyperparameters, and data augmentation techniques to improve performance. It highlights participating in the Visual Anomaly Detection 2.0 Challenge, which leverages the MVTec AD dataset and Anomalib, as an opportunity for practical application and innovation in the field.

The article presents a detailed tutorial on using Anomalib and FiftyOne to perform visual anomaly detection on datasets, particularly focusing on machine learning applications in critical fields such as fraud detection, network security, and medical diagnosis. It explains the process of setting up the Anomalib tool, utilizing the MVTec AD anomaly dataset, and training machine learning models for segmentation tasks aimed at identifying defects in images. The guide emphasizes the importance of high-dimensional visual data and outlines steps to evaluate and compare the performance of different anomaly detection models, specifically using the PaDiM and PatchCore algorithms. Furthermore, the tutorial highlights the potential of enhancing model performance through various techniques including algorithm selection, hyperparameter tuning, and data augmentation, encouraging users to explore custom solutions for scalable visual anomaly detection applications.

The May '24 AI, Machine Learning, and Data Science Meetup recently took place, featuring discussions on Reinforcement Learning from Human Feedback (RLHF), Supervised Fine-Tuning (SFT), legal search applications using the Milvus Project and DSPy, and ensuring safe and reliable deployment of large language models. The event also highlighted a charity donation to AI4ALL, an organization that provides education and mentorship in artificial intelligence for historically excluded talent. Upcoming Meetups will cover topics such as human+AI collaboration, real-world applications of anomaly detection, robot perception and control using vision with action, and more. The Computer Vision Meetups are sponsored by Voxel51, the company behind the open source FiftyOne computer vision toolset.

FiftyOne Teams 1.6 introduces enhanced security controls for AI teams, emphasizing the importance of incorporating security from the beginning of an AI project's lifecycle. The platform is designed to manage unstructured data and AI model development for global enterprises. Key features include dataset collaboration, roles and permissions, cloud storage integration, dataset version control, workflow automation, and enterprise support. In this latest release, FiftyOne Teams 1.6 offers fully air-gapped deployments, user groups, and pluggable authentication with automatic group membership management.