July 2024 Summaries

Researchers at the University of Hull have developed a technique to identify AI-generated fake images by examining eye reflections, which may provide a new weapon in the battle against deepfakes. Meanwhile, researchers are exploring alternatives to transformer architectures in AI, with test-time training models emerging as a promising contender. These models could potentially process vast amounts of data more efficiently than current transformer models. Additionally, researchers are investigating ways to enhance AI's spatial intelligence, which involves developing algorithms capable of realistically extrapolating images into three-dimensional reconstructions. This work aims to bridge the gap between current AI limitations and artificial general intelligence (AGI). Furthermore, a new framework for efficient portrait animation called LivePortrait has been developed, which can synthesize lifelike videos from a single source image using it as an appearance reference, while deriving motion from a driving video, audio, text, or generation. The researchers also explored the cognitive impact of extended AI interaction in their newsletter “Educating AI.” They discussed how immersive engagement with AI tools like ChatGPT and Claude can lead to subtle yet significant changes in our thinking patterns and self-perception. Moreover, an episode from the ML Street Talk podcast featured Murray Shanahan discussing his work on AI consciousness and the space of possible minds, exploring the intersection of artificial intelligence, consciousness, and philosophy. Finally, a paper titled "Data curation via joint example selection further accelerates multimodal learning" presents an approach that could revolutionize how we train large-scale multimodal models by intelligently selecting batches of data rather than individual examples.

This week’s newsletter highlights Voxel51’s latest blogs and key resources, including a blog on leveraging NVIDIA's Medical Foundation Models to segment CT scans automatically using FiftyOne!, and another on how leading automotive OEMs and Tier 1 suppliers are aligning data and models with effective workflows for successful visual AI applications. Additionally, the Hugging Face Hub now allows filtering datasets by library, making it easier to find the perfect dataset for a use case. A publicly available dataset from Wayve, called WayveScenes101, has also been added to FiftyOne, serving as a benchmark for novel view synthesis models for driving applications.

The Voxel51 bi-weekly digest covers recent developments in AI, machine learning, and computer vision. OMG-LLaVA combines robust pixel-level understanding with reasoning abilities in a single end-to-end trained model, achieving performance comparable to specialized methods on multiple benchmarks. Agility Robotics has deployed its Digit humanoid robots in logistics operations, while CARMEN is a small robot designed to help people with mild cognitive impairment learn skills to improve memory and executive functioning at home. The FiftyOne team released the Florence2 plugin for integrating the model into their open-source computer vision tool. Good reads include a series on building with large language models (LLMs) and advice from experts, such as Perplexity CEO Aravin Srinivas discussing his company's approach to indexing the web and creating an AI knowledge assistant. New research on tokenization is also highlighted, particularly in the context of multimodal learning, where it enables unifying diverse modalities into a common representation space. Upcoming events include various conferences and meetups for AI, machine learning, and computer vision professionals.

The automotive industry is under pressure to develop advanced AI capabilities for vehicles, but many projects struggle due to blindspots in their AI development process. These blindspots can lead to model failures, data issues, and slow progress. To overcome these challenges, it's essential to connect data and models in the AI process, making it possible for teams to refine their models and applications together with their data. A solution like FiftyOne Teams can help simplify and automate how AI builders explore, manage, visualize, and curate visual data and models, enabling teams to accelerate time to delivery for new capabilities and improvements. By bridging the gap between data and models, teams can identify and address blindspots, leading to faster success and more reliable AI systems.

The July '24 AI, Machine Learning and Computer Vision Meetup covered various topics such as performance optimization for multimodal LLMs, five handy ways to use embeddings, and responsible and unbiased GenAI for computer vision. Neha Sharma, Harpreet Sahota, and Daniel Gural were the speakers at this event. The next meetup is scheduled for August 8th, 2024, featuring talks on Generative AI for video, evaluating RAG models for LLMs, and why you should evaluate your end-to-end LLM applications with in-house data. Voxel51 sponsors these Meetups, which aim to bring together communities of data scientists, machine learning engineers, and open source enthusiasts who want to share and expand their knowledge of AI and complementary technologies.

The blog discusses how Nvidia's VISTA-3D, a medical foundation model, can be used to segment CT scans automatically. It provides a step-by-step tutorial on how to leverage this model for better analysis of medical imagery. The TotalSegmentator dataset is used as an example, which contains hundreds of CT scans annotated with 107 different body parts on each scan. The VISTA-3D model can handle various cases and make sharp and precise segmentations due to the 3D nature of a CT scan. This technology has the potential to revolutionize healthcare by improving medical imaging analysis and leading to better treatment options for patients.