June 2026 Summaries

A drone-based security system utilizes MediaMTX to process PTZ camera and drone feeds, sampling frames every two seconds for analysis by a Roboflow Inference Server running RF-DETR to detect people and cars, thereby automating the tracking of intruders in restricted zones without human intervention until an alert is triggered. The architecture involves streaming video through MediaMTX, sending frames to FastAPI workers, and using a combination of PolygonZone and ByteTrack to confirm breaches, which then dispatches a drone to the site. The design simplifies integration by using lightweight services and keeping computer vision concerns separate, enhancing efficiency through frame sampling and JPEG encoding at reduced resolutions. This approach leverages pre-trained models with the flexibility to switch to fine-tuned models for specific detection needs, such as aerial perspectives, by utilizing Roboflow's ecosystem for labeling and training datasets. The system demonstrates the potential for drones to cover expansive areas more efficiently than fixed cameras, offering a practical and adaptable solution for perimeter security.

Text prompt object detection, as explained in the tutorial, is a method that allows users to identify objects in images using plain-text descriptions, bypassing the need for a labeled dataset or model training. This approach, optimized through the SAM3 model in Roboflow Workflows, offers a rapid prototyping solution where users input class names to detect objects, receiving bounding boxes and segmentation masks in return. The technique is particularly advantageous for scenarios like plant disease detection, where symptoms can vary widely and building comprehensive datasets is time-consuming. By leveraging zero-shot object detection, SAM3 can recognize new concepts based on large image-text datasets, making it suitable for quick testing and dataset bootstrapping. However, while it offers significant flexibility and speed, the method may trade off some precision and requires careful prompt crafting to ensure accurate results. The tutorial details a workflow setup where SAM3 processes an image and returns annotated results, emphasizing its utility in rapid prototyping and dataset creation before transitioning to more consistently accurate, trained models like RF-DETR.

Human-in-the-loop computer vision is a method that allows operators to directly influence the development of computer vision models by providing feedback on the model's accuracy in real-time deployments. This approach is highlighted in a recent Roboflow webinar where Riaz Virani, an Enterprise Engineer, discusses how Roboflow's Vision Events platform incorporates Operator Feedback to improve model accuracy by capturing operator input on whether detections are correct or incorrect. This feedback mechanism ensures continuous improvement of models by incorporating real-world data and operator insights into future training runs. Roboflow's system enables seamless integration of operator judgments into the model development process, thereby enhancing the accuracy of the deployed models without relying on cumbersome side systems or manual data transfers. The platform offers features like a built-in Event Dashboard, smart filters, and natural-language queries through an MCP server, allowing for efficient and effective analysis of model performance across various locations and use cases.

Promptable object detection is an innovative approach that allows users to detect objects using text prompts rather than relying on pre-trained models with fixed classes. This method, demonstrated using Roboflow Workflows and the SAM 3 model, enables rapid prototyping and validation of ideas by providing immediate results without the need for collecting and annotating datasets. By simply typing a class name like "soda can," users can detect and count objects instantly, making it particularly useful for exploring long-tail or niche objects and validating computer vision applications before investing in dedicated model training. While promptable object detection excels at prototyping, it may not be the optimal choice for production deployments that require repeated detection of the same objects, where trained models like RF-DETR offer higher accuracy and efficiency. The guide also explores refining prompts for improved detection performance and transitioning to a trained model for scalable application deployment, illustrating the flexibility and potential of promptable object detection across various use cases.

The text outlines a method for automatically detecting injection molding defects using a Roboflow RF-DETR model, which is trained on images annotated with defect labels such as cracks, breaks, and surface contamination. Once trained, the model is deployed in a Roboflow Workflow, where it identifies defect regions and produces annotated images. A subsequent integration with Gemini 2.5 Pro allows for AI-powered inspection observations, resulting in concise inspection reports that are overlaid on the images. The workflow, originally designed for general plastic components, can be adapted for use with medical components like syringes and IV connectors by retraining the model with medical-specific data. This system aims to reduce material waste and production costs associated with defects in the injection molding process, which can amount to significant financial losses industry-wide. The text further explains that the workflow can be automated using Roboflow Agent, which constructs the pipeline based on a textual description of the desired process, thus facilitating its application in different manufacturing contexts, including medical device production.

An automated system for detecting foreign object debris (FOD) on airport runways is developed using Roboflow, which integrates the RF-DETR model for localizing debris and pavement holes, and Gemini 2.5 Pro for classifying objects and generating inspection summaries. This system addresses the high economic impact of FOD, estimated at $22.7 billion annually, by offering a continuous and efficient solution compared to traditional manual inspections. The process involves training the RF-DETR model on a public FOD dataset, achieving high accuracy with 99.0% mAP@50, and deploying the model within a Roboflow Workflow. This pipeline allows for the detection and annotation of debris on runways, providing maintenance personnel with precise locations of potential hazards. The workflow is adaptable to airport-specific datasets to account for unique conditions, and can be enhanced with surveillance technologies to improve detection and reporting, ultimately aiding in runway safety and maintenance efficiency.

The text discusses a two-stage computer vision pipeline for automating pharmaceutical quality control using Roboflow's RF-DETR and a vision language model (VLM) to detect and classify tablet defects, addressing common manufacturing issues such as capping, lamination, chipping, and cracking. The pipeline involves training an RF-DETR model to locate tablets, followed by the VLM classifying each tablet's defect type, which is then processed through a Custom Python Block to provide a pass or fail verdict accompanied by a structured report. This system allows for independent evolution of the detection and classification stages, as the detector remains constant while the VLM prompt can be updated to recognize new defect categories without retraining. The pipeline's adaptability is highlighted, as it can be applied to various products by changing the detection dataset and updating the VLM prompt, thus facilitating efficient inspection processes without frequent retraining cycles.

The text discusses the automation of transmission line inspections using Roboflow's RF-DETR model to detect hazards such as foreign objects and damaged cables. By integrating this model into a Roboflow Workflow, each inspection returns a PASS or FAIL verdict, a fault count, and a JSON report, which can be applied to drone footage or fixed camera feeds. The workflow, which uses a dataset from Roboflow Universe, involves several steps including object detection, class name remapping, visualization of bounding boxes and labels, and a final detection summary that informs maintenance teams about necessary actions. The approach offers a cost-effective and efficient alternative to traditional inspection methods, which are costly and prone to error. The system's deployment via Roboflow Inference allows it to run on various platforms, and the data collected over time can improve model performance and reveal hazard patterns. The flexibility of the workflow permits easy scaling and adaptation to new hazard types by updating the dataset and retraining the model.

Video analytics with vision AI is an advanced method that utilizes computer vision models to analyze video streams by detecting, tracking, and interpreting objects and activities without human supervision. This approach replaces traditional rule-based systems with trained models capable of answering complex questions, such as whether a package is damaged. The process involves three stages: object detection using an RF-DETR model to locate objects, multi-object tracking with ByteTrack to assign and maintain unique IDs for each object, and multimodal inspection via Gemini 2.5 Pro to assess the condition of each object. This tutorial demonstrates the application of these techniques in an automated conveyor box inspection system, providing real-time quality control in manufacturing environments. The described workflow can be adapted for various industries, including logistics, retail, and traffic management, enhancing efficiency and accuracy in monitoring and inspecting objects in video feeds.

RF-DETR Keypoint is an advanced real-time transformer model designed for keypoint detection, building upon the highly regarded RF-DETR architecture used in object detection and instance segmentation. This model predicts bounding boxes and keypoint coordinates within a single forward pass, without requiring non-maximum suppression (NMS) or heatmaps, and it includes confidence scores and uncertainty ellipses for each keypoint. It is versatile, allowing for fine-tuning on various keypoint layouts for different object classes beyond the default 17 keypoints trained on the COCO dataset. The tutorial details the process of fine-tuning the RF-DETR Keypoint model for custom applications, such as detecting 33 landmarks on a basketball court using the basketball-court-detection-2 dataset. The guide walks through initializing the model, training, and evaluating it on test images, while emphasizing the importance of maintaining consistency in keypoint annotation to accurately capture spatial structures. The tutorial also highlights the options of training with Roboflow's no-code platform or using an open-source Python package, providing insights into configuring training parameters and evaluating results through visualization tools.

The Vision AI Maturity Model is a strategic framework designed to transition vision AI from isolated pilot projects to scalable, production-ready capabilities within industrial enterprises. Many organizations face challenges as models that perform well in testing often falter in real-world factory conditions, leading to abandoned projects and missed opportunities for significant ROI. The model outlines five levels, each addressing specific barriers to scalability, from initial evaluation to achieving standardized, repeatable deployments across multiple sites. The framework emphasizes building organizational capability rather than focusing solely on model performance, promoting the reuse of solutions to reduce costs and reliance on external vendors. By Level 5, enterprises can efficiently deploy vision AI systems, integrating them into operations with minimal setup time and leveraging an internal team to manage and expand the system autonomously. This maturity model helps companies overcome common pitfalls, ensuring that vision AI becomes a sustainable, impactful part of their operations.

Roboflow and Standard Bots have announced a partnership aimed at enhancing the capabilities of industrial robots by integrating visual intelligence, allowing robots to see, understand, and act autonomously without requiring complex engineering interventions. This collaboration enables Standard Bots customers to deploy custom-trained vision models from Roboflow directly onto their robots, simplifying tasks like defect detection and part recognition. The integration promises to transform the economics of automation by reducing deployment time and making robots adaptable to new tasks with ease, thus increasing their value over time. Unveiled at Automate 2026, this partnership is set to make physical AI more accessible across various industries by providing a seamless experience in deploying visual intelligence on factory floors, ultimately allowing manufacturers to quickly adapt their robotic systems to evolving needs and challenges.

Automating steel surface defect inspection is achieved by training an RF-DETR Small model and integrating it into a Roboflow Workflow that categorizes strips into pass, review, or fail, moving beyond a simple pass-fail evaluation. Trained inspectors often miss 30 to 40 percent of defects, which can lead to quality issues downstream in processes like stamping or coating. By using a public dataset annotated for six defect classes, the system trains a model that achieves significant accuracy in defect detection. The Workflow involves steps to visualize bounding boxes, sort defects based on confidence levels, and log inspection events, ensuring a comprehensive quality check for each strip. The model's adaptability allows for continuous improvement as it learns from human-reviewed cases, thereby increasingly supporting inspectors rather than replacing them. This approach not only improves defect detection but also ensures a robust inspection system adaptable to evolving defect classes and conditions.

Mostafa Ibrahim's tutorial, published on June 22, 2026, introduces a method to automate planogram compliance using Roboflow Workflows, RF-DETR, and Gemini, addressing the costly issue of poor shelf execution in retail environments. Traditional manual inspections are inefficient for large-scale operations, where planograms can fall out of compliance by 10% weekly, significantly impacting sales. The guide provides a step-by-step process to build a planogram compliance workflow that detects empty shelves and brand mixing violations with AI models, bypassing the need for fixed planograms by auditing general shelf organization in real-time. The workflow involves creating a dataset, training an RF-DETR model, and configuring a series of blocks to detect empty zones, identify products, and generate compliance reports using structured JSON. The system is designed to run as a REST endpoint, enabling seamless integration with audit tools and real-time alerts, and can be deployed on edge devices for lower latency, ensuring that brands can maintain compliance across various store layouts and product categories efficiently.

RF-DETR Keypoint is a real-time, end-to-end pose detection model introduced by Roboflow that surpasses YOLO26-pose in both accuracy and speed while providing calibrated per-keypoint uncertainty from user data. It is designed to seamlessly integrate with Roboflow's platform, allowing users to label, train, and deploy skeleton-based models for various applications such as sports analytics and robot guidance. The model differs from other pose models by predicting a structured set of keypoints in a single forward pass and offering flexibility in defining arbitrary skeletons, not limited to the COCO benchmark. It is trained using weight-sharing neural architecture search, enabling a single set of weights to run across multiple resolutions without retraining. Released under the Apache 2.0 license, RF-DETR Keypoint allows for commercial use without copyleft obligations, addressing licensing barriers that may have previously hindered deployment.

An automated welding defect detection system was developed using Roboflow's RF-DETR model and Gemini 2.5 Pro to enhance weld quality inspection processes, crucial in manufacturing workflows reliant on robotic welding. Despite controlled welding parameters, defects can arise, necessitating early detection to prevent faulty components from advancing in production. The global welding market is experiencing significant growth, emphasizing the importance of automated inspection systems that utilize computer vision for evaluating weld quality consistently and with minimal human intervention. The RF-DETR model is trained to identify and classify welds as "Good Weld" or "Bad Weld" using a dataset with varied visual representations, enabling the model to generalize across different inspection scenarios. Once trained, the model's performance is evaluated through metrics like precision and recall, and then deployed in Roboflow Workflows alongside Gemini 2.5 Pro, which generates inspection summaries based on the model's predictions. This integration results in an efficient inspection process where weld regions are visually marked, and quality observations are summarized, facilitating easier review by operators and quality engineers. The tutorial underscores the potential of computer vision to automate quality inspection and improve early detection of welding issues, paving the way for more reliable manufacturing processes.

In the context of industrial computer vision, choosing between off-the-shelf and custom models is crucial for project success. Off-the-shelf models, like RF-DETR pretrained on Microsoft COCO, offer rapid prototyping capabilities and are suitable for tasks involving common objects. However, they may fall short in specialized industrial contexts, such as detecting helmets on construction sites, because they lack domain-specific training. Custom models, on the other hand, are fine-tuned with domain-specific datasets, enhancing reliability for specialized tasks. While off-the-shelf models provide a quick baseline and aid in prototyping, custom models, once trained with appropriate datasets, offer higher performance and adaptability for production environments. Fine-tuning RF-DETR on specific environments leads to successful detection of domain-specific objects, bridging the gap from demonstration to production without extensive augmentation or complex architectures. Roboflow facilitates this transition seamlessly by allowing users to prototype, fine-tune, and deploy models within the same platform.

Brian Guilfoyle's article, published on June 18, 2026, addresses the common issue of stale results when updating Roboflow Workflows deployed to the edge with a self-hosted inference server. This situation often arises due to caching mechanisms, where the server continues to serve an older workflow definition despite updates. The article explains how caching works in the context of three dependencies: the server image, the workflow definition, and model weights, each of which updates independently. The workflow definition is stored in a cache with a time-to-live (TTL) and may not immediately reflect changes made on the Roboflow platform. To address this, users can bypass the cache by setting `use_cache=False` or by restarting the server. The article also suggests using version tags in workflow outputs to easily identify the version being used. Guilfoyle emphasizes that seemingly stale predictions are usually a result of a local dependency being out of sync rather than a system failure, and can be resolved with patience or a deliberate refresh.

An electronic Quality Management System (eQMS) in manufacturing is software that digitizes and centralizes quality processes, such as document control, corrective and preventive actions, and nonconformance management, which traditionally relied on paper and spreadsheets. While eQMS is essential for compliance, visibility, and speed, it has limitations as it depends on human detection of defects. Vision AI addresses these gaps by inspecting all production in real time, providing objective data that automatically generates nonconformance records, and offering comprehensive defect trend data crucial for corrective and preventive actions. This integration enhances the quality system by connecting real-time production insights with digitized quality management, ensuring a more robust and proactive approach to quality assurance.

RF-DETR Keypoint is a novel real-time keypoint detection model that outperforms the YOLO26x-pose on both accuracy and speed on the COCO Keypoints dataset. This model, part of the RF-DETR family, is designed to be NAS-trained, allowing it to adapt to various input resolutions for optimal speed or accuracy without retraining. It introduces a probabilistic approach to keypoint detection, predicting a full distribution over each keypoint’s location, thus addressing hyperparameter issues that affect most keypoint models. Unlike other models, RF-DETR Keypoint learns keypoint bandwidths dynamically, eliminating the need for hand-tuned parameters and making it adaptable to custom datasets. The model leverages a full-covariance 2D Gaussian distribution for keypoint uncertainty, which remains usable downstream, enhancing applications such as tracking and camera calibration. This preview is released under the Apache 2.0 license, promoting open use without restrictive obligations, and aims to gather real-world feedback to refine the final model suite.

An automated system for detecting surface defects on machined metal medical parts is outlined, leveraging the RF-DETR Small model trained on a dataset from Roboflow Universe. The system aims to streamline quality control in the medical device industry, where precision and compliance are crucial, by identifying defects introduced during various manufacturing stages. This process involves training the model to detect defect regions, deploying it through Roboflow Workflows, and employing Gemini 2.5 Pro to generate inspection reports based on detected defects, without inventing new findings. The workflow, which combines object detection with vision-language models, outputs annotated images with defect detections and inspection summaries, serving as an inspection aid rather than a definitive quality assessment. The system is designed for integration into production lines, with potential adaptability to manufacturer-specific conditions, enhancing efficiency by allowing human inspectors to focus on flagged parts.

Physical AI is a transformative technology that enables systems to perceive, reason, and act in the real world through robots, sensors, and actuators, impacting fields such as robotics, autonomous vehicles, and smart manufacturing. Unlike purely software-based AI, physical AI requires an accurate understanding of the environment, as perception is the foundational step upon which reasoning and action depend. The effectiveness of these systems is heavily reliant on computer vision, which transforms raw sensor data into structured environmental insights necessary for decision-making. In robotics, for example, perception systems allow machines to navigate and manipulate objects in complex settings, while autonomous vehicles use sensor fusion to create a coherent understanding of their surroundings, enabling them to make split-second decisions. In smart manufacturing, vision models enhance quality control by detecting defects more efficiently than human inspectors. Despite their potential, physical AI systems face significant challenges, including the sim-to-real gap, data scarcity, hardware limitations, and safety concerns, which necessitate robust data infrastructure and continuous real-world testing. Roboflow provides essential tools for managing datasets, training models, and deploying them on edge devices, facilitating the transition from prototypes to operational systems across various industries.

ICU Medical's recall of IV gravity burette sets in 2025 due to a missing shutoff valve highlights the importance of verifying medical device assembly to prevent uncontrolled fluid flow into patients. This text outlines a workflow using Roboflow to automate the verification process by leveraging Google Gemini's ability to identify devices from images and infer their necessary components. By comparing the expected components with those visible in the image, the system flags any missing parts and provides a structured report with pass/fail status and confidence levels. The workflow includes several processing blocks, such as image input, JSON parsing, and Python scripting, to ensure accurate detection and logging of components. It can seamlessly adapt to different medical devices without additional setup, and its application is suitable for both cloud and edge device deployments, allowing for high-confidence results to be handled automatically and uncertain cases to be reviewed by humans.

Aarnav Shah's blog post discusses the automation of tire sidewall Optical Character Recognition (OCR) by integrating a custom-trained RF-DETR detector with a multimodal Large Language Model (LLM) to efficiently read and log critical tire information, such as DOT code, size, and brand, into a database. The system is implemented as a Vision Agent within Roboflow Workflows, where the RF-DETR model precisely isolates and crops tire images, allowing the LLM to focus on high-density text for optimal OCR accuracy. This approach aims to alleviate the challenges of manual tire inspection, which is prone to human error, by providing a reliable and expedited method for capturing and documenting tire metadata, thus minimizing the risks associated with incorrect tire specifications and enhancing fleet maintenance. The article outlines a detailed multi-stage framework for setting up this automated pipeline, emphasizing the importance of training data preparation, model training, and performance evaluation to ensure the system's effectiveness in diverse environments.

Track Class Lock is a new feature in Roboflow's workflow aimed at reducing the flickering of object detection labels in videos by stabilizing the classification of objects once a high confidence level is reached. This tool addresses a common issue where an object might be misclassified between frames (e.g., a "bottle" detected as a "remote" momentarily), which can cause downstream problems for applications relying on these labels. The Track Class Lock works by accumulating votes for each detected class across frames, locking the class when it reaches a predetermined confidence margin, and only allowing changes if a new class is consistently identified over several high-confidence frames. This ensures that genuine reclassifications are permitted while temporary errors are ignored, thus providing stable and reliable object tracking. Users can experiment with this feature through a pre-wired workflow setup or build it manually by integrating it with detection models and trackers, enhancing the reliability of video-based object detection systems.

Automated IV bag monitoring can be achieved using Roboflow by training an RF-DETR model to detect IV bags and classify their fill levels, followed by inspection for visible leaks using Gemini 2.5 Pro. This process is increasingly important as the demand for IV therapy grows, driven by factors such as chronic disease prevalence, increased surgical procedures, and a growing elderly population. The tutorial outlines a step-by-step approach to building this system, starting with preparing a dataset of IV bag images, training the model, and evaluating its performance, which achieved high accuracy in detecting and classifying fill levels. The workflow integrates these capabilities into Roboflow Workflows, allowing for a comprehensive IV bag inspection pipeline that includes fill-level detection, leakage assessment, and a text overlay of inspection results on the final annotated image. While the tutorial uses a small dataset as a proof of concept, a production system would require a larger, more diverse dataset to ensure reliability and accuracy in real-world applications, especially in a manufacturing setting where integration with industrial cameras could further enhance quality control processes.

The article, authored by Aarnav Shah, explores the integration of language models (LLMs) into vision pipelines, specifically within the Roboflow Workflow, to enhance object detection systems that traditionally excel at localization and classification but struggle with tasks requiring text interpretation, contextual judgment, and structured output. It discusses scenarios where adding an LLM is beneficial, such as text extraction or handling high visual variability, and situations where it may not be necessary, like when latency or cost constraints are critical. The guide outlines a practical example of building a book cataloging workflow using a two-stage architecture that involves a fast, specialized detector for spatial tasks and a vision-capable LLM for reasoning tasks, highlighting the importance of choosing the right model for the reasoning layer for optimal performance. This approach is applicable to a variety of fields beyond book cataloging, such as retail auditing and industrial inspection, where the combination of detection and reasoning is required.

Vision agents offer an innovative advancement in computer vision by integrating fast object detection models with large multimodal models (LMMs) for reasoning, enabling systems to not only detect objects but also understand and act upon them autonomously. Utilizing Roboflow's RF-DETR model for real-time detection, these agents follow a four-stage process: perceive, reason, act, and iterate, transforming raw visual input into actionable insights without requiring users to write code. This setup involves a perception layer that processes visual data, a reasoning layer powered by LMMs such as Gemini and GPT for interpreting context, and an action layer that executes decisions based on the insights gained. The architecture is designed to be efficient and scalable, isolating relevant image areas for LMM processing to avoid inefficiencies and hallucinations, and employs a closed-loop system to continuously update and improve its outputs. The blog provides a practical example of building a vision agent for hydration monitoring, demonstrating the potential to adapt this framework to various physical tasks by swapping components and configuring workflows through Roboflow's low-code platform.

The text discusses an automated workflow for medical device packaging inspection and UDI label verification using Roboflow's RF-DETR and Gemini 2.5 Pro. This workflow is designed to enhance the quality control of medical device packaging by detecting package damage and verifying the presence and readability of UDI-related markers like barcodes, QR codes, or DataMatrix codes. The RF-DETR model is trained on a general package damage dataset to identify package defects, and the Gemini 2.5 Pro processes annotated images to evaluate UDI marker visibility, providing inspection summaries. The system aims to streamline packaging quality control in production environments by reducing manual inspections and integrating into packaging lines with industrial cameras, enhancing traceability and compliance with regulatory standards.

Claude Fable 5, Anthropic's latest visual reasoning model, is now integrated into Roboflow, enabling users to effortlessly incorporate it into production pipelines without requiring API wiring. Fable 5 has demonstrated strong capabilities in object and document understanding, scoring highly in Roboflow Vision Evals, making it ideal for tasks involving image analysis, document reading, and scene interpretation. However, it struggles with counting and localization tasks, where specialized models like RF-DETR are recommended for better accuracy and cost efficiency. While Fable 5 is the most expensive and slower among its peers, it excels in reasoning tasks, suggesting its use in scenarios where judgment or interpretation is required, while other models might be preferable for high-volume or latency-sensitive applications. Users are encouraged to test Fable 5 within their own systems to determine its suitability, as Roboflow's model blocks allow for easy swapping based on performance needs.

Roboflow now supports the YOLO26 semantic segmentation model, allowing users to label data, train models, and deploy them all within a single platform. Semantic segmentation involves classifying every pixel in an image to create a comprehensive class map, which is particularly useful in applications like autonomous driving and medical imaging. The YOLO26 model, built on the lineage of YOLO11 and YOLOv8, extends the YOLO architecture by bringing real-time performance to dense, pixel-wise predictions. Roboflow offers tools for efficient data labeling, including AI-assisted features, and hosts the training infrastructure, eliminating the need for teams to manage their own hardware. Once trained, models can be deployed via the Roboflow cloud API or on local hardware using Roboflow Inference, with options for low-latency, on-device deployments. This integration streamlines the workflow from data preparation to model deployment, consolidating all steps within the Roboflow environment.

Anthropic's Claude Fable 5, touted as its most advanced model for vision tasks, claims state-of-the-art capabilities, but falls short in real-world settings, ranking 10th on the Roboflow Vision Evals leaderboard with a score of 74.63%. Despite excelling in object understanding and visual reasoning, it lags behind competitors like Google's Gemini 3.5 Flash and OpenAI's GPT-5.4 in overall performance, speed, and cost-effectiveness. The model struggles particularly with object counting in cluttered scenes, highlighting the limitations of current vision-language models (VLMs) in production environments. While Claude Fable 5 excels in visual question answering and document extraction, it is recommended to pair it with fine-tuned models for tasks like object detection and counting, where specialized detectors offer superior accuracy and lower costs.

A vision AI pipeline is presented to improve the accuracy of reading and validating batch numbers and expiry dates on medical packaging lines, addressing the challenges of manual inspections that often miss errors due to varying label layouts and ink density shifts. The process involves training a localization model using Roboflow Workflows to detect and crop the relevant strips, followed by text extraction using Google Gemini, and automated validation of extracted data. This pipeline, applicable beyond pharmaceuticals to various medical device packaging, maintains its structure across different datasets, allowing for integration into manufacturing execution systems to prevent mislabeling and reduce recall risks. The methodology emphasizes the importance of automated verification in maintaining traceability and compliance in medical device manufacturing.

YOLO27, anticipated to be unveiled at YOLO Vision 2026 in Shenzhen, marks the next evolution in the YOLO family of real-time computer vision models, introducing 3D perception through models like YOLO-Depth for monocular depth estimation and YOLO-StereoDepth for binocular disparity depth. This advancement shifts the focus from merely detecting what is in an image to understanding the spatial context and distance, which is significant for applications in robotics, logistics, and manufacturing. While the comprehensive details such as benchmarks, model sizes, task coverage, and licensing terms remain undisclosed, existing models like RF-DETR and Depth Anything V2 offer robust alternatives for depth estimation and object detection until YOLO27's release. These models can be integrated into vision pipelines using platforms like Roboflow Workflows, enabling real-time, depth-aware applications in the interim.

Vision AI solution catalog entries are standardized packages rather than raw models, designed to overcome common challenges in AI deployment across multiple sites by including six key artifacts: a trained model with documented accuracy thresholds, an inference pipeline configured for target hardware, an HMI pattern for operator interaction, an integration template for systems like PLC and SCADA, a deployment runbook, and acceptance criteria for testing at receiving sites. Traditional approaches often fail as each site starts from scratch, leading to siloed knowledge and high costs, while a standardized catalog enables efficient scaling by allowing plants to adapt a pre-vetted solution to local variations, reducing time to value significantly. This methodology is a crucial step in the Vision AI Maturity Model, enabling operational integration by turning successful pilots into reusable assets for an entire network, while avoiding pitfalls such as bespoke integrations and catalog drift. The Roboflow Vision AI Center of Excellence Blueprint provides guidance on building these catalog entries, including a maturity model and roles to manage the process, ensuring that every new deployment is faster and cheaper, driven by a template established through a proven use case.

Message Queuing Telemetry Transport (MQTT) is a lightweight publish/subscribe protocol widely used in manufacturing for real-time data transfer between devices such as PLCs, sensors, and vision models. It efficiently facilitates immediate plant-floor actions by connecting vision AI detections to operational systems without the need for rebuilding existing integrations. MQTT's architecture allows devices to publish messages to topics, enabling multiple subscribers to react instantaneously, making it ideal for industrial environments with constrained networks. This protocol is integral to Industry 4.0 and IIoT architectures, notably aiding in applications like defect detection, inventory counting, safety monitoring, and maintenance alerts. Manufacturers, like those using Roboflow's platform, benefit from MQTT's ability to seamlessly integrate vision AI with existing systems, offering scalability and reduced latency for critical operations. With its focus on decoupling communication processes, MQTT supports new systems joining the network effortlessly while maintaining a structured and secure messaging environment.

The tutorial outlines the process of creating a textile inspection prototype using an RF-DETR model and Roboflow Workflow to detect and alert fabric defects like holes, stains, and thread irregularities. This system automates defect detection and integrates with platforms like Slack for alert notifications, potentially extending to MES systems and PLCs to log incidents and trigger responses without altering the vision pipeline. It emphasizes the efficiency of automated inspections over traditional methods reliant on human operators, thus preventing rejected products and material waste by catching defects early. The tutorial guides through preparing a textile defect dataset, training the model, building the workflow, and configuring Slack alerts, ensuring notifications are issued only when defects are identified. This structured workflow separates detection, visualization, and alert delivery, allowing manufacturers to maintain quality control through integrated systems that track incidents and automate responses.

Building a Vision AI Center of Excellence involves a structured approach outlined in three steps: Define, Pilot, and Codify. The process begins with defining a North Star metric, identifying three use cases of varying difficulty, and establishing a committed working group, which includes a vendor counterpart, an internal engineering lead, and a crucial operations leader. The pilot phase tests the operating model using the easiest use case to ensure any failures are process-related rather than technical, setting up the catalog-entry shape and establishing baseline measurements. Codification involves promoting the first successful solution into a version 0 catalog, setting role assignments, and planning a quarterly portfolio review to transition towards plant-led deployments. This structured sequence ensures the creation of a reusable, scalable system that enhances future AI deployments, as opposed to merely achieving a singular impressive pilot, thereby forming the foundation of a Vision AI Center of Excellence.

The blog post discusses the applications and advantages of using YOLO and Vision-Language Models (VLMs) in computer vision tasks, emphasizing that the choice between them depends on the specific requirements of a project. YOLO, a real-time computer vision model, excels in tasks requiring predefined object detection, speed, and efficiency, making it suitable for environments like manufacturing, traffic monitoring, and security where real-time processing and fixed object categories are crucial. Conversely, VLMs offer flexibility in understanding and reasoning about images, allowing for natural language interaction and open-ended tasks such as visual question answering, making them ideal for scenarios where understanding context or dealing with unknown objects is necessary. The article illustrates the complementary nature of YOLO and VLMs through examples of workflows in Roboflow, highlighting that YOLO is optimized for production environments while VLMs are suited for tasks requiring reasoning and language-based interaction.

A Vision AI program's success relies on three key roles: the Builder, the Scaler, and the Operator, each responsible for different stages of the AI solution lifecycle. The Builder initiates the solution by transforming a use case into a deployable model, the Scaler expands this model across different sites while ensuring adaptability and reducing deployment costs, and the Operator maintains the system's reliability and trust over time. The Roboflow Vision AI Center of Excellence Blueprint emphasizes that these roles can be filled internally, by a vendor, or through a hybrid approach, and suggests that the program should evolve from vendor-led to internally owned as it matures. The effectiveness of the program can be measured by speed of implementation, successful first-attempt deployments, and workforce upskilling, which collectively indicate the transition from a pilot phase to a sustainable capability. The blueprint further outlines how staffing should adapt over time, shifting from vendor-dominant in the early stages to primarily internal staffing, fostering autonomy and reducing dependency on external services.

Cosmos 3 is an advanced foundation model for physical AI, designed to manage vision reasoning and multimodal generation across various media such as text, image, video, sound, and action. Released under the OpenMDW 1.1 license, it comes in two variants, Super (32B) and Nano (8B), and is available on GitHub. The model excels in processing fixed-camera footage, as demonstrated in tests involving an airport gate, a warehouse, and a kitchen assembly line, showing its ability to segment and track slow-changing states more effectively than fast-moving actions. While Cosmos 3 performs well on VANTAGE-Bench, challenges remain, especially with scenes containing many similar small objects and fast actions, highlighting the importance of scene framing and spatial grounding. Despite its strengths, Cosmos 3 still requires iterations of data labeling, training, and deployment to achieve reliable operational use, especially in complex environments like kitchens where ingredient placement and timing are critical.

Vision AI rollouts in manufacturing often fail when a top-down approach, known as the Push Trap, is used, assuming that a solution successful in one plant will work universally, leading to resistance and stalled implementations. The Pull Model offers a scalable alternative by allowing plants to choose from a vetted Standard Solution Catalog maintained by headquarters. This approach not only respects local variations but also ensures that learning and improvements compound across sites. Unlike the Push Trap, the Pull Model avoids the pitfalls of DIY solutions, vendor lock-ins, and underutilized platforms by fostering internal capabilities and providing complete, adaptable solutions. This model has been successfully implemented by organizations like USG, which connected 50 manufacturing sites with robust, scalable vision AI solutions. The strategy involves building a catalog of repeatable solutions rather than focusing on isolated successes, thereby creating a sustainable and efficient deployment process across the entire network.

SnackTrack is an innovative web application designed to monitor eating habits in real-time using computer vision technology. By employing a custom-trained object detection model on Roboflow, the app discerns between apples and cookies, rewarding or penalizing users with points based on their snack choices. The system integrates Roboflow for object detection and MediaPipe for facial landmark detection, allowing it to identify when a snack is near the mouth, thus logging an eating event. Developed with a Flask backend, OpenCV, and a frontend using JavaScript, the application maintains real-time updates through Server-Sent Events (SSE). Users can replicate the project by following a detailed process involving dataset collection, model training, and application setup, with the flexibility to adapt the system to recognize different snacks. The project demonstrates a practical implementation of AI in enhancing personal health monitoring, with the code and dataset publicly available for customization and further exploration.

Integrating computer vision into manufacturing systems using Roboflow enhances the operational efficiency of factories by connecting vision AI with existing industrial software stacks like PLC, SCADA, MES, and ERP through protocols such as EtherNet/IP, MQTT, and OPC UA. Vision AI acts as an advanced sensor, providing detailed product inspections that inform automated decision-making processes across various layers of the manufacturing architecture defined by the ISA-95 model. These layers range from real-time machinery control with PLCs to high-level business planning in ERPs, each requiring specific data outputs from the vision system. Roboflow's platform facilitates this integration by enabling custom-trained models to run edge inference, allowing for rapid defect detection and seamless communication with plant systems. This setup improves production quality, reduces scrap costs, and provides actionable insights, which are crucial for optimizing planning and financial strategies. By leveraging standard protocols, manufacturers can incrementally enhance their existing infrastructure without the need for custom middleware, thus integrating vision AI where it is most needed and allowing data-driven improvements throughout the production hierarchy.

Modern sports analytics AI systems utilize computer vision to process match footage into structured data that can be leveraged by coaches, analysts, and product teams for various applications, such as player detection, tracking, and performance analysis. This advanced technology is exemplified by PlayVision's use in basketball and Fletcher Sports' application in tennis. The text details a tutorial on soccer formation analysis using Roboflow, focusing on detecting players in a single broadcast frame and employing a Vision Language Model to infer tactical formations without relying on pre-labeled data. The process involves training a custom detection model, deploying it through Roboflow workflows, and integrating it with Gemini 2.5 Pro for visual reasoning, ultimately producing structured JSON outputs that outline team formations, attacking directions, and tactical reasoning. The tutorial highlights the potential of sports analytics AI to transform raw video data into actionable insights, enabling significant advancements in tactical planning and performance monitoring without the need for costly hardware installations.

IP cameras and AI cameras serve different purposes in video surveillance and operational decision-making, with IP cameras primarily designed to record and store video footage for later review, while AI cameras analyze video streams in real-time to generate actionable insights and alerts. This shift in functionality is becoming increasingly important in industrial environments, where AI cameras help detect and respond to events such as safety breaches, equipment defects, and operational inefficiencies without the need for continuous human monitoring. Integrated systems like Roboflow's AI1 simplify deployment by combining essential components such as camera hardware, lighting, compute, and AI software into a single device, facilitating real-time computer vision applications in manufacturing and logistics. The adoption of AI cameras is driven by their ability to transform video into structured data that supports quick operational decisions, contrasting with traditional IP cameras that focus on basic recording tasks. As organizations seek to enhance their surveillance capabilities, the choice between IP and AI cameras depends on whether the emphasis is on simple recording or on generating operational insights that can lead to improvements in safety, quality, and efficiency.

Many industrial enterprises find themselves trapped in prolonged pilot phases for vision AI initiatives, with a common misconception being that the number of pilots indicates program health. However, successful scaling is determined by the robustness of the surrounding system, such as a governed solution catalog, clear ownership, and consistent review processes. Roboflow's Vision AI Center of Excellence Blueprint offers a diagnostic tool consisting of ten yes/no questions that help organizations assess their readiness to scale. A score of six or more suggests a functioning capability, while a score of three to five indicates a forming but incomplete Center of Excellence. A score of two or fewer denotes "pilot purgatory," where the program remains stagnant without scalable progress. The diagnostic emphasizes system readiness over superficial metrics, guiding companies in transitioning from pilot phases to scalable, impactful AI deployments.

The discussion focuses on the strategic decision between renting and owning vision AI capabilities, emphasizing the long-term benefits of ownership over renting. Renting entails a vendor-managed setup where deployments are controlled externally, leading to persistent costs and dependency, whereas owning allows an organization to build internal capabilities, ensuring data and model portability. A healthy vendor relationship is characterized by diminishing vendor involvement over time, transitioning from building to consulting, and eventually leaving the organization self-sufficient. The Roboflow Vision AI Center of Excellence Blueprint advocates for ownership, offering a model where vendors initially assist with development but gradually reduce their role, empowering internal teams to take over deployments. This approach is presented as more sustainable and cost-effective, contrasting with rental models that increase dependency.

Anomaly detection using YOLO object detection models provides a detailed approach to identifying visible defects in various industries such as manufacturing, logistics, agriculture, and healthcare. By annotating defect classes like scratches or dents and incorporating clean images as null examples, users can train models in platforms like Roboflow to not only detect the presence of anomalies but also pinpoint their location and classify their type with confidence scores. This process enhances quality control by enabling early detection of defects, thereby reducing waste and improving product quality. YOLO models, owing to their speed and real-time capabilities, are particularly suited for visual inspections where anomalies are labelable, while RF-DETR offers an alternative with transformer-based detection for similar use cases. The tutorial outlines steps from collecting and annotating images to deploying a trained model for real-world application, emphasizing the workflow's ability to provide structured insights that support both human review and automation systems, thus facilitating continuous model improvement through active learning.