September 2026 Summaries

Roboflow has launched Roboflow Labs, an applied research division focused on improving AI systems’ ability to see, reason, and act in three-dimensional real-world environments across industries. The division plans to publish research and develop open edge-deployable models, collaborate with frontier AI labs on model evaluation, distribution, training data, and multimodal environments, and work with enterprises to create benchmarks addressing commercial use cases where current models fall short. Its efforts build on Roboflow’s platform, which supports millions of developers and domain experts who have annotated billions of images and videos and deployed hundreds of thousands of models for applications including cancer research, construction safety, conservation, and drone navigation. Roboflow highlights its Apache 2.0-licensed RF-DETR model, public Playground benchmarks, enterprise partnerships, investor backing, and ongoing hiring for research, engineering, operations, and growth roles in San Francisco.

Roboflow’s manufacturing visual-inspection example combines RF-DETR defect detection, ByteTrack object tracking, Gemini-based visual reasoning, and workflow automation to identify potentially damaged cartons and prepare them for human review. The system uses three workflows: OCR reads a station label from the first video frame, a monitoring workflow detects cartons and damage, tracks packages through an inspection zone, saves image evidence, and issues MQTT notifications for flagged packages, while a triage workflow asks Gemini to classify visible damage, estimate severity, suggest a likely cause, and recommend an operator action. A Python runner coordinates the workflows, preserves original and annotated images, prevents duplicate triage events, submits high-severity cases to a webhook, and records outcomes in a JSON report. The approach emphasizes that model assessments, MQTT acknowledgements, and webhook responses do not independently confirm real-world action or correctness, so it recommends evaluating detection, tracking, reasoning, evidence accuracy, escalation performance, and delivery separately while retaining human review for uncertain or invalid cases.

Class imbalance in manufacturing defect detection can cause models to perform well on common defects while missing rare but important ones, making per-class AP a more useful metric than overall mAP. Using an RF-DETR Small model on a fabric dataset with a 50:1 Stitch-to-Hole imbalance, the study compared data-level approaches while keeping evaluation splits and training configuration constant. It recommends collecting additional real rare-defect examples through active learning when visual diversity is insufficient, oversampling when existing rare examples are representative but infrequent, targeted augmentation when known production conditions cause errors, and synthetic data when important scenarios are difficult to capture. Adding roughly 200 images through each intervention, oversampling produced the best improvement in Hole mAP@50, increasing it from 0.135 to 0.157, while synthetic data reached 0.151; targeted brightness, blur, and rotation augmentation reduced performance to 0.050, indicating that augmentations must reflect meaningful real-world variation. Per-class weighting and focal loss can alter training emphasis but cannot replace missing visual examples, and the recommended workflow is to version datasets, retrain under consistent conditions, and compare rare-class performance after each change.

Sobel Edge Detection is a computationally efficient computer vision method for identifying object and region boundaries by measuring horizontal and vertical changes in pixel intensity with two 3×3 convolution kernels. Its workflow typically converts an image to grayscale, calculates horizontal and vertical gradients, combines them into a gradient magnitude map, optionally determines gradient direction, and applies a threshold to create a binary edge image. Sobel provides useful edge-strength and orientation information with modest inherent smoothing, making it suitable for feature extraction, texture and shape analysis, image enhancement, and preprocessing, but it remains sensitive to noise, can generate thick edges, and may detect texture or shadow details that are not relevant boundaries. Compared with Canny detection, Sobel is simpler and faster but generally produces less precise and noisier results because Canny adds smoothing, non-maximum suppression, double thresholding, and hysteresis. The post also explains how to create, test, configure, and deploy a Sobel workflow in Roboflow Workflows through a Custom Python block or Roboflow Agent, with options for cloud or local deployment on image, video, webcam, and RTSP inputs.

Agentic computer vision combines specialized visual perception, contextual reasoning, controlled actions, and verification into a closed-loop system that can move beyond detecting objects to completing operational tasks. The approach uses fast models such as RF-DETR to identify structured visual evidence, tracking and temporal logic to recognize meaningful events over time, and vision-language models such as Gemini to interpret ambiguous situations using policies and scene context. Roboflow Workflows is presented as an orchestration platform for connecting these components with integrations including notifications, databases, webhooks, and industrial protocols, while limiting models to approved, bounded action choices. A dock-door monitoring example illustrates the design: a workflow detects and tracks trucks, triggers analysis only when a truck exceeds a dwell-time threshold, sends a context-rich frame to Gemini to classify the dock as loaded, idle, or blocked, and returns structured results for display, storage, or escalation. The article emphasizes that VLM reasoning should supplement rather than replace deterministic vision and control systems, particularly for safety-critical, hard-real-time, regulated, or simple high-volume tasks. It recommends evaluating perception, reasoning, actions, and verification separately, logging decision evidence, constraining action spaces, applying thresholds, and retaining human review for consequential decisions.

Roboflow announced pricing and product changes aimed at making vision AI development and deployment more accessible, predictable, and affordable. Effective September 18, 2026, its Serverless API will shift from variable GPU-hour billing to model-specific per-image pricing, with volume discounts beginning at 250,000 images per month and claimed cost reductions for most workspaces, including over 40% for some RF-DETR models. The previously paid Core Plan now offers a free tier with 10 credits, private projects, model training and evaluation tools, model-weight downloads, and unlimited seats; the credits are estimated to support roughly 30 training runs or 80,000 RF-DETR Nano inferences. Roboflow Inference for locally hosted models and workflows will no longer consume credits on any plan, while advanced edge deployment capabilities and certain commercial licenses remain Enterprise offerings. Existing customers will retain their current pricing terms while gaining added features, seat charges will end, and Public Plan workspaces will move to the free Core tier, with individualized transition details to be provided by email.

Roboflow’s evaluation finds GPT-6 Astra to be its strongest tested vision-language model, leading benchmarks in object detection, counting, and visual reasoning while supporting text-based and example-based box prompting, segmentation, video re-identification, and robot-control experiments. Astra achieved 82.1% mAP@50 for detection at low reasoning effort, with notable performance on fine-grained categories and context-dependent concepts, and it can use positive and negative visual examples to substantially improve results or transfer targets across images. Although its language understanding helps it identify and label regions more accurately than SAM 3 in some segmentation tasks, SAM 3 produces more precise boundaries, making a combined Astra-plus-SAM workflow potentially preferable. In video, Astra maintained player identities, assigned teams, and read jersey numbers across sampled basketball footage, but its per-frame cost makes it better suited to periodic high-level interpretation alongside local detectors and trackers. Robot demonstrations suggested that Astra can use visual feedback to guide actions, though outcomes remain highly dependent on calibration, controllers, and physical setup. Despite its performance, Astra is expensive and relatively slow, with higher reasoning effort offering modest gains at substantially greater cost, so Roboflow recommends testing cheaper models first and validating Astra on task-specific datasets.

Roboflow reports that GPT-6 Astra ranked first in its vision evaluations as of September 17, 2026, demonstrating strong object detection, contextual text recognition, and the undocumented ability to generate object-outline polygons through structured JSON prompts. While Astra’s polygon outputs can support tasks such as counting, cropping, and labeling, they are not pixel-perfect dense masks and can become costly because detailed polygon vertices consume output tokens. The post recommends combining Astra with Segment Anything Model 3 (SAM3): Astra identifies and classifies objects with bounding boxes, particularly distinguishing visually similar or complex classes such as cashews and hazelnuts, while SAM3 converts each box into a precise mask without needing to classify the object itself. This two-model workflow is presented as more accurate and often cheaper than requesting Astra polygons directly, whereas SAM3 alone may misclassify look-alike objects. Roboflow also notes that the resulting labeled data can be used to train an RF-DETR real-time segmentation model.

Laplacian Edge Detection is a computer vision method that identifies object and region boundaries by measuring second-order changes in pixel intensity, unlike Sobel detection, which uses first-order gradients. Its typical workflow converts an image to grayscale, applies Gaussian blur to reduce noise, convolves the image with a Laplacian kernel, finds zero crossings where responses change sign, and thresholds weak responses to create a binary edge map. The method is rotation-invariant, can localize thin edges precisely, and captures fine details, but its high sensitivity to noise can produce false or double edges and it does not provide edge-direction information. Compared with Sobel and Canny, Laplacian detection is simpler and potentially sharper than Sobel but generally noisier, while Canny uses a more complex multistage process to produce cleaner, more connected edges. The article also demonstrates integrating the technique into Roboflow Workflows through a Custom Python block, allowing users to configure threshold inputs, process images or video, and deploy the resulting workflow through cloud or local infrastructure.

Canny Edge Detection is a widely used computer vision algorithm that identifies object and region boundaries by finding abrupt changes in image intensity while reducing noise and false detections. Developed by John Canny, it processes images through five stages: grayscale conversion and Gaussian blurring for noise reduction, Sobel-based gradient calculations to measure edge strength and direction, non-maximum suppression to thin edge responses, double thresholding to classify strong and weak edges, and hysteresis to preserve weak edges connected to strong ones while discarding isolated noise. The method produces thin, defined edge maps useful in industrial inspection, medical imaging, OCR, facial analysis, architectural imaging, and satellite imagery. The post also explains how Roboflow Workflows can implement Canny detection through a Custom Python block or an AI-generated workflow, with adjustable low and high thresholds, and describes deployment options for cloud or local hardware processing of images, videos, webcams, and RTSP streams.

GPT-6 Astra accepts text and images but not video files, so video understanding can be achieved by sending timestamped image frames in a request and requiring structured JSON event outputs. Roboflow tested this approach for tennis event recognition, taco-line ingredient monitoring, and loading-dock activity timelines, finding that Astra could identify actions and events with useful temporal accuracy when sampling rates matched the task’s complexity. Tennis required at least 4 frames per second to reliably determine point outcomes, while slower sampling was sufficient for food preparation and logistics monitoring. Processing roughly 100 frames takes two to three minutes and costs about $1, making continuous video analysis expensive at scale. The post recommends combining conventional detectors or classifiers for routine visual states, such as truck presence or gate position, with Astra for harder contextual action-recognition tasks like distinguishing loading from unloading, reducing production costs substantially.

Line scan cameras create long, continuous images of moving materials such as steel, fabric, and coated films, but these strips are poorly suited to object-detection models that require smaller, roughly square inputs. The described Roboflow Workflow addresses this by using an Image Slicer to divide strips into overlapping 640×640 tiles, an RF-DETR model to detect defects in each tile, and Detections Stitch to remap predictions to the original image and use non-maximum suppression to remove duplicates at overlapping boundaries. Overlap preserves complete views of defects that might otherwise be split across tiles, while stitching consolidates repeated detections into single boxes. The tutorial uses a 1,900-image steel surface defect dataset with six classes, trains an RF-DETR model that achieved 74.7% mAP@50, and creates a synthetic strip by joining test images to demonstrate the approach. For production use, the workflow can be adapted with real line-scan imagery and models trained on specific materials, with tile dimensions, overlap, confidence thresholds, and duplicate suppression tuned to balance detection accuracy and processing speed.

Roboflow’s webinar demonstrates an edge-based computer vision system for end-of-line inspection that identifies missing components on a carburetor assembly before shipment. Using side and overhead cameras, a rotating pedestal, RF-DETR Seg instance segmentation, and labels for both present components and expected missing-part locations, the system detects items such as screws, nuts, valves, and identifying marks while helping operators locate problems. The model was trained on 680 varied images with augmentations and annotation assistance from SAM 3 and prior custom models, emphasizing realistic camera, lighting, and component-view conditions. Roboflow Workflows combines preprocessing, segmentation, ByteTrack tracking, stabilization, counts, and visual overlays into an HMI that displays camera feeds and alerts that clear when components are reinstalled. Deployed locally on an NVIDIA Jetson through Roboflow Inference, the demonstrated application processes two camera views at roughly 8 to 9 frames per second, avoiding cloud latency and supporting on-site image processing.

Roboflow Enterprise Offline Mode enables computer vision systems to run models and Workflows entirely on local hardware for facilities where camera data, inference outputs, and production decisions must remain within isolated networks. The approach supports fully disconnected, internally isolated, and restricted-egress environments, with local Inference Servers processing camera or RTSP streams and delivering results to plant systems through integrations such as OPC UA, PLC, Modbus, MQTT, databases, and local files. Before disconnecting, teams must cache model weights and Workflow definitions using a persistent Docker cache, then enable offline loading; cached models have 30-day leases that require planned renewal, while network isolation itself must be enforced separately at the operating-system or network level. Local Workflows can include detection, segmentation, OCR, tracking, filtering, quality decisions, and supported local vision-language models, though cloud-dependent blocks such as OpenAI, Gemini, and Slack cannot operate in a strict air gap. The deployment example outlines a PCB inspection pipeline in which RF-DETR analyzes an RTSP stream on a local GPU system and writes pass/fail results to an OPC UA server. Model improvement remains possible through a manual feedback loop that collects selected production cases locally, transfers approved samples to a connected environment for labeling and retraining, and caches updated models and Workflows before returning them to production.

Roboflow’s Auto Label feature uses GPT-6 Astra as its default zero-shot object detection model, allowing users to provide clear class names and automatically generate bounding boxes without training examples or manual annotation. The workflow can be completed through Roboflow’s interface by importing raw images, creating an object detection project, entering classes such as hard hat, mask, and gloves, running a preview, and applying labels across a batch, or through the Roboflow MCP server, where compatible AI agents can initiate the same process from a natural-language prompt. Astra relies only on class names rather than descriptions, returns all detections without confidence filtering, and produces boxes rather than segmentation masks, making human review essential to remove duplicate or inaccurate boxes, adjust loose annotations, correct classes, and add missed objects. Once reviewed and approved, labeled images can be added to a dataset for versioning, training, or export. Compared with Gemini and SAM 3, Astra is positioned as a contextual, reasoning-oriented option for detection tasks, while other models may be more suitable for multimodal analysis or precise segmentation.

Gaze detection estimates where a person is looking at a given moment, whereas gaze tracking follows those estimates across time to analyze attention patterns, fixation duration, and gaze movement. A typical computer-vision system processes camera or video input by detecting faces, cropping and preparing face regions, estimating gaze direction from eye, facial, and head-pose cues with models such as L2CS-Net, mapping direction to a screen position or scene object when needed, and associating predictions over frames through multi-object tracking and smoothing. The guide demonstrates a Roboflow-based implementation using Workflows for face detection with SAM 3, OC-SORT identity tracking, and detection stabilization, alongside Roboflow Inference and Python to run gaze estimation, store frame-level detections, draw smoothed gaze arrows, and reconstruct an annotated video. Accuracy and real-time performance can be affected by lighting, resolution, blur, occluded eyes, rapid gaze changes, false detections, and inconsistent tracking, but the technology has uses in accessibility, human-computer interaction, gaming and virtual reality, usability research, medical studies, and monitoring attention in automotive and aviation settings.

Edge AI inference in manufacturing runs trained computer vision models near production lines to meet strict real-time requirements, avoid transmitting high-bandwidth camera feeds, and keep inspections operating during WAN outages. The approach separates cloud-based model training, analytics, and fleet administration from local inference, where cameras analyze parts for defects, assembly errors, OCR compliance, counts, and safety conditions before sending machine-readable results to PLCs through protocols such as OPC UA, Modbus TCP, EtherNet/IP, or digital I/O. Hardware options range from NVIDIA Jetson devices and industrial PCs to Raspberry Pi systems and integrated AI cameras, with selection based on camera throughput, environmental conditions, I/O, lifecycle support, and total pipeline capacity rather than model speed alone. The guide recommends using compact, optimized models such as RF-DETR Nano, validating quantization and accuracy against representative production data, deploying local execution through Roboflow Inference, and keeping PLCs responsible for deterministic timing, safety interlocks, and actuator control. Scaling across multiple lines requires fleet monitoring, controlled updates, IT/OT network segmentation, security controls, offline caching, and records for validation, traceability, quality systems, and continuous model improvement from production events.

Confidence thresholds in object detection determine the trade-off between false positives and missed detections, making them a deployment decision based on the relative cost of errors rather than a setting that should be left at a default value or inferred from mAP. The recommended process is to use a precision-recall curve to select several candidate thresholds, test them under otherwise identical conditions in a Roboflow Workflow, and validate the preferred option on a fixed, annotated golden set that reflects real production conditions such as varying lighting, distance, occlusion, and scene density. In a warehouse RF-DETR example, a threshold of 0.39 provided a better balance of 98.6% precision and 93.6% recall than lower or higher alternatives, though performance differed by object class. When classes require different trade-offs, per-class confidence filters can apply stricter or more permissive cutoffs while retaining a global default. Threshold changes cannot solve all detection issues: duplicate boxes should generally be addressed with non-maximum suppression and its IoU setting, while uncertain predictions can be routed to human review, active learning, model consensus, or secondary verification. Thresholds should be revisited whenever the model, production data, or the operational cost of errors changes.

A Roboflow tutorial describes an active-learning workflow for identifying computer vision edge cases that standard validation metrics may overlook because production images often differ in lighting, texture, camera setup, occlusion, and defect rarity. Using fabric-defect detection as an example, it trains a baseline model on plain-background images, then applies the model to a separate set of unlabeled patterned-fabric images and flags samples with no detections or prediction confidence below 0.60. A Roboflow Workflow counts detections, finds minimum confidence, selectively sends flagged images to Google Gemini for explanations of likely visual challenges, annotates predictions, and exports results to a CSV for review. Users then filter flagged images, manually label them in Roboflow Annotate, add them to the training dataset, and retrain the same model for comparison. In the example, this targeted addition of difficult images increased mAP@50 from 65.7% to 71.4%, precision from 81.0% to 86.5%, recall from 72.2% to 77.3%, and F1 from 68.9% to 81.5%, illustrating how repeated production-data review can reduce missed detections and help models adapt to real-world distribution shifts.