November 2025 Summaries

Roboflow's labeling interface provides a range of tools to annotate images, including the drag and select tool, bounding box, polygon, brush annotation tools, and smart polygon feature, each offering unique capabilities for various annotation needs. Users can modify annotations by resizing, moving, or changing their labels, and they can utilize features like the Class Selector for precise label management. The interface also supports automatic labeling through Label Assist, which employs pretrained models, and offers functionalities for managing null annotations, undoing, redoing, and repeating actions. Additionally, users can zoom into images for detailed work and utilize annotation lists and image attributes to organize and manage datasets effectively.

The Roboflow dashboard provides a comprehensive labeling interface to facilitate image annotation using various tools such as bounding boxes, polygons, and brushes. The toolbar, found on the right side of the interface, offers features for creating precise annotations, including the Drag and Select tool for editing and moving annotations, the Bounding Box and Polygon tools for outlining objects, and the Brush tool for pixel-level mask annotations. The Smart Polygon tool enhances segmentation tasks by allowing detailed polygon adjustments, while Label Assist utilizes models for automatic labeling. Users can manage annotations through features like undo, redo, and class selection, and employ the Zoom tool for detailed editing. The Annotation List and Image Attributes provide organizational and informational support, ensuring efficient annotation processes within datasets.

Meta has unveiled SAM 3D, a pioneering generative model that enables comprehensive 3D reconstruction from a single 2D image, designed to accommodate real-world complexities like occlusion and cluttered environments. SAM 3D consists of two specialized models: SAM 3D Objects, which focuses on robust object and scene reconstruction, and SAM 3D Body, which provides detailed human body pose and shape estimation. The models utilize a multi-stage training pipeline blending synthetic and real-world data, leveraging transformer-based architectures to enhance predictive accuracy and efficiency. SAM 3D's innovative features include the Mixture-of-Transformers architecture for structured attention and a dual-decoder design for flexible output formats. This advancement opens new avenues for applications in augmented reality, robotics, gaming, and creative industries, offering near real-time, high-quality 3D understanding of natural images. Additionally, Meta has made these models accessible through open-source repositories, allowing for further research and development.

Roboflow Annotate offers a versatile and efficient platform for annotating images, allowing users to employ bounding boxes and polygons for this purpose. Users can manually draw these annotations or utilize advanced tools such as Label Assist, which leverages model checkpoints for annotation recommendations, and Smart Polygon, which enables quick polygon creation. Additionally, Label Assist with the Segment Anything Model (SAM) provides enhanced precision in polygon annotations. While bounding boxes are simpler and quicker to implement, polygons offer greater accuracy, which can improve model performance, particularly for segmentation tasks where polygons or masks are necessary for precise item separation. Masks are the most detailed option, providing pixel-perfect control for complex boundaries, making them ideal for high-fidelity segmentation needs.

Image augmentation is a technique used to enhance the performance of machine learning models by applying various transformations to training images, improving their ability to generalize to new data. It is recommended to initially train models without augmentations to establish a baseline performance, and only introduce augmentations if the model does not perform well. Augmentations are applied offline to increase reproducibility, reduce training time, and lower costs since they are CPU-intensive processes that can delay GPU operations during training. There are two main types of augmentations: image-level, which alters the entire image to simulate diverse visual conditions, and bounding box-level, which specifically modifies content within bounding boxes for more targeted improvements. Roboflow offers both basic and enhanced augmentation options, with enhanced features being premium offerings, and users can manage the number of augmented images through the "Maximum Version Size" setting to control dataset size and composition.

Segment Anything 3 (SAM 3), developed by Meta and released on November 19, 2025, is a cutting-edge zero-shot instance segmentation model that can generate precise segmentation masks for objects in images or videos using text or point prompts. This guide provides a comprehensive walkthrough on fine-tuning SAM 3 with a custom dataset to enhance its performance in specific domains, demonstrating steps such as dataset preparation, version creation, and initiating a training job. It also details deploying the fine-tuned model using Roboflow Workflows, which allows for cloud or local deployment via a serverless API or an open-source inference server. SAM 3's unique feature, Promptable Composable Segmentation (PCS), enables it to identify objects based on text prompts, making it a versatile tool for various segmentation tasks.

Promptable Concept Segmentation (PCS) represents an advancement in computer vision that allows segmentation models to identify and label objects based on natural language prompts or example images, rather than being confined to a predefined set of object categories. This approach enhances the flexibility and scalability of segmentation, enabling more human-like interaction with vision systems by allowing descriptions such as "red apple" or "striped cat" to guide the segmentation process. PCS is particularly impactful in fields like autonomous navigation and medical imaging, where it facilitates the recognition of unique or rare objects not covered by traditional segmentation methods. The third generation of the Segment Anything Model (SAM 3) incorporates a dual encoder-decoder transformer architecture, combining text and image processing to deliver prompt-controlled, open-vocabulary segmentation. Additionally, PCS utilizes a large-scale dataset, SA-Co, and a data engine involving human-model-in-the-loop annotation to ensure comprehensive training and evaluation, thus bridging the gap between vision and language processing in AI systems.

Released by Meta on November 19, 2025, Segment Anything 3 (SAM 3) is an advanced zero-shot image segmentation model capable of detecting, segmenting, and tracking objects in both images and videos using concept prompts. Unlike its predecessors, SAM 3 supports Promptable Concept Segmentation (PCS) and open-vocabulary instance detection, allowing it to identify multiple instances of a concept with unique masks and IDs simultaneously. This model integrates several segmentation paradigms, including single-image and video processing, interactive refinement, and concept-driven detection, unified under a single interface. SAM 3 can run efficiently on various platforms, such as the Roboflow Playground, and can be fine-tuned for domain-specific tasks. The model is licensed under a custom license by Meta, facilitating public use and adaptation. SAM 3's architecture builds upon previous versions with improvements in zero-shot segmentation, small object performance, boundary accuracy, and robustness across domains, and is supported by a comprehensive dataset known as the SA-Co dataset, developed through a multi-phase data engine involving AI and human annotators.

Segment Anything 3 (SAM 3) is introduced as a cutting-edge vision foundation model integrated into the Roboflow platform, designed to transform computer vision application development by detecting, segmenting, and tracking objects in images and videos based on prompts. SAM 3's integration, in partnership with Meta, enhances Roboflow's capabilities, allowing users to create production-ready custom visual applications efficiently. The model, which can be accessed through Roboflow's cloud infrastructure or deployed locally, supports rapid prototyping, fine-tuning, and automated data labeling, addressing the challenges of building vision applications. SAM 3's compatibility with Roboflow's ecosystem facilitates testing against other models, such as Gemini and Claude, offering a versatile tool for developers. Additionally, a new benchmark, SA-Co, has been created to evaluate SAM 3's performance in open-vocabulary recognition tasks.

RF-DETR Segmentation, released in October 2025, is a cutting-edge instance segmentation model designed to precisely locate objects in images at the pixel level. The accompanying guide details training an RF-DETR Segmentation model using the rfdetr Python package, focusing on detecting cracks in concrete, and describes the process of preparing datasets in the Microsoft COCO JSON Segmentation format, configuring the training environment with necessary dependencies, and employing optimal training strategies like gradient accumulation. The guide further explains deploying the trained model to the cloud using Roboflow Workflows, which allows for the creation of computer vision applications with features such as object tracking and visualization of model predictions. The model can be integrated into a serverless cloud API or deployed on personal hardware, showcasing its versatility and practical application potential.

Annotated datasets are crucial for solving computer vision problems like object detection, and over time, various file formats such as PASCAL VOC XML and COCO JSON have become standard for these annotations. This proliferation of formats often necessitates time-consuming conversions between them, detracting from more valuable tasks like enhancing deep learning models. PASCAL VOC XML, funded by the European Union, provides annotations in XML format for each image, while COCO JSON, originating from a Microsoft paper, offers a JSON format that supports a variety of vision tasks across large datasets. Tools like Roboflow simplify the conversion process, allowing users to convert between numerous annotation formats with ease, thus facilitating experimentation across different machine learning frameworks. Additionally, the text highlights the importance of cleaning datasets to fix inconsistencies and suggests best practices for splitting datasets to ensure consistency and avoid version drift.

Resizing images is a vital step in computer vision preprocessing, as it can significantly speed up machine learning model training by reducing the number of pixels the network needs to process. Different strategies for resizing must be carefully considered to avoid distortion and maintain the integrity of the images, especially when dealing with varied image sizes or converting images to square formats. Techniques such as progressive resizing, maintaining aspect ratios, and using padding can help optimize image preprocessing, with the choice of method depending on the specific requirements of the model architecture and the dataset's characteristics. Generally, it is more effective to downsize larger images to match smaller ones rather than stretching smaller images, as stretching can obscure key features. Various software libraries like OpenCV, PIL, PyTorch, and TensorFlow offer tools for resizing, and it is crucial to maintain consistency across the dataset to ensure the model learns from distortion-free inputs.

Inference latency, a crucial metric in machine learning, measures the time a model takes to generate a prediction after receiving input, significantly impacting the performance and usability of real-world applications like autonomous vehicles and industrial safety systems. The inference process consists of several stages, including input processing, model inference, and post-processing, each contributing to the overall delay. Factors influencing inference latency include model architecture, hardware capabilities, precision formats, and batch sizes, with potential trade-offs between latency and throughput. To minimize latency without compromising accuracy, strategies such as model pruning, quantization, knowledge distillation, and hardware-specific optimizations are employed. Additionally, deployment tools like Roboflow Inference enhance real-time performance by optimizing inference pipelines and supporting local deployments, thus ensuring low-latency, reliable predictions essential for applications requiring immediate responses.

As of November 2025, Roboflow has been recognized as the top image recognition product on G2, a prominent software review platform, outperforming major competitors like Google, Amazon, and Microsoft, with a high user rating of 4.8 out of 5. The company's success is attributed to its user-friendly approach and its ability to integrate computer vision into diverse workflows, serving over a million users globally, including more than half of the Fortune 100 companies. Roboflow has been instrumental in helping organizations like BNSF Railway and USG Corporation leverage AI to enhance operational efficiency and transition from traditional processes to innovative, AI-driven methods. The platform's capabilities extend beyond manufacturing, as demonstrated by Ogilvy's project with Audi, which uses vision AI for consumer engagement, and Ulteig's engineering applications, showcasing its versatility across various industries. Roboflow offers comprehensive tools for building, training, and deploying computer vision models, supported by advanced architectures like RF-DETR, accommodating both edge devices and cloud-based solutions.

Depth estimation is a computer vision task that predicts the distance between the camera and objects in an image, resulting in a depth map that is crucial for applications like autonomous driving and augmented reality. Several models are explored, including Depth Anything V2, DepthCrafter, MiDaS, Depth Pro, Marigold, and FoundationStereo, each with unique strengths and weaknesses. Depth Anything V2 is noted for its efficiency and accuracy, particularly in complex scenes, while DepthCrafter excels in video depth estimation with temporal consistency. MiDaS shows strong cross-dataset transferability, and Depth Pro offers high-resolution metric depth maps without requiring camera intrinsics. Marigold, leveraging pretrained generative models, excels in fine detail but is relatively slower, whereas FoundationStereo extends zero-shot capabilities to stereo depth estimation. The article also discusses how to implement these models using Roboflow Workflows for tasks such as object detection and depth estimation, highlighting Depth Anything V2 for its balance of speed and accuracy.

Image segmentation is a sophisticated computer vision technique that provides a pixel-level understanding of images by creating masks that outline the exact pixels occupied by each object. It contrasts with image classification, which assigns a single label to an entire image, and object detection, which uses bounding boxes to localize objects. Image segmentation is divided into semantic, instance, and panoptic categories, each offering varying levels of detail and object distinction. Traditional methods such as thresholding, color-space segmentation, edge detection, and watershed segmentation have been used historically but have limitations like sensitivity to lighting and texture changes. Deep learning advancements have revolutionized segmentation, with modern models like U-Net, Mask R-CNN, and panoptic segmentation models offering superior accuracy and contextual understanding. Newer approaches, like the Segment Anything Model (SAM), allow for flexible, interactive segmentation guided by human input. Roboflow simplifies the segmentation process through tools for data annotation, model training, and workflow deployment, supporting practical applications in fields like manufacturing quality control, medical imaging, agriculture, autonomous vehicles, and environmental mapping.

In the realm of machine learning, particularly in computer vision, the quality of input data plays a crucial role in the accuracy of model outputs, emphasizing the importance of high-quality data collection and effective preprocessing and augmentation techniques. Image preprocessing involves formatting images to meet specific requirements, such as resizing and orienting, to improve model performance and efficiency, while image augmentation creates variations of images to expose models to diverse scenarios, thereby enhancing their ability to generalize from training data. Preprocessing steps are applied to both training and test datasets, whereas augmentation is exclusive to training data, helping models learn from a wider range of situations, especially when datasets are limited. Understanding and correctly applying these techniques, such as resizing, orientation correction, grayscale transformation, and random flips, are pivotal to optimizing model performance, with tools like Roboflow offering support for these processes by automating annotation corrections and providing evaluation metrics.

In the article "When to Use Grayscale as a Preprocessing Step," Joseph Nelson explores the conditions under which converting images to grayscale can be beneficial or detrimental in machine learning models. He explains that while grayscale images are computationally efficient due to their single-channel nature, they may reduce model accuracy in tasks where color is a significant factor, such as distinguishing between similar colors like traffic light states or lane markings in self-driving cars. Nelson suggests that for shape-dominant tasks, grayscale can simplify inputs without losing accuracy, and training models on both color and grayscale images can enhance generalization. He also highlights other color spaces like HSV, LAB, and YUV as alternatives to grayscale, offering advantages in specific contexts such as improving lighting robustness or stabilizing contrast. Additionally, using grayscale can improve inference speed and reduce model size on resource-constrained devices, though the gains might not be substantial on modern hardware.

In computer vision projects, the quantity of data is crucial, but the quality and diversity of the dataset are equally important for developing effective models. Simply increasing the number of images does not linearly improve model accuracy; rather, the gains diminish as the dataset grows, following a power-law curve. The key is to balance the dataset size with variety and label accuracy, ensuring it represents the real-world scenarios the model will encounter. This involves including different lighting conditions, angles, and contexts while maintaining class balance and addressing labeling quality. Effective data management also involves splitting the dataset into training, validation, and testing subsets properly, employing active learning to focus on the most informative images, and using computational resources efficiently. By adopting these strategies, one can build robust computer vision models that perform well in production with a manageable amount of data, leveraging tools like Roboflow for streamlining the process from data collection to deployment.

Object detection on iOS has evolved into a robust on-device capability, facilitating real-time applications such as augmented reality and safety monitoring without relying on the cloud. The transition to on-device processing is powered by efficient models optimized for Apple's Neural Engine, offering enhanced performance and privacy. Key models like Roboflow's RF-DETR, YOLO11, MobileNet SSD, and EfficientDet stand out for their ability to be deployed on iOS devices via CoreML or Swift SDKs, each providing unique advantages in terms of speed, accuracy, and resource efficiency. Criteria for selecting the best model for iOS deployment focus on compatibility with CoreML, real-time performance, memory efficiency, benchmark accuracy, and quantization capabilities. RF-DETR, for instance, is praised for its transformer-based architecture that aligns well with Apple's neural processing, while YOLO11 and MobileNet SSD offer proven performance and efficiency for various use cases. EfficientDet introduces a compound scaling strategy, enhancing its multi-scale detection capabilities. Overall, deploying these models on iOS involves understanding constraints like the Apple Neural Engine's capabilities and optimizing models through quantization and custom domain training for efficient, real-time applications.

Microsoft's Custom Vision Service is scheduled to be retired on September 25, 2028, prompting users to consider alternatives like Roboflow for their AI needs. Roboflow offers an end-to-end vision AI platform that simplifies the process of building, training, and deploying custom computer vision models, even for those without extensive machine learning expertise. The platform enhances efficiency with features like model-assisted labeling, annotation heatmaps, and Smart Polygon for quick annotations, along with access to over 50,000 pre-trained models for auto-labeling images. It ensures platform flexibility by allowing seamless integration with various tools and workflows and supports deployment on multiple devices, including NVIDIA Jetson, Raspberry Pi, and mobile devices. Additionally, Roboflow's active learning feature enables models to automatically improve by re-annotating and retraining on low-confidence data. The migration process from Microsoft Custom Vision to Roboflow involves exporting datasets in COCO JSON format, utilizing a Python script to convert data, and then importing it into Roboflow for model training and deployment. This transition offers users enhanced control, flexibility, and a streamlined path from dataset to production.

Pose estimation, a key computer vision technique for identifying anatomical landmarks on the human body, has advanced significantly with the evolution of transformer architectures and vision language models. Modern models like YOLO11 Pose, MediaPipe Pose, and HRNet deliver unprecedented accuracy while maintaining real-time performance, facilitating applications in fitness, sports analytics, healthcare, and more. YOLO11 Pose excels in balancing accuracy and speed, making it suitable for a wide range of applications, while MediaPipe Pose and MoveNet are optimized for mobile and edge devices. HRNet offers precise localization for detailed movement analysis, and ViTPose represents cutting-edge transformer-based accuracy for human pose estimation. These models cater to diverse deployment needs, from mobile apps to cloud-based systems, and support customization for domain-specific applications. With robust frameworks and user-friendly deployment options, these models are transforming industries by enabling real-time, high-precision human pose analysis.

Audi Reader, developed by Ogilvy Spain using Roboflow, exemplifies how consumer experience AI can transform products into intuitive interfaces by utilizing camera-native technology to explain features instantly. This innovation allows drivers to use the MyAudi app to aim their phones at dashboard icons or buttons, instantly receiving an explanation and actionable steps like booking service, all without consulting a traditional manual. The AI system, trained on over 10,000 images of Audi models, employs a two-stage vision pipeline for icon detection and classification, demonstrating improved user interaction by addressing the gap where product capability often outpaces user comprehension. This approach not only enhances user experience by making information accessible in real-time but also sets a precedent for other consumer products, from kitchen appliances to power tools, to adopt similar AI-driven interfaces. The Audi Reader project highlights how context-aware, camera-native interfaces can revolutionize consumer interactions, offering immediate comprehension and action without the need for deep technical expertise in AI development.

Industrial robotics is undergoing a significant transformation with the integration of Vision AI, allowing robots to adapt and function in unstructured environments where traditional robots struggle. Almond, a company specializing in AI-powered manufacturing, exemplifies this shift by enabling robots to handle complex tasks without major modifications to existing manufacturing lines. Vision AI empowers robots to perceive their surroundings, make data-driven decisions, and adapt to variability, as demonstrated in Almond's project involving the sorting of complex sheet metal parts. The development of vision-enabled robots is facilitated by tools like Roboflow, which streamline the process from data collection and labeling to training and deployment. Almond's approach combines multiple vision models for precise object detection and sorting, enabling robots to achieve human-level accuracy. This innovation results in agile, affordable robotics solutions that offer rapid ROI, transforming robots into adaptable, collaborative partners in manufacturing environments. The future of this technology lies in the convergence of vision, language, and foundational models, allowing seamless human-robot interaction and adaptive task execution.

Object detection is a key task in computer vision that involves identifying and locating objects within images, transforming visuals into structured data for various applications such as manufacturing, agriculture, and warehousing. Mobile object detection extends this capability to devices like smartphones and drones by optimizing models to run locally, providing benefits like instant results, enhanced privacy, offline functionality, and reduced costs by eliminating the need for cloud-based processing. Technologies such as TensorFlow Lite, Core ML, and ONNX facilitate the deployment of machine learning models on edge devices, while frameworks like RF-DETR and ncnn enhance model performance and efficiency. These advancements enable real-time, on-device decision-making across diverse settings, from inspecting solar panels with drones to real-time video analysis, all while maintaining user privacy and reducing reliance on cloud services. The blog post also introduces various tools and frameworks that support the deployment of object detection models on mobile and embedded systems, emphasizing the role of on-device computation in making real-time, cost-effective, and private decisions in the field.

AI is revolutionizing scientific research by drastically reducing the time required for data collection and analysis, which traditionally took decades, to just weeks. At the Field Museum of Natural History in Chicago, researchers digitized 13,500 insect specimens using vision AI, nearly matching the 19,000 images collected over the previous 20 years, showcasing AI's potential to transform data-driven discovery. By utilizing tools like Roboflow, scientists can automate data collection and analysis, creating workflows that enhance reproducibility in research across various fields, from biology to materials science. Vision AI systems, such as the DrawerDissect workflow, have demonstrated their capability to expedite research processes, automate measurements, and integrate metadata, thereby allowing scientists to test hypotheses more efficiently and revisit longstanding theories with unprecedented statistical power. The growing accessibility of these AI tools means that researchers, even without extensive machine learning expertise, can leverage AI to accelerate their work, transforming their approach to scientific inquiry and enabling macroevolutionary analyses at scales previously deemed unfeasible.