July 2022 Summaries

Roboflow is committed to democratizing computer vision by leveraging open-source tools and fostering a collaborative community through its Roboflow Universe, which boasts over 90,000 datasets and 7,000 pre-trained models. The company maintains a significant library of open-source computer vision notebooks and actively supports projects like YOLOv5 and OpenCV, underscoring the importance of open-source collaboration in the tech industry. Roboflow's engineering team heavily incorporates open-source code in its operations, contributing to and hosting repositories on GitHub to enhance accessibility and integration of computer vision tools. They provide resources such as the roboflow-python package and inference server tools, enabling users to integrate Roboflow into applications across various platforms, including edge devices like NVIDIA Jetson and Raspberry Pi. The company also offers example projects and deployment tools to demonstrate practical applications of their technology, encouraging community engagement and contributions to open-source computer vision initiatives.

SegFormer, a new state-of-the-art semantic segmentation algorithm, is introduced as an advancement in transformer models, offering innovations that address previous limitations like low resolution and high computational cost. This blog post details a step-by-step guide on using Pytorch Lightning to train SegFormer on a custom dataset, specifically utilizing the public Balloons dataset while highlighting the process of creating a semantic segmentation dataset and defining a Pytorch Lightning model class. SegFormer distinguishes itself with a unique "positional-encoding-free and hierarchical Transformer encoder" and a "lightweight All-MLP decoder design," enabling it to operate at arbitrary resolutions and efficiently combine local and global attention for high-quality representations. The guide also covers evaluating the model on a test dataset, visualizing results, and the potential applications of semantic segmentation in fields like self-driving cars and robotics. The tutorial concludes with instructions on training, testing, and visualizing model outputs in TensorBoard, emphasizing the algorithm's efficiency and effectiveness in various semantic segmentation tasks.

Tiling is a technique used in computer vision to enhance the accuracy of detecting small objects, particularly in aerial images, by dividing an image into smaller, more manageable sections for both training and inference. This approach allows models to focus on smaller details, similar to zooming in, which can improve detection accuracy significantly. However, it is crucial to apply tiling consistently across both training and inference processes; otherwise, it may lead to false positives, as seen in models trained with tiling but inferred without it. While tiling boosts accuracy, it can negatively impact performance by increasing training times and slowing down inference speed due to the generation of numerous smaller images. Therefore, finding a balance between accuracy and performance is essential, and experimenting with different tile sizes can help achieve optimal results. Despite these challenges, tiling remains a valuable method for enhancing object detection in specific scenarios, though it is no longer supported as a parameter in the Roboflow Hosted Inference API, requiring alternatives like Slicing Aided Hyper Inference (SAHI) for implementation.

Polygon annotations have shown to provide significant benefits in training object detection models by offering more detailed information about object shapes and locations compared to traditional bounding boxes. This additional data helps models learn more effectively, leading to improved prediction accuracy, particularly during image augmentations like rotation or cropping where bounding boxes might not fit tightly and lose precision. Polygons not only retain a tight fit during transformations, mitigating the double-augmentation problem, but they also enable new augmentation types such as the copy/paste method, which enhances a model's ability to recognize objects in varied contexts. While newer models like YOLOv5 can directly utilize polygon annotations, others may require pre-augmentation with tools like Roboflow for optimal training. Furthermore, polygons can be converted to bounding boxes, facilitating experimentation with different model types and enhancing preprocessing steps by removing background pixels, thus enriching the capabilities of object detection and classification projects.

TensorFlow Lite (TFLite) is an open-source library developed by Google designed to deploy machine learning models on edge devices such as mobile phones, embedded systems, and on-device platforms. It supports use cases like natural language processing and computer vision, allowing for on-device inference tasks such as image classification and object detection. TFLite helps optimize models for smaller sizes and lower latency, converting TensorFlow models into a .tflite format using the TensorFlow Lite Converter for efficient edge deployment. For those interested in computer vision applications, there are guides available for training custom models with YOLOv4 Tiny and deploying them on platforms like Android, iOS, or Raspberry Pi. Additionally, Roboflow provides an iOS SDK for deploying computer vision models to edge devices.

YOLOv7, developed by WongKinYiu and AlexeyAB, represents a significant advancement in real-time object detection, offering state-of-the-art performance on the MS COCO benchmark using PyTorch and achieving impressive frame rates. While these results are exceptional, the limited utility of COCO's 80 classes in practical applications necessitates custom training for real-world relevance. A tutorial hosted on Google Colab by Roboflow guides users through training a YOLOv7 model on custom datasets, leveraging Roboflow Universe's extensive data repository. The process involves installing dependencies, loading a custom dataset, training the model, and evaluating its performance. Users can fine-tune their model settings and even deploy the model using Roboflow's deployment options or on edge devices. Additionally, for more complex tasks such as instance segmentation, further resources and tutorials are available.

Roboflow's changelog for May and June 2022 highlights significant updates and enhancements across its product offerings, including the launch of a new role for image labeling, support for oriented bounding boxes, and YOLOv6, as well as improvements in edge inference and model deployment. The release of Roboflow Universe V2, a comprehensive computer vision dataset and model resource, introduces features like a Model tab, enhanced search capabilities, and options for video and webcam inference. Roboflow also expanded its ecosystem by integrating with platforms such as Azure Marketplace and Salesforce, creating a new knowledge base, and hosting various events, including recruiting and team meetups. Additional achievements include the development of several user projects and the publication of 17 blog posts and 3 YouTube videos, as well as recognition from notable organizations and media outlets.

YOLOv6, a recent addition to the YOLO (You Only Look Once) family of models, was released by Meituan in June 2022, quickly gaining popularity with over 2,000 stars and 300 forks on GitHub. Designed for state-of-the-art performance on the COCO dataset benchmark, YOLOv6 introduces innovations such as an efficient decoupled head with SIoU loss and a hardware-friendly design for its Backbone and Neck. The model is available in three sizes—nano, tiny, and small—with plans for larger versions. An essential aspect of training YOLOv6 is preparing a high-quality dataset, and tools like Roboflow can assist in managing datasets, labeling data, and converting annotations to the YOLOv6 format. The training process involves setting up a development environment with dependencies, configuring custom training options, and evaluating model performance through metrics like mean average precision (mAP). YOLOv6 can also be converted to ONNX for better portability and integrated with active learning strategies to improve model performance.