Vision-Language Model Benchmarks

Roboflow’s RF100-VL benchmark evaluates how well vision-language models perform object detection by identifying target classes and returning accurate bounding boxes across diverse domains such as aerial imagery, industrial scenes, documents, medical images, sports, and wildlife. In a webinar, Roboflow Research Engineer Matvei Popov compares GPT-6 Astra, Gemini 3.8 Flash, Qwen3.8-Max, and Claude Fable 5.1 using the RF20-VL-FSOD subset, testing zero-shot, one-shot, and 10-shot prompting approaches. Results are measured with mAP50-95 and the confidence-independent Rex F1@mIoU metric, reflecting both classification and localization quality. GPT-6 Astra leads the overall comparison, although Gemini 3.8 Flash performs best on the aerial-imagery category, illustrating that overall rankings may not predict performance in a particular domain. Few-shot examples generally help stronger models understand ambiguous classes and annotation conventions, but additional examples can sometimes reduce performance, particularly when class labels are already clear. The findings encourage developers to use public benchmarks as a starting point while testing models, prompts, and fresh representative images from their own environments before selecting a model for annotation or detection workflows.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.