Vision Language Models (VLMs) are emerging as powerful tools in artificial intelligence by integrating image and text inputs to generate meaningful outputs, applicable in fields like autonomous vehicles and medical imaging. This blog focuses on benchmarking various VLMs, including open-source models like Qwen2-VL-7B, against the previously top-ranked GPT-4o for an image classification task using the Caltech256 dataset. The results reveal that Qwen2-VL-7B is closing the performance gap with GPT-4o, achieving high accuracy while using less GPU memory, although GPT-4o still leads in overall metrics. The experiments highlight the potential of open-source models to rival closed-source counterparts and underline the importance of model selection based on task-specific requirements, such as the number of classes, which can significantly impact performance.