Evaluating computer use models with Microsoft

Microsoft has introduced Fara-7B, a compact vision language model (VLM) that sets a new standard for performance and efficiency within its class. Developed in collaboration with Browserbase, this model enables seamless training and evaluation of browser-based agents with reliable access to real websites, promoting consistent execution environments crucial for reinforcement learning and model evaluation. Fara-7B excels in speed and cost-efficiency, outperforming similar open-source models on the WebVoyager dataset, thanks to sub-second inference and reduced computational expenses. The evaluation process, facilitated by Browserbase's deterministic infrastructure, ensures fairness and reliability through human-verified assessments of model tasks. This initiative is part of a broader industry shift towards real-world web training, practical deployment optimization, and transparency in model evaluation, aiming to establish more consistent and trusted benchmarks. Fara-7B, available on platforms like HuggingFace and Azure AI Foundry, represents an advancement in small, open models, emphasizing the importance of genuine web interaction and human feedback in model development.