To Think or Not to Think: A Router for Hybrid LLMs

Amir Mohseni's project explores the development of a router for hybrid Large Language Models (LLMs) that automatically determines whether a task requires reasoning or not, optimizing token usage during computation. The initiative was inspired by OpenAI's advancements in LLMs, particularly the introduction of test-time compute, allowing models to allocate more tokens for complex queries. Mohseni's router, which was built using synthetic data and tested on models like Qwen3-8B, aims to streamline the decision-making process in hybrid models like Qwen3 by automatically choosing between "think" and "no-think" modes based on the task's complexity. This approach notably improves performance over non-thinking baselines while using fewer tokens than full reasoning modes. Despite its promising results, the project has limitations, such as a lack of multilingual and multimodal data and evaluations largely constrained to specific model architectures. The research underscores the potential of automated reasoning mode selection, coinciding with OpenAI's release of GPT-5, which features a built-in router for similar purposes.