September 2025 Summaries

Bogdan Grechuk, an Associate Professor of Mathematics at the University of Leicester, explores the use of artificial intelligence in solving complex mathematical problems, specifically Diophantine equations. Despite AI's potential, Grechuk highlights challenges, such as models producing incorrect yet convincing solutions, which wastes time and requires verification through formal proof systems. He collaborates with Surge AI to push AI capabilities by creating problems that current models struggle to solve without guidance. Grechuk successfully guided GPT-5 through a complex problem by transforming it into an elliptic curve and finding solutions, illustrating that while AI can handle PhD-level problems with direct inputs, it still requires further development to independently solve research-level questions. He envisions a future where AI could autonomously contribute to mathematical research beyond executing existing methods.

The case study examines how three advanced coding models—Gemini 2.5 Pro, Claude Sonnet 4, and GPT-5—approach solving a specific software engineering problem from the SWE-bench, which tests coding agents by having them fix real GitHub issues using only shell commands. The study reveals how these models handle missing information and errors, with Gemini 2.5 Pro falling into a spiraling loop of hallucinations, Claude Sonnet 4 recovering after initial missteps, and GPT-5 successfully navigating the task without hallucinations. The key takeaway is the importance of recognizing missing information and verifying assumptions, as failure to do so can lead to continuous errors and flawed solutions. The study highlights the cognitive behaviors that distinguish robust reasoning from brittle performance, emphasizing the need for models to navigate uncertainty effectively. The findings underscore the challenges and opportunities in developing more reliable autonomous coding systems, which can learn to manage uncertainty and adapt to real-world complexities.

Benchmarks in artificial intelligence, often designed for academic purposes rather than practical applications, are criticized for failing to accurately measure AI capabilities in real-world scenarios. These metrics, such as IFEval, are frequently gamed and do not capture complex tasks like creativity or meaningful interaction, leading to misleading representations of AI progress. Frontier researchers prefer human evaluations as they offer a more nuanced assessment of AI performance, valuing creativity and wisdom over standardized metrics. The reliance on flawed benchmarks can result in a "death spiral," where AI models achieve high scores on artificial tests but fail to deliver in practical applications, eroding trust and stalling progress. The industry's future success depends on developing benchmarks that genuinely reflect AI's potential and align with ambitious real-world objectives.