The best biology AI won't do biology

A public BioMysteryBench-style evaluation found that Anthropic’s highest-scoring biology model is restricted to vetted partners, while its publicly accessible counterpart refuses chemistry and biology API requests. In tests of three bioinformatics tasks involving organism identification, bacterial classification, and transcription-factor motif discovery, Claude Opus 4.8 solved all three but cost far more than cheaper alternatives, while Google’s Gemma 4-31B and GLM-5.2 each solved two tasks, with Gemma completing its successes for roughly four cents. The results suggest lower-cost models can handle short, tool-based biological lookups but often fail on longer multistep workflows because of time or turn limits, whereas frontier models offer greater persistence. A multi-model synthesis approach also failed on two harder tasks when all participating models shared the same incorrect assumptions, indicating that model panels help only when at least one model reaches the correct conclusion. The author cautions that the sample is small but makes the testing harness and methodology open source, arguing that model selection should depend on task complexity and cost rather than headline benchmark performance alone.

No tracked trend matches for this post yet.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.