Home / Companies / Sourcegraph / Blog / Post Details
Content Deep Dive

Sourcegraph MCP server and a cheaper model beat a Mythos-class model alone

Blog post from Sourcegraph

Post Details
Company
Date Published
Author
Stephanie Jarmak
Word Count
619
Company Posts That Month
15
Language
English
Hacker News Points
-
Summary

On the CodeScaleBench tasks aimed at assessing agent performance in large codebases, Claude Sonnet 4.6 integrated with the Sourcegraph MCP server outperformed Fable 5 by succeeding in six out of nine tasks, while also being more cost-effective in terms of quality points. Claude Sonnet 4.6, which does not store source code on disk and instead uses search, symbol resolution, and reference following, scored 0.698, whereas Fable 5, which processes the entire repository locally, scored 0.568 and incurred nearly double the cost per quality point. The results particularly highlighted Sonnet 4.6's superiority in cross-repository discovery tasks, such as vulnerability tracing and cross-organization dependency following, though one task did favor Fable 5, indicating genuine comparative assessment. These findings suggest that a more affordable model with efficient code retrieval capabilities could outperform a more expensive, locally-operating model in certain cross-repository tasks, though further evaluation is needed for tasks requiring deep single-file analysis.

Trends Found in this Post
Trend Post Mentions Total Month Mentions Posts Companies MoM
MCP 4 6,026 689 188 -15%
Observability 1 3,430 674 183 +0%