| We've added Gemini 4 Argon and Muse Wonder to the sudo L7 leaderboard:
Gemini 4 Argon: 18.9% (#7)
Muse Wonder: 15.6% (… |
@HelloSurgeAI |
Company |
Original |
2026-10-14 |
3,375 |
25 |
2 |
9 |
1 |
6 |
| Real software engineering happens in repositories with years of accumulated history.
Legacy code nobody wants to touch… |
@HelloSurgeAI |
Company |
Original |
2026-10-13 |
908 |
17 |
1 |
5 |
0 |
3 |
| RT @echen: another Surge benchmark on the Muse Wonder model card 👀
Muse scores 33.2% on GDP.pdf. #3 on our leaderboard… |
@HelloSurgeAI |
Company |
Repost |
2026-10-12 |
61 |
0 |
1 |
0 |
0 |
0 |
| 🇫🇷 Le Chonk joins the sudo L7 leaderboard 🇫🇷
We just added @MistralAI's Mistral Large 4 to sudo L7, our benchmark for … |
@HelloSurgeAI |
Company |
Original |
2026-10-12 |
19,349 |
266 |
37 |
4 |
2 |
33 |
| RT @lennysan: Sweet new benchmark from Surge looking at how close AI is getting to a staff-level software engineering. … |
@HelloSurgeAI |
Company |
Repost |
2026-10-09 |
69 |
0 |
7 |
0 |
0 |
0 |
| An AI agent found every expiring rent discount in a commercial real estate workbook.
Then it missed three break clau… |
@HelloSurgeAI |
Company |
Original |
2026-10-09 |
1,009 |
20 |
1 |
13 |
0 |
3 |
| Introducing sudo L7: Can Coding Agents Act Like Staff Engineers?
Coding agents are getting very good at solving well-d… |
@HelloSurgeAI |
Company |
Original |
2026-10-08 |
3,592 |
74 |
9 |
18 |
6 |
27 |
| RT @echen: introducing sudo L7, our new benchmark for staff-level software engineering.
coding agents are getting very… |
@HelloSurgeAI |
Company |
Repost |
2026-10-08 |
50 |
0 |
7 |
0 |
0 |
0 |
| Only 2 of 66 runs caught a $25.7M pricing error.
One screenshot. One unit mistake. A 100× overstatement.
See what els… |
@HelloSurgeAI |
Company |
Original |
2026-10-06 |
639 |
4 |
2 |
1 |
0 |
2 |
| For their Mistral Large 4 release, @MistralAI ran a blind Surge human eval, with our expert software engineers evaluati… |
@HelloSurgeAI |
Company |
Quote |
2026-10-06 |
19,425 |
340 |
32 |
18 |
5 |
35 |
| RT @echen: 🥖 LGTM, le Chonk 🥖
in their latest Mistral Large 4 release, @MistralAI ran a blind Surge human eval across … |
@HelloSurgeAI |
Company |
Repost |
2026-10-06 |
42 |
0 |
8 |
0 |
0 |
0 |
| RT @ritvikpandey21: A big focus for our team at @Pulse__AI is making the data AI agents receive more useful.
Huge con… |
@HelloSurgeAI |
Company |
Repost |
2026-10-02 |
68 |
0 |
7 |
0 |
0 |
0 |
| RT @echen: GDP.pdf just got a little brother: introducing GDP.xlsx
the world runs on spreadsheets. somewhere in Sheet2… |
@HelloSurgeAI |
Company |
Repost |
2026-09-30 |
49 |
0 |
3 |
0 |
0 |
0 |
| GDP.pdf on GPT-6.1 Sol:
Astra-level performance at one-fifth the cost.
Nice job! https://t.co/Y3uwUZGUXR |
@HelloSurgeAI |
Company |
Original |
2026-09-29 |
574 |
6 |
0 |
0 |
0 |
0 |
| RT @echen: other models: "the tenant's liability is unlimited"
gpt-6.1: "not according to page 14"
great job, sol http… |
@HelloSurgeAI |
Company |
Repost |
2026-09-29 |
53 |
0 |
1 |
0 |
0 |
0 |
| A benchmark shouldn’t just be hard to climb.
Climbing it should make the model stronger in ways that transfer.
So wh… |
@HelloSurgeAI |
Company |
Original |
2026-09-29 |
2,040 |
13 |
1 |
2 |
0 |
7 |
| RT @echen: New research:
What Does GDP.pdf Teach, Beyond PDFs?
GDP.pdf has become a widely used frontier eval, so we … |
@HelloSurgeAI |
Company |
Repost |
2026-09-29 |
46 |
0 |
2 |
0 |
0 |
0 |
| RT @echen: normal parents: proud when their kid gets a B+ on their report card
me: proud when Sonnet 5.5 gets 61.6% on… |
@HelloSurgeAI |
Company |
Repost |
2026-09-28 |
62 |
0 |
3 |
0 |
0 |
0 |
| Last week we launched DAYJOB, our benchmark for evaluating if agents can survive a 9-to-5.
DAYJOB: Healthcare and DAYJ… |
@HelloSurgeAI |
Company |
Original |
2026-09-28 |
2,116 |
15 |
2 |
1 |
0 |
7 |
| What does filling out a spreadsheet have to do with fixing a PowerShell parser?
More than we expected.
We post-traine… |
@HelloSurgeAI |
Company |
Original |
2026-09-25 |
992 |
10 |
2 |
2 |
0 |
5 |
| We trained Qwen3-4B on 1,000 ComplexConstraints examples.
Rubric pass rate:
57.9% → 73.4%
The trained 4B model landed… |
@HelloSurgeAI |
Company |
Original |
2026-09-24 |
2,470 |
30 |
4 |
0 |
0 |
19 |
| Introducing DAYJOB: our latest benchmark suite for economically valuable agents.
Models can already do impressive work… |
@HelloSurgeAI |
Company |
Original |
2026-09-23 |
2,579 |
41 |
2 |
4 |
0 |
25 |
| RT @echen: can agents survive a 9 to 5? introducing DAYJOB, our new benchmark for economically valuable agents.
GDPval… |
@HelloSurgeAI |
Company |
Repost |
2026-09-23 |
51 |
0 |
9 |
0 |
0 |
0 |
| RT @echen: one of my favorite examples from our Kimi coding post-training run:
it had to write a Zstandard decompresso… |
@HelloSurgeAI |
Company |
Repost |
2026-09-16 |
61 |
0 |
3 |
0 |
0 |
0 |
| Anthropic recently published new work on automated alignment researchers: Claude agents that search the literature, pro… |
@HelloSurgeAI |
Company |
Original |
2026-09-15 |
2,088 |
16 |
3 |
1 |
1 |
14 |
| RT @echen: i agree with dario.
i used to work on feed algorithms at google, fb, and twitter. we optimized for engageme… |
@HelloSurgeAI |
Company |
Repost |
2026-09-14 |
48 |
0 |
16 |
0 |
0 |
0 |
| 1,700 agentic coding tasks improved Kimi K2.7 on all five coding benchmarks we tested.
We post-trained Kimi K2.7 using… |
@HelloSurgeAI |
Company |
Original |
2026-09-11 |
3,727 |
38 |
3 |
0 |
0 |
34 |
| RT @echen: An intern knows how to write code. A staff engineer knows how to make it *shippable*.
We post-trained kimi … |
@HelloSurgeAI |
Company |
Repost |
2026-09-11 |
59 |
0 |
3 |
0 |
0 |
0 |
| Four major frontier models shipped last week: Claude Fable 5.1, Muse Spark 1.3, Gemini 3.8 Flash, and GPT-6 Astra.
We’… |
@HelloSurgeAI |
Company |
Original |
2026-09-10 |
4,985 |
42 |
1 |
5 |
1 |
17 |
| RT @echen: @ArtificialAnlys just added GDP.pdf to their Intelligence Index.
Which means their definition of intelligen… |
@HelloSurgeAI |
Company |
Repost |
2026-09-05 |
61 |
0 |
2 |
0 |
0 |
0 |
| RT @echen: leaked audio from an evals offsite https://t.co/06yeneBG0f |
@HelloSurgeAI |
Company |
Repost |
2026-09-04 |
63 |
0 |
6 |
0 |
0 |
0 |
| @DeepSeek_AI V4 Pro scores 59.7 on the Tuesday Work Index, our composite measure of frontier AI at work.
That is up 10… |
@HelloSurgeAI |
Company |
Original |
2026-08-27 |
827 |
5 |
2 |
1 |
0 |
0 |
| What should AI optimize for?
Not just efficiency. Not just engagement. Not just doing more work for us.
@echen joined… |
@HelloSurgeAI |
Company |
Original |
2026-08-26 |
2,654 |
12 |
3 |
2 |
0 |
11 |
| Taste isn't something you get from facts alone.
All expert work — whether it's designing a website, pushing the bounda… |
@HelloSurgeAI |
Company |
Original |
2026-08-25 |
3,476 |
22 |
2 |
3 |
0 |
9 |
| RT @echen: taste isn't a reasoning problem. you can't rubric your way from "correct" to "great."
great writing, design… |
@HelloSurgeAI |
Company |
Repost |
2026-08-25 |
75 |
0 |
2 |
0 |
0 |
0 |
| No, a mystery model didn't just blow past Fable on agentic coding.
We ran Ox Alpha on our internal, unreleased agentic… |
@HelloSurgeAI |
Company |
Quote |
2026-08-23 |
11,433 |
94 |
2 |
12 |
2 |
18 |
| SURGE MODEL REPORT: QWEN 3.8 MAX
Qwen 3.8 Max scores 58.7 on our Tuesday Work Index, our composite measure of frontier… |
@HelloSurgeAI |
Company |
Original |
2026-08-20 |
1,499 |
11 |
2 |
2 |
0 |
4 |
| Introducing Tuesday: A Frontier Index for AI at Work
Frontier models can solve extraordinary problems. But can they ge… |
@HelloSurgeAI |
Company |
Original |
2026-08-19 |
20,530 |
82 |
8 |
10 |
2 |
67 |
| RT @echen: it's tuesday morning. you have a stack of work in front of you.
read your email. understand the chart from … |
@HelloSurgeAI |
Company |
Repost |
2026-08-18 |
56 |
0 |
3 |
0 |
0 |
0 |
| The easy coding problems are solved. The hard ones are the whole job now.
We're hiring senior engineers to find where … |
@HelloSurgeAI |
Company |
Original |
2026-08-06 |
2,448 |
9 |
0 |
3 |
0 |
6 |
| We post-trained a model on office work. It also improved at coding. New paper from our research team!
The training run… |
@HelloSurgeAI |
Company |
Original |
2026-08-05 |
7,647 |
97 |
3 |
3 |
1 |
54 |
| Most engineers will use AI. A few will get to shape it.
We're hiring senior engineers to build the coding benchmarks f… |
@HelloSurgeAI |
Company |
Original |
2026-08-03 |
2,955 |
19 |
0 |
3 |
0 |
14 |
| Frontier Data, Off the Shelf.
The training data, evals, and RL environments we build for frontier labs, now available … |
@HelloSurgeAI |
Company |
Original |
2026-07-31 |
13,260 |
68 |
6 |
1 |
1 |
72 |
| Our Chartography benchmark measures whether models can read professional charts.
@AnthropicAI’s contribution to scienc… |
@HelloSurgeAI |
Company |
Quote |
2026-07-27 |
2,380 |
14 |
1 |
1 |
0 |
0 |
| Yesterday, Fable found a counterexample to the Jacobian conjecture, an 87-year-old problem.
Was that a one-off, or can… |
@HelloSurgeAI |
Company |
Quote |
2026-07-20 |
13,574 |
50 |
2 |
5 |
1 |
21 |
| Today we rolled out Chartography: a benchmark for the charts professionals actually work from.
Kaplan-Meier curves, co… |
@HelloSurgeAI |
Company |
Original |
2026-07-16 |
1,605 |
16 |
2 |
2 |
1 |
8 |
| @OpenAI's GPT-5.6 launched yesterday, and its model card cites GDP.pdf, our benchmark built from real-world prompts and… |
@HelloSurgeAI |
Company |
Original |
2026-07-10 |
1,312 |
21 |
2 |
1 |
0 |
2 |
| Deeper Instructions, Stronger Generalization: Training on ComplexConstraints
Given the chance, a model will reward hac… |
@HelloSurgeAI |
Company |
Original |
2026-07-01 |
1,145 |
15 |
1 |
1 |
0 |
7 |
| Last week, we released HANDBOOK.md: a benchmark for long-context agentic instruction following.
HANDBOOK drops an agen… |
@HelloSurgeAI |
Company |
Original |
2026-06-29 |
1,035 |
13 |
0 |
2 |
1 |
9 |
| Introducing ComplexConstraints — a new IF benchmark designed to test whether models can handle the kinds of constraints… |
@HelloSurgeAI |
Company |
Original |
2026-06-03 |
1,230 |
22 |
1 |
1 |
0 |
4 |
| Congrats to the Microsoft AI team on MAI-Thinking-1!
There's an art and science to post-training - navigating tradeoff… |
@HelloSurgeAI |
Company |
Original |
2026-06-03 |
1,461 |
20 |
0 |
0 |
0 |
4 |
| We trained Qwen3.5 to match GPT-5.5 in tool use.
The important question around RL environments is: do they teach gener… |
@HelloSurgeAI |
Company |
Original |
2026-06-02 |
1,945 |
16 |
0 |
1 |
0 |
17 |
| length looks like authority. bullets look like rigor. flattery looks like understanding. emojis look like care.
none o… |
@HelloSurgeAI |
Company |
Original |
2026-05-21 |
1,208 |
23 |
2 |
0 |
1 |
4 |
| GDP.pdf was accepted to the CVPR 2026 Workshop on Multimodal Reasoning
can frontier models handle the three-letter doc… |
@HelloSurgeAI |
Company |
Original |
2026-04-29 |
1,406 |
11 |
0 |
2 |
0 |
6 |
| We took a very different path to the frontier. Zero venture capital. Zero growth hacks.
To our entire team and the epi… |
@HelloSurgeAI |
Company |
Quote |
2026-04-17 |
6,585 |
47 |
1 |
0 |
0 |
15 |
| 📄 Introducing GDP.pdf: an expert multimodal reasoning benchmark for the documents that run the world. 📄
We've spent ye… |
@HelloSurgeAI |
Company |
Original |
2026-04-14 |
1,335 |
21 |
1 |
1 |
0 |
9 |
| Big news: our CEO @echen has been named #73 on @Forbes' list of the 250 Greatest Living Self-Made Americans.
That's ab… |
@HelloSurgeAI |
Company |
Original |
2026-04-09 |
8,399 |
11 |
0 |
0 |
1 |
2 |
| Riemann-bench was just accepted at an ICLR 2026 workshop!
We built Riemann-bench to test moonshot mathematics. We work… |
@HelloSurgeAI |
Company |
Original |
2026-04-03 |
6,505 |
60 |
11 |
2 |
1 |
29 |
| When we built GSM8K with OpenAI five years ago, it represented the absolute frontier of what was possible. Today, the i… |
@HelloSurgeAI |
Company |
Original |
2026-03-24 |
45,514 |
276 |
47 |
12 |
6 |
116 |
| Let’s look at how frontier agents (even Opus 4.6, GPT-5.2, and Gemini 3.1 Pro!) struggle at solving tasks in Enterprise… |
@HelloSurgeAI |
Company |
Original |
2026-02-26 |
3,107 |
23 |
4 |
1 |
1 |
9 |
| Everyone’s building $100M "agentic" models, so we built a simulated company to see if they could actually hold down a j… |
@HelloSurgeAI |
Company |
Original |
2026-02-20 |
4,550 |
34 |
2 |
7 |
0 |
30 |
| We’ve finally done it. Forbes just ranked our CEO *54* spots above Taylor Swift on their America’s Greatest Innovators … |
@HelloSurgeAI |
Company |
Original |
2026-02-11 |
1,522 |
25 |
0 |
1 |
0 |
0 |
| We put Opus 4.6 through our Hemingway-bench Writing Leaderboard. How did it fare?
Claude continues to dominate GPT-5.2… |
@HelloSurgeAI |
Company |
Original |
2026-02-10 |
2,128 |
13 |
2 |
1 |
0 |
3 |
| "Prognosticative pastry." "A hound circling a tree, nose to bark."
Believe it or not, those quotes aren't jokes. They'… |
@HelloSurgeAI |
Company |
Original |
2026-02-04 |
846 |
18 |
0 |
1 |
1 |
5 |
| Our CEO @edwinchenai just revealed what's really happening inside the frontier labs on Unsupervised Learning with @jaco… |
@HelloSurgeAI |
Company |
Original |
2025-12-17 |
1,719 |
14 |
3 |
1 |
0 |
3 |
| We made the Inc 2025 Best in Business list.
Even though the Silicon Valley playbook says we shouldn’t exist.
We had:… |
@HelloSurgeAI |
Company |
Original |
2025-12-12 |
1,429 |
22 |
4 |
2 |
0 |
0 |
| Imagine measuring a personal assistant’s ability by asking them to write emails without using the letter "c" or any com… |
@HelloSurgeAI |
Company |
Original |
2025-12-10 |
1,165 |
6 |
4 |
3 |
0 |
3 |
| A new episode of Lenny’s Podcast with our CEO @echen just dropped.
What they covered:
• Why Anthropic and Google are … |
@HelloSurgeAI |
Company |
Original |
2025-12-08 |
4,751 |
38 |
8 |
2 |
1 |
16 |
| GPT-5.1: now with 20% more warmth and personality 😅
When GPT-5 launched in Aug, users were furious that they lost 4o.
… |
@HelloSurgeAI |
Company |
Quote |
2025-11-12 |
11,002 |
23 |
1 |
2 |
0 |
9 |
| Everyone's acting like models are ready to replace humans in work settings.
We put that to the test by creating an ent… |
@HelloSurgeAI |
Company |
Original |
2025-11-11 |
247,168 |
539 |
72 |
26 |
26 |
589 |
| Is AI more likely to make you a billion dollars … or lose it? |
@HelloSurgeAI |
Company |
Quote |
2025-11-06 |
2,940 |
19 |
1 |
1 |
0 |
4 |
| Teaching LLMs to follow instructions? Step 1.
Teaching them to have taste? That's the endgame.
An 8-line poem about th… |
@HelloSurgeAI |
Company |
Original |
2025-10-21 |
1,879 |
9 |
1 |
1 |
0 |
4 |
