| Do agent teams pay off?
We ran GPT-6 Sol and Claude Opus 5.5 on Vibe Code Bench, alone and as teams, at medium and ma… |
@ValsAI |
Company |
Original |
2026-10-09 |
13,523 |
185 |
8 |
14 |
5 |
54 |
| RT @nikilravi: It was great to chat with lots of policy folks at Vals' DC summit, and to see the level of interest in e… |
@ValsAI |
Company |
Repost |
2026-10-09 |
274 |
0 |
1 |
0 |
0 |
0 |
| What happens when a teenager pushes back on an AI’s safety guardrails?
1 in 5 teens now talk to AI chatbots about ment… |
@ValsAI |
Company |
Original |
2026-10-08 |
2,965 |
39 |
3 |
2 |
3 |
4 |
| We tested Mercury Decide, a new model from @_inception_ai, on the same benchmark we used for TypeSafe's Jev.
It mat… |
@ValsAI |
Company |
Original |
2026-10-08 |
5,907 |
69 |
11 |
5 |
3 |
19 |
| RT @RayanKrishnan: The diffusion of personal agents (Grok bot, Muse, Dots, Instinct, etc) may cause a rapid increase in… |
@ValsAI |
Company |
Repost |
2026-10-08 |
245 |
0 |
5 |
0 |
0 |
0 |
| A $6 model is now the third best on Vibe Code Bench. Claude Haiku 5.5 scores 90.4%, up 79% and 89 spots from its predec… |
@ValsAI |
Company |
Original |
2026-10-08 |
6,390 |
158 |
8 |
16 |
4 |
18 |
| Xiaomi open-sourced the RL environments behind MiMo v2.6. In two-thirds of the coding tasks, the answer is still in the… |
@ValsAI |
Company |
Original |
2026-10-07 |
122,804 |
1,171 |
74 |
45 |
68 |
504 |
| Meet Omar, a founding member of the technical staff at Vals.
Originally from Almaty, Kazakstan, Omar moved to the US a… |
@ValsAI |
Company |
Original |
2026-10-06 |
4,479 |
25 |
0 |
2 |
1 |
1 |
| Can a model that doesn’t generate text compete with frontier LLMs?
We independently tested TypeSafe’s Jev against 11 o… |
@ValsAI |
Company |
Original |
2026-10-06 |
24,436 |
252 |
18 |
8 |
5 |
81 |
| Mistral Large 4 is now the #1 open-weight model on HLAB and #9 among open-weight models on the Vals Index. https://t.co… |
@ValsAI |
Company |
Original |
2026-10-06 |
31,867 |
414 |
36 |
16 |
10 |
50 |
| How we think about AI long term depends on what we value. Rayan on what he's hearing in DC. |
@ValsAI |
Company |
Quote |
2026-10-06 |
7,408 |
10 |
1 |
1 |
0 |
2 |
| For decades, researchers have sought materials that sort electrons by spin while their magnetism cancels.
In 3 days, 9… |
@ValsAI |
Company |
Original |
2026-10-05 |
1,295,743 |
7,808 |
555 |
180 |
363 |
2,947 |
| Last month the Vals AI team hit the road for campus recruiting. We spent time at @UCBerkeley, @MIT, @GeorgiaTech, and @… |
@ValsAI |
Company |
Original |
2026-10-05 |
4,996 |
39 |
4 |
4 |
0 |
7 |
| RT @langstonnashold: Come join us! |
@ValsAI |
Company |
Repost |
2026-10-04 |
259 |
0 |
1 |
0 |
0 |
0 |
| RT @nikilravi: If you're in SF for COLM and are interested in evals, come join us this Thursday! |
@ValsAI |
Company |
Repost |
2026-10-04 |
246 |
0 |
2 |
0 |
0 |
0 |
| Attending COLM? Join us for an intimate dinner with members of the Vals team to discuss all things evals.
Registratio… |
@ValsAI |
Company |
Original |
2026-10-04 |
11,636 |
32 |
5 |
2 |
2 |
15 |
| RT @nikilravi: Web search evals typically don't do a good job disentangling the model's capability from the capability … |
@ValsAI |
Company |
Repost |
2026-10-03 |
266 |
0 |
2 |
0 |
0 |
0 |
| RT @KeenableAI: Love seeing more work on better search evals for agents. @ValsAI uses private, expert-written tasks.
W… |
@ValsAI |
Company |
Repost |
2026-10-03 |
240 |
0 |
4 |
0 |
0 |
0 |
| Rayan on MTS this morning! |
@ValsAI |
Company |
Quote |
2026-10-02 |
3,103 |
13 |
0 |
2 |
0 |
5 |
| StepFun debuts on the Vals leaderboard with Step 5 Preview, ranking #7 among open-weight models, just ahead of Qwen 3.8… |
@ValsAI |
Company |
Original |
2026-10-02 |
8,770 |
70 |
4 |
4 |
4 |
10 |
| RT @MTSlive: Vals AI CEO @RayanKrishnan argues "Claude is the salesman for Claude," because the model can eventually fi… |
@ValsAI |
Company |
Repost |
2026-10-02 |
219 |
0 |
3 |
0 |
0 |
0 |
| We realized that web search wasn’t being measured correctly and we weren’t alone. Other companies have created their ow… |
@ValsAI |
Company |
Original |
2026-10-02 |
11,308 |
86 |
12 |
8 |
6 |
22 |
| RT @RayanKrishnan: https://t.co/5oPSEvEnNl |
@ValsAI |
Company |
Repost |
2026-10-02 |
234 |
0 |
7 |
0 |
0 |
0 |
| Why do public benchmarks saturate so fast?
@willcb tells us the "open secret" on The Bench. https://t.co/dWVtyvhEQM |
@ValsAI |
Company |
Original |
2026-10-02 |
4,422 |
35 |
2 |
5 |
0 |
16 |
| We are live-streaming Gemini 4 Argon playing Kerbal Space Program.
So far, Gemini 4 sits right between Claude Fable 5… |
@ValsAI |
Company |
Original |
2026-10-01 |
22,088 |
288 |
10 |
12 |
1 |
38 |
| RT @RayanKrishnan: Google featured many of our benchmark results in their Gemini 4 Argon lunch. Here are my personal th… |
@ValsAI |
Company |
Repost |
2026-10-01 |
260 |
0 |
14 |
0 |
0 |
0 |
| How well do AI Detectors work? And are general-purpose language models catching up?
Our new AI Detection Evaluation us… |
@ValsAI |
Company |
Original |
2026-10-01 |
38,705 |
214 |
8 |
15 |
5 |
97 |
| Join us for #SFTechWeek to dig into AI evaluation for mental health and AI child safety.
Whether you build, test, or … |
@ValsAI |
Company |
Original |
2026-10-01 |
1,850 |
9 |
1 |
3 |
0 |
1 |
| Big jump on our RSI Index. |
@ValsAI |
Company |
Quote |
2026-09-30 |
8,769 |
125 |
2 |
3 |
0 |
10 |
| Gemini is on top of the Vals Index for the first time.
Gemini 4 Argon takes the #1 spot on the Vals Index at 68.9%. h… |
@ValsAI |
Company |
Original |
2026-09-30 |
231,078 |
1,207 |
99 |
46 |
47 |
122 |
| Why off-the-shelf AI models aren't enough for the enterprise. Prime Intellect's @willcb and Rayan discuss owning your i… |
@ValsAI |
Company |
Original |
2026-09-30 |
9,783 |
41 |
5 |
4 |
2 |
19 |
| GPT 6.1 Sol is here, and it’s an overall improvement over GPT-6 Sol. It now ranks #7, three spots above GPT-6 Sol. http… |
@ValsAI |
Company |
Original |
2026-09-30 |
13,355 |
177 |
16 |
4 |
1 |
25 |
| Fireworks' Ember-1 is now on Vals. We tested it on our finance and legal benchmarks, and it performs close to Kimi K3 w… |
@ValsAI |
Company |
Original |
2026-09-30 |
7,309 |
40 |
3 |
2 |
2 |
5 |
| We’ve released v2.1 of the Vals Index!
The new iteration replaces Terminal Bench 2.1 with Terminal Bench 4, and adds o… |
@ValsAI |
Company |
Original |
2026-09-29 |
6,030 |
130 |
3 |
4 |
0 |
7 |
| Rayan will be speaking @aiDotEngineer in October! |
@ValsAI |
Company |
Quote |
2026-09-29 |
2,067 |
13 |
1 |
1 |
0 |
1 |
| How has Claude Sonnet 5.5's writing changed from Sonnet 5?
Using our eval traces, we compared both models on the same… |
@ValsAI |
Company |
Original |
2026-09-29 |
10,722 |
132 |
2 |
6 |
1 |
40 |
| We asked ten Claude Sonnet 5.5 agents to use Lean to prove the lowest-energy arrangement of seven electrons on a sphere… |
@ValsAI |
Company |
Original |
2026-09-29 |
23,599 |
541 |
40 |
16 |
12 |
120 |
| When does more reasoning stop being useful and start becoming overhead?
We ran Open AI and Anthropic’s best models: GP… |
@ValsAI |
Company |
Original |
2026-09-28 |
7,795 |
156 |
10 |
7 |
3 |
32 |
| Anthropic's "mid-tier" model is now the second best on the Vals Index.
Sonnet 5.5 debuts at #2 at just 0.47 points beh… |
@ValsAI |
Company |
Original |
2026-09-28 |
13,049 |
343 |
16 |
15 |
6 |
29 |
| It was a big week for model releases: Claude Opus 5.5, GPT-6 Sol, GPT-6 Luna, Grok 4.7, MiMo V2.6 Pro / Flash all dropp… |
@ValsAI |
Company |
Original |
2026-09-26 |
4,245 |
111 |
8 |
10 |
0 |
19 |
| How does "thinking harder" change how models perform on math proofs?
We ran GPT-6 Astra, Opus 5, and Opus 5.5 on every… |
@ValsAI |
Company |
Original |
2026-09-25 |
14,872 |
231 |
19 |
24 |
4 |
60 |
| RT @langstonnashold: Found a super interesting instance of attempted reward hacking in Terminal Bench Science from Meta… |
@ValsAI |
Company |
Repost |
2026-09-24 |
248 |
0 |
15 |
0 |
0 |
0 |
| GPT-6 Luna is 100x cheaper than Astra per token, while landing within 8 points of it on the Vals https://t.co/kPAGXIsNf… |
@ValsAI |
Company |
Original |
2026-09-23 |
6,399 |
147 |
4 |
8 |
0 |
19 |
| GPT-6 Astra landed on every planet and moon in KSP with a surface (14 worlds) and flew a kerbal home from 13 of them. O… |
@ValsAI |
Company |
Original |
2026-09-23 |
12,851 |
339 |
11 |
10 |
4 |
52 |
| The Washington Post just released an article about our work on AI Child Safety. They key takeaway: child safety can’t b… |
@ValsAI |
Company |
Original |
2026-09-23 |
2,325 |
17 |
5 |
3 |
1 |
3 |
| The much anticipated GPT-6 Sol is here and ranks #8 on the Vals Index, approximately on par with GPT 5.6 Sol at 1/2 of … |
@ValsAI |
Company |
Original |
2026-09-23 |
31,127 |
290 |
16 |
32 |
5 |
38 |
| Opus 5.5 is #1 on the Vals Index, up 2 spots and 2 pts from Opus 5. Anthropic regains the top three spots on the Index,… |
@ValsAI |
Company |
Quote |
2026-09-23 |
6,958 |
126 |
8 |
4 |
0 |
15 |
| MiMo V2.6 Pro and Flash are the new #1 and #2 open-weight models on the Vals Index, respectively. https://t.co/1SKL9kw0… |
@ValsAI |
Company |
Original |
2026-09-23 |
15,702 |
404 |
24 |
10 |
4 |
39 |
| Ant Group’s Ling 3.0 Flash Fin is a finance-specialized open-weight model that delivers strong financial analysis at bu… |
@ValsAI |
Company |
Original |
2026-09-22 |
5,831 |
45 |
4 |
4 |
1 |
11 |
| We asked ten Claude Opus 5.5 agents to devise a faster shortest-path algorithm and prove it in Lean. Within 15 hours, t… |
@ValsAI |
Company |
Original |
2026-09-22 |
1,338,426 |
3,430 |
244 |
73 |
110 |
1,213 |
| Claude Opus 5.5 takes #1 on RSI Index and is the first model to beat the published reference on LM Training under our p… |
@ValsAI |
Company |
Original |
2026-09-22 |
117,127 |
737 |
34 |
17 |
7 |
105 |
| RT @nikilravi: If you're interested in making coding benchmarks for your internal repos and workflows, check out https:… |
@ValsAI |
Company |
Repost |
2026-09-22 |
238 |
0 |
1 |
0 |
0 |
0 |
| Post-launch, xAI has updated its SDK, significantly improving its performance. It is now #10 on the Vals Index. https:/… |
@ValsAI |
Company |
Original |
2026-09-22 |
47,951 |
223 |
4 |
12 |
9 |
21 |
| We evaluated Grok 4.7 across the Vals benchmark suite. It ranks #24 on the Vals Index at 54.2%, down 5.0 points from Gr… |
@ValsAI |
Company |
Original |
2026-09-21 |
291,480 |
539 |
25 |
38 |
38 |
96 |
| AI models are advancing faster than legacy benchmarks can keep up.
@TechCrunch @LucasRopek1 visited us to see how we’… |
@ValsAI |
Company |
Quote |
2026-09-19 |
4,418 |
36 |
1 |
5 |
0 |
5 |
| Grok Voice Transcribe 2.0 is now #2 on Voice Code Bench—up 13pp and 12 spots from its predecessor, Grok Voice Transcri… |
@ValsAI |
Company |
Original |
2026-09-19 |
2,337 |
19 |
1 |
2 |
0 |
4 |
| What tasks are left that humans find easy but today's models still find hard?
Two such tasks are computer use and game… |
@ValsAI |
Company |
Original |
2026-09-18 |
25,032 |
280 |
18 |
21 |
14 |
85 |
| Tencent's Hy4 Preview just landed #4 among open-weight models on the Vals Index.
It is also the cheapest model in the … |
@ValsAI |
Company |
Original |
2026-09-18 |
5,496 |
54 |
4 |
8 |
1 |
11 |
| GPT-6 Astra has beaten Factorio: Space Age after over 165 hours in-game time and 2 days wall-clock time.
Space Age has… |
@ValsAI |
Company |
Original |
2026-09-17 |
510,392 |
2,881 |
137 |
78 |
64 |
483 |
| OpenAI launched Astra for Law today, and used the Vals Legal Research Bench to validate it.
On OpenAI’s self-reported … |
@ValsAI |
Company |
Original |
2026-09-17 |
9,690 |
97 |
8 |
4 |
1 |
20 |
| Existing coding benchmarks stop at the first working version. We are releasing Vibe Code Bench 1-100 today, to measure … |
@ValsAI |
Company |
Original |
2026-09-17 |
8,825 |
106 |
8 |
10 |
4 |
22 |
| Terminal-Bench 4.0 is now live on Vals.
It is a benchmark of 66 new tasks that ask an agent to do real terminal work … |
@ValsAI |
Company |
Original |
2026-09-16 |
11,697 |
146 |
4 |
13 |
1 |
26 |
| RT @BloombergTV: Vals AI co-founder and CEO Rayan Krishnan says investment in testing and evaluating AI hasn’t kept pac… |
@ValsAI |
Company |
Repost |
2026-09-16 |
205 |
0 |
6 |
0 |
0 |
0 |
| Scientific discovery is the next frontier for AI systems. However, new scientific results are difficult to verify, and … |
@ValsAI |
Company |
Original |
2026-09-16 |
11,692 |
141 |
13 |
6 |
3 |
41 |
| GPT-6 Astra had gotten further than any AI system had ever gone in Minecraft.
It was able to set up a semi-automatic b… |
@ValsAI |
Company |
Original |
2026-09-15 |
799,823 |
7,426 |
440 |
109 |
161 |
1,715 |
| AI cheating is on the rise…
On Terminal-Bench-2.1, models are given tools that could give them the solution directly, … |
@ValsAI |
Company |
Original |
2026-09-15 |
91,790 |
247 |
14 |
20 |
16 |
76 |
| We ran Mercury 2.5 across our benchmark suite and although performance is lower, it is extremely fast. https://t.co/8OA… |
@ValsAI |
Company |
Original |
2026-09-14 |
4,740 |
26 |
0 |
2 |
0 |
3 |
| Watch the full interview now! |
@ValsAI |
Company |
Quote |
2026-09-14 |
3,315 |
13 |
1 |
1 |
0 |
2 |
| We ran Devin Fusion on our Code Migration benchmark and the performance of Devin Fusion with GPT-6 Astra and SWE-2 side… |
@ValsAI |
Company |
Quote |
2026-09-14 |
14,602 |
122 |
7 |
9 |
2 |
35 |
| RT @bcherny: Fable solved the Cyphral Distich (a 370 year old cypher). Super cool way to use Claude
https://t.co/0fgTd… |
@ValsAI |
Company |
Repost |
2026-09-14 |
158 |
0 |
156 |
0 |
0 |
0 |
| RT @RayanKrishnan: With AI safety topics going mainstream, the public seems anxious and disconnected from the reality o… |
@ValsAI |
Company |
Repost |
2026-09-14 |
227 |
0 |
17 |
0 |
0 |
0 |
| It's encouraging to see industry leaders call for rigorous, independent evaluations as AI capabilities advance. Investm… |
@ValsAI |
Company |
Quote |
2026-09-12 |
7,131 |
82 |
17 |
5 |
1 |
8 |