Home / Companies / Arize / Blog / Post Details
Content Deep Dive

Anthropic says it fixed Claude’s writing. I ran the evals to check.

Blog post from Arize

Post Details
Company
Date Published
Author
Jim Bennett
Word Count
2,509
Company Posts That Month
23
Language
English
Hacker News Points
-
Post removed?
No
Summary

An Arize employee evaluated Anthropic’s claim that Claude Opus 5.5 produces less “Claudish” writing by comparing it with Opus 5 across 20 fixed research briefs spanning multiple genres and domains. Using identical prompts and settings, two runs per model, human annotations of 153 stylistic issues, and evaluators for rhetorical patterns and em-dash use, the study found that em dashes dropped from 12.9 to 0.05 per 1,000 words and identified Claudisms fell from 4.79 to 2.38 per 1,000 words, a 50% reduction. The analysis argues that these patterns are primarily rhetorical moves—such as announcing a point’s importance, using contrast reframes, intensifying verdicts, or teasing insights—rather than a small set of recognizable words. Opus 5.5 still retained some of these habits and showed newer tendencies including more bold lead-in bullets, more three-part lists, and slightly longer responses. Iterating on the automated judge against annotated examples improved its precision by distinguishing filler from phrases that conveyed necessary information, leading to the conclusion that Opus 5.5’s writing is substantially improved but not fully “fixed,” while demonstrating the value of controlled, annotation-driven model evaluation over informal impressions.

Trends Found in this Post
Trend Post Mentions Total Month Mentions Posts Companies MoM
Opus 5.5 22 523 34 18 -
LLM 2 6,030 1,165 243 +15%
Use This Data

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.