I put a ghost in Claude Code. Then I evaluated it

Claude Code mods are TypeScript-based extensions that can alter prompts, tool calls, terminal UI, and built-in behaviors more extensively than existing hooks, though they run with user permissions and are not sandboxed. Claude Code includes several built-in mods, including support for AGENTS.md instructions, an enhanced diff view, mod-authoring assistance, organizational security protections, telemetry, and an optional side agent that identifies possible overlooked issues. To examine how mods can be measured, the author built “trick-or-treat,” a Halloween-themed mod that uses Claude Haiku to classify tool-using turns as careful “treats” or sloppy “tricks,” displays verdicts and tallies, and blocks selected destructive shell commands. Tracing experiments with Arize AX found that the mod did not affect task completion but added token costs and modest latency, while blocked commands could often be replaced by alternative commands that achieved the same result, showing that regex-based guards are not security boundaries. Evaluation across three versions of the mod improved agreement with a reference judge from 9 of 20 turns to 16 of 20 by adding tool outputs and respecting explicit user instructions, but it also increased input-token use and exposed policy disagreements about whether untested but openly acknowledged edits should be penalized. The account concludes that mods should be treated like AI agents: their operational costs and behavior should be traced, their judgments should be evaluated with well-contextualized and independently reviewed references, and their security limits should be understood before relying on them.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.