Jev Applied: Scoring experiment quality with a decision model

GrowthBook is developing an experiment quality score that evaluates whether hypotheses clearly identify a change, predict a measurable outcome, explain why it should occur, and align with selected goal metrics. Rather than relying on a general chat model to produce variable written scores, the team uses TypeSafe’s Jev decision model to answer narrowly defined yes-or-no classification questions and combines those answers in code. In tests using 30 hand-labeled hypotheses, Jev processed an experiment in about 0.1 seconds compared with roughly 1.3 seconds for gpt-5.4-mini, while an estimated 10,000 passes would cost about $0.25 at 600 input tokens each. This speed and cost allow checks to rerun shortly after edits to experiment hypotheses or metrics, providing feedback during experiment setup instead of after results are available. GrowthBook will continue using chat models for custom, open-ended checks, while recommending decision models for fixed classification tasks that can be broken into small, testable questions and validated against human-labeled examples.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.