Judgment, not generation: rebuilding our AI API Classifier on Jev | Harness Blog

Harness describes rebuilding its API-classification pipeline for identifying endpoints that interact with large language models, a security-relevant task because such endpoints may expose organizations to prompt injection, sensitive-data leakage, and unreviewed third-party vendors. The original system used multiple generative LLM calls to summarize traffic spans and classify each endpoint, costing about $176 per thousand endpoints, while a simplified one-call LLM approach reduced that to roughly $22 per thousand but retained high latency and self-reported confidence labels. A hybrid alternative used Jev, a purpose-built decision model, for binary classification and calibrated probabilities, while reserving LLM generation for human-readable explanations on positive or uncertain cases; it was estimated to cost about $0.14 per thousand endpoints and achieved median latency of 0.80 seconds. On a human-verified set of 500 endpoints, Jev reached 98.6% accuracy for identifying AI APIs, with all seven errors concentrated in a 0.3–0.8 uncertainty range covering only 3.4% of examples, enabling targeted LLM fallback or human review. The post argues that maintaining a fixed four-field output contract allowed the decision engine to be replaced without altering downstream dashboards, enrichment workflows, or product interfaces.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.