Introducing jevals: typed decisions for agent evals and guardrails

jevals is an open-source Python library in alpha for evaluating AI agents and applying production guardrails through Jev-style typed decision models, which return structured choices, scores, and probabilities rather than generating freeform judgment text. It includes 37 built-in checks spanning agent behavior, response quality, and security, including tool selection, groundedness, task completion, scope adherence, prompt injection, and sensitive-information handling, while also supporting custom Python or YAML evaluations. The same definitions can be used in development, replayed against saved traces, and connected to gates that allow, block, or escalate agent actions for human review. The project supports hosted Jev, local Kev and Laya models, and conventional LLM backends, although thresholds may need recalibration across models. Its creators argue that decision models can reduce cost and latency for classification-style evaluations, citing a limited RAG benchmark where four jevals metrics cost an estimated $0.03 per 1,000 samples compared with $2.60 for Ragas using GPT-4.1-mini, while noting that results are setup-specific and should be validated independently. Jev is also now available as a judge in Openlayer, which continues to provide classifier evaluation features such as F1 scores, confusion matrices, drift monitoring, and explainability.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.