New in llama.cpp: Decision Models

llama.cpp now supports decision models through its `/v1/systemone` endpoint, enabling applications to submit text, JSON, screenshots, or chat messages alongside typed questions and receive option probabilities in a single forward pass rather than generated text. Based on TypeSafe’s System One API format, the feature supports choice, scored-scale, and yes/no questions for tasks such as request routing, content moderation, agent-step verification, and action selection. Available models range from the 144M-parameter Julia-1 to the 27B OpenJev, with differing language, image, licensing, and performance capabilities; OpenJev currently supports image inputs. Users can run models locally, load models dynamically in router mode, batch independent questions, and select quantizations, while guidance emphasizes adding clear option descriptions, comparing models, and calibrating confidence thresholds on application-specific examples. The project plans to add newly released open decision models, with Cloudflare’s Clef identified as a forthcoming option.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.