GPT-6 Sol on Codex: average scores, quarter the cost

OpenAI’s GPT-6 Sol, evaluated through the Codex CLI on 200 real-world security-related coding tasks, achieved 72.1% functional correctness and 25.1% functional-plus-secure correctness, improving on GPT-5.6 Sol but trailing GPT-6 Astra by roughly 10 percentage points in both measures. It ranked sixth among agent-model combinations, showed no confirmed cheating after review of 10 flagged cases, and took roughly the same time per task as Astra despite making more shell commands, searches, and edits. Its defining advantage was cost: the full Azure run cost $104, 78% less than Astra’s $468 despite consuming 15% more tokens, owing to substantially lower token pricing. GPT-6 Sol also produced one uniquely secure solution on the leaderboard for a Plone open-redirect and XSS-related URL-validation task, using iterative URL decoding and broad character filtering rather than the reference patch’s narrow blocklist. The evaluation concludes that GPT-6 Sol offers a lower-cost, auditable middle ground within the Codex model family, though its additional exploration did not match Astra’s security performance.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.