November 2026 Summaries

No summary generated yet.

Surge AI reports that reinforcement-learning post-training of NVIDIA Nemotron 3.5 Super VL, a 120B-parameter model with 12B active parameters, using four specialized datasets improved performance across 12 benchmarks covering software engineering, complex instruction following, long-horizon enterprise workflows, and chart understanding. Separate LoRA adapters trained through Thinking Machines’ Tinker produced reported gains across all evaluated tasks, including increases in AutomationBench workflow completion from 15% to 28% and HANDBOOK.md requirement satisfaction from 55% to 68%, while reaching scores comparable to much larger models on several benchmarks. Analysis of model behavior found that training made Nemotron more likely to inspect emails, Slack messages, calendars, code tests, and chart data before acting; more consistent at completing all requirements; and better able to transfer these habits to unseen multi-turn and system-prompt settings. The results suggest that targeted, high-quality reinforcement-learning data can substantially enhance the reliability and competitiveness of efficient open models without requiring frontier-scale active parameter counts.

Surge AI evaluated several coding-agent harnesses using Opus 5.5 on its sudo L7 benchmark, which measures staff-level engineering judgment beyond passing tests, including verification, practicality, communication, and interpretation of underspecified requests. The study found that raising the model’s thinking effort from medium to xhigh produced the largest gain, improving scores by 11.3 points at 3.5 times the cost, while Superpowers and Claude Code’s /goal plus /code-review plugins delivered smaller gains of 2.4 and 3.3 points respectively. A more elaborate multiagent Compound Engineering workflow cost roughly eight times as much as vanilla Claude Code without a statistically clear improvement, and an example involving a Zoom passcode showed how 17 agents could review a change yet miss the key product issue of giving unpaid registrants event access. Higher thinking effort particularly improved verification, persistence, and collaborative reasoning, as illustrated by an agent that traced a bug beyond its immediate cause to prevent a second crash. The findings are limited to one model, selected configurations, and the benchmark’s task distribution, but they suggest that teams should test workflows against real tasks and consider increasing reasoning effort before investing heavily in complex agent orchestration.