November 2026 Summaries

A Patronus AI study of 12 open reasoning models on 20 relatively easy agentic coding tasks found that newer releases of DeepSeek, GLM, and Kimi used substantially less linguistic filler in their reasoning than predecessors, reducing filler-word shares by an average of 41% while matching or modestly improving task success. The analysis classified self-talk, articles, auxiliary verbs, certain connectors, and self-references as filler after excluding code and commands; article use showed the largest reductions, while self-talk declined less consistently. Kimi K3.1 Pro had the leanest reasoning at 9.8% filler, whereas Qwen3.8 Flash was highest at 23.7%, close to the 20.8% measured in human-written SWE-bench bug reports. Terser prose did not reliably reduce cost: DeepSeek V4.1 Flash used 145% more reasoning tokens than its earlier version and Kimi K3.1 Pro used 44% more, while only GLM-5.3-Flash became cheaper. The authors suggest possible explanations including reinforcement-learning length incentives, distillation from terse teacher models, outcome-based training that permits private shorthand, and pressure from structured agent formats, but emphasize that these hypotheses remain unproven and that results may not generalize to harder tasks or other reasoning settings.

GlobeBench is an open-source benchmark for evaluating whether language models can accurately simulate environment responses after agent trajectories, containing 1,632 tasks spanning five domains, 19 simulation skills, and four capabilities: state coherence, causal correctness, interface fidelity, and constraint enforcement. Tasks provide a trajectory and final action, with model predictions compared against deterministic execution-derived reference states; 1,137 tasks were created by the authors and 495 adapted from existing executable backends. Across 20 general-purpose and specialized world models, GPT-6 Astra achieved the highest success rate at 49.57% (809 tasks), while no model exceeded 50%, 631 tasks were solved by none, and four Claude models incurred many failures through safety-related refusals. Performance was especially weak on schema conformance, long-context tasks whose decisive evidence occurred over 40 turns earlier, and actions with indirect effects on unnamed entities, where pooled success fell to roughly 5%. Incorrect predictions most often appeared as confident, well-structured outputs containing one or more wrong values, highlighting risks in using imperfect simulators for reinforcement-learning training. The authors note that the benchmark measures only single transitions rather than compounding rollout errors, and have released the tasks, answers, code, graders, model outputs, and judgments for reproducibility and further research.