September 2026 Summaries

Surge AI describes its role in Anthropic’s research on automated alignment researchers, which used Claude-based agents to search literature, propose safety interventions, train and evaluate models, and iteratively improve methods across ten alignment problems such as deception, sycophancy, jailbreaks, privacy violations, and reward hacking. Surge built and operated the human comparison baseline, recruiting 28 experienced AI safety researchers who each had up to eight hours to submit a single proposal for selected failures, while also managing structured submissions, quality control, and expert review. Anthropic reported that its automated researchers produced stronger methods than the human-proposed baselines on all seven failures with human comparisons, though it noted that agents had the advantage of repeated iteration. According to the account, successful methods improved safety metrics without substantially reducing general capabilities and showed transfer to held-out benchmarks, open-ended audits, and models up to 4.7 times larger, illustrating a potential role for automated systems in complementing human alignment research.

Surge AI reports that reinforcement-learning-only post-training of Kimi K2.7 on 1,700 coding tasks improved its Pass@1 performance across five external coding benchmarks by 4.7 to 20 percentage points, with gains transferring across three different agent harnesses and, in some cases, surpassing reported results from larger models. Analysis of paired trajectories suggests the trained model became more effective not by taking more steps, but by reducing median trajectory lengths while improving four software-engineering behaviors: retaining every specification detail, testing requirements rather than only supported implementation paths, preserving existing functionality against regressions, and constructing independent checks when no reference answer is available. The training data combined repository-change and terminal-deliverable tasks with hidden checks, while a reward function gave partial credit for completed requested behavior but assigned zero reward if existing tests regressed. Although the authors say they have not conducted ablations proving the reward design caused the changes, they argue that the results show high-quality RL environments can teach transferable engineering habits, helping a model move from solving the core task to reliably delivering complete, shippable software.

Surge AI’s comparison of recent frontier-model releases finds that Fable 5.1 leads overall on its Tuesday Work Index with a score of 68.7 and tops benchmarks for professional chart reasoning, policy-document instruction following, and enterprise agent performance, though its cost efficiency varies by task. Muse Spark 1.3 records the largest generation-to-generation index gain and leads ComplexConstraints, where it combines strong performance on interacting professional requirements with favorable costs, while showing uneven results on chart reasoning and advanced mathematics. Gemini 3.8 Flash reaches 61.1 on the index and is highlighted for cost-efficient hard reasoning, particularly after a 12-point improvement on Riemann-bench, although higher reasoning settings do not consistently improve its results across workloads. Together, the releases suggest that top-end capability continues to advance while lower-cost models narrow the gap, and GPT-6 Astra remains absent from the comparison pending completion of its evaluation.