February 2026 Summaries

CoderForge-Preview is the largest open dataset of coding agent trajectories available, comprising 258,000 test-verified trajectories from 51,000 tasks across 1,655 repositories, designed to address the shortage of high-quality open training data that limits the advancement of open-weight coding models. The dataset was used to train models with 32 billion and 4 billion parameters, achieving significant performance improvements, particularly with Qwen-3 32B, which ranked highest among open-data models in the ≤32B parameter range on SWE-Bench Verified. The dataset generation involved using Qwen3-Coder-480B and filtering through rejection sampling, resulting in 155,000 successful trajectories out of 258,000 generated. The data is drawn from sources like R2E-Gym, SWE-Smith, and SWE-Rebench, and is set within a standardized action/observation interface using the OpenHands scaffold. The work emphasizes training only on successful trajectories to enhance task resolution efficiency, and the dataset underwent a thorough license audit to ensure responsible use by retaining only trajectories under permissive open-source licenses. Despite its success, the dataset has limitations such as adaptability to different scaffolds, a focus on bug-fixing tasks, and lack of user interaction modeling, but it aims to drive further advancements in open-source AI development by providing a foundation for exploring agentic reinforcement learning and larger model scales.

Automatic speech recognition (ASR) systems, while achieving near-human parity on general benchmarks, struggle significantly with transcribing short, high-stakes utterances such as street names, especially when pronounced by non-English speakers. Evaluations using the SF Streets and US Streets datasets reveal that models from companies like OpenAI, Deepgram, Google, and Microsoft exhibit an average error rate of 39% for street names, highlighting a gap in real-world reliability and an 18-point accuracy disparity between English and non-English primary speakers. This disparity leads to costly practical consequences, such as increased taxi fares and driving time. To address these issues, researchers developed a synthetic data generation technique using cross-lingual style transfer, which improves ASR performance by up to 60% with minimal data. This method demonstrates that enhancing model robustness is achievable without extensive data collection, and the release of the SF Streets and US Streets datasets aims to encourage further research into improving ASR systems' reliability in diverse linguistic environments.

Diffusion Language Models (DLMs) offer a promising alternative to traditional autoregressive language models by allowing for parallel generation and leveraging bidirectional context for tasks such as text infilling and refinement. However, standard DLMs face inefficiencies due to the incompatibility of KV caching with full bidirectional attention and the need for numerous refinement steps to maintain quality, which is both computationally expensive and time-consuming. The CDLM approach addresses these bottlenecks by employing a training-based acceleration scheme that enforces within-block temporal consistency and utilizes a block-wise causal mask, enabling exact KV caching and reducing the number of refinement steps without significant loss of accuracy. This method leads to substantial latency improvements and increased throughput, making CDLM particularly effective for math and coding tasks by achieving faster inference, fewer steps, and higher efficiency, all while maintaining competitive accuracy. Through system-level analysis, CDLM is shown to strike a balance between computational intensity and memory use, making it an efficient choice for small-batch settings.

Together's Dedicated Container Inference is a specialized solution designed to optimize production-grade orchestration for custom AI models, particularly those requiring GPU-intensive workloads. Unlike traditional inference platforms that focus on a single abstraction, Together offers a flexible, container-based framework that allows users to run custom inference code in production without building their own orchestration layer, addressing needs such as autoscaling, queuing, traffic control, and monitoring. This approach supports diverse workloads, including video generation and avatar synthesis, by enabling multiple independent queues, policy-driven traffic control, and predictable behavior during demand spikes. Together's platform facilitates seamless transitions from model training to deployment, minimizing operational overhead and enhancing model performance through hands-on optimization. By allowing teams to focus on building products rather than managing clusters, it delivers substantial speed and cost efficiencies, making previously uneconomical models viable for production.

In the realm of AI applications that demand long context lengths, a novel approach called cache-aware prefill–decode disaggregation (CPD) is enhancing the efficiency of inference systems. As AI tasks like multi-turn conversations and coding copilots become increasingly common, handling large prompts efficiently is crucial. Traditional systems struggle with varying time-to-first-token (TTFT) due to shared context demands, especially when dealing with both new (cold) and previously encountered (warm) requests. CPD addresses this by employing a three-tiered system that separates heavy computation from context reuse, thus optimizing hardware utilization and reducing latency through efficient cache management. By distinguishing between requests with high and low context reuse, CPD effectively allocates resources, preventing cold requests from monopolizing capacity and ensuring warm requests are processed swiftly. Evaluations show that CPD enhances throughput by up to 40% and maintains lower latency under high load compared to conventional models, making it a significant advancement in handling long-context AI workloads.

Research into the behavior of large language models (LLMs) reveals that near-unconstrained generation exposes unique insights into their innate preferences and biases, which are not apparent when they are constrained by specific prompts or templates. By using open-ended, topic-neutral seed prompts, researchers observed that different model families exhibit distinct semantic tendencies and levels of content complexity, with some models like GPT-OSS defaulting to advanced programming and mathematics, while others, such as Qwen, produce multiple-choice questions. The study also highlights that models tend to generate repetitive or degenerate text, which can serve as indicators of safety and privacy risks. These findings reveal systematic patterns that persist across various setups, suggesting that understanding LLMs requires examining their default generative behaviors alongside standard benchmark evaluations.

TogetherCoder-Preview is a significant advancement in open-source AI, offering the largest open dataset of coding agent trajectories, comprised of 161,000 test-verified trajectories across 54,000 tasks from 1,639 repositories. This initiative aims to address the critical limitation of high-quality open training data in the AI research community, enabling wide-scale research and development by making both dataset and model weights fully open. The dataset was curated using rejection sampling to ensure high quality, and the resulting models trained on it achieved notable performance, with the 32B model attaining a 59.4% pass rate on SWE-Bench Verified, ranking it highly among open-weight and open-data models. The project's methodology includes generating agent trajectories from diverse task sources and systematically filtering solutions for quality, supporting robust training of long-horizon coding agents. While the dataset exhibits significant scale and context length, the study acknowledges limitations such as adaptability to different scaffolds and scope beyond bug-fixing tasks, suggesting future efforts will involve exploring larger model scales and reinforcement learning enhancements.

Rime's Arcana V3 line addresses the challenge of natural code-switching in voice agents, offering seamless transitions between languages without the latency issues caused by routing between language-specific models. Together AI has integrated Rime Arcana V3 Turbo and V3 into their Model Library, providing efficient bilingual and multilingual capabilities. V3 Turbo delivers near-instantaneous English-Spanish code-switching, while V3 supports 11 languages, maintaining prosody and accent consistency. These models operate on Together AI's infrastructure, which ensures high performance, low latency, and full compliance with industry standards. This integration benefits bilingual metro markets, regulated services, and international call centers by reducing the need for multiple models and enhancing customer service experiences.

Rapid advancements in AI capabilities are driving a surge in AI-native applications across various sectors, necessitating a significant expansion in infrastructure to support this growth. Together AI, through its AI Native Cloud, is aiding over a million AI engineers and researchers by building AI factories designed to scale applications and maintain consistent user experiences. These factories integrate comprehensive AI stack layers, including GPU clusters and optimized inference stacks, underpinned by deep systems research to achieve peak performance. With the aim of constructing AI factories in multiple locations with vast capacities, Together AI has appointed Alon Gavrielov as VP of Infrastructure Strategy, drawing on his experience at Cloudflare to enhance infrastructure strategy and expand AI factories. Gavrielov emphasizes the critical role of robust infrastructure in the success of AI-native companies, as Together AI endeavors to facilitate seamless scaling and innovation for AI teams.

Together Evaluations introduces a comprehensive framework for assessing the quality of large language models (LLMs), facilitating comparisons between open-source, fine-tuned, and proprietary models. This framework allows teams to make data-driven decisions regarding model selection and optimization by evaluating models side-by-side using standardized metrics and methodologies. The latest update includes support for closed-source frontier models from major providers like OpenAI, Anthropic, and Google, enabling cross-model benchmarking. The platform also offers capabilities to evaluate fine-tuned models using Together AI's deployment options and provides resources such as a deep dive and cookbook to guide users through optimizing and evaluating models, demonstrating how open-source models can outperform closed-source ones with significant cost and speed advantages. The evaluation services also include automated prompt optimization using frameworks like GEPA, which enhances prompt efficacy through iterative LLM-guided reflection. Users can leverage the platform via UI, API, or Python client, with comprehensive documentation and tutorials available to aid in the use of these new features.

The text discusses the use of preference optimization to train open-source language models (LLMs) that outperform GPT-5.2 in aligning with human preferences, using the Reward Bench 2 benchmark. The study highlights that models like GPT-OSS 120B and Qwen3 235B can be fine-tuned to match or surpass GPT-5.2 in human preference alignment through Direct Preference Optimization (DPO), a method that optimizes models based on preference pairs. The concept of LLM-as-a-judge is explored, where LLMs are used to evaluate other LLM outputs by focusing on simpler classification tasks, such as determining which response is better or if a text contains harmful content. The experiment reveals that while Qwen3 235B outperforms GPT-5.2 without tuning, GPT-OSS 120B shows significant improvement post-fine-tuning, particularly in math and subjective response quality. The analysis underscores the cost-effectiveness and flexibility of open-source models, which offer transparency and lower costs compared to closed-source alternatives like GPT-5.2, making them a promising option for production evaluation systems.