September 2026 Summaries

Together AI describes how to build and deploy a Jev-like classification model by fine-tuning Qwen3.5 4B on roughly 38,000 normalized examples drawn from datasets covering natural-language inference, yes/no questions, banking intents, news topics, sentiment, policy decisions, routing, and research-paper taxonomy. The process involves cloning the Tev1 repository, configuring a Together API key, downloading and preparing source datasets, launching a fine-tuning job through Together AI’s service, and deploying the resulting model to a dedicated endpoint, with training estimated to cost about $17 and take roughly 25 minutes. The deployed model accepts structured JSON containing a state, question, and labeled answer options, then returns a selected option in a constrained format, illustrated by correctly identifying a duplicate subscription charge as a support intent. The post also provides inference settings and a system prompt intended to ensure deterministic, instruction-resistant classification behavior, and notes that endpoints can be stopped when no longer needed or replaced by Together’s hosted experimental Tev1 model.

Model rollouts provide a managed way to migrate live endpoint traffic from an existing deployment to a new model through canary, blue-green, or rolling strategies, replacing manual cutovers with staged capacity scaling, health checks, routing-propagation waits, and optional metric gates. Canary rollouts move traffic through defined percentages and can compare router-side latency, error rate, or in-flight requests between source and target deployments, automatically pausing for human review when regressions, unavailable metrics, capacity shortages, or policy constraints occur. Operators can create, start, monitor, pause, resume, promote, cancel, or reverse rollouts through the CLI, REST API, or Python SDK; cancellation freezes the current traffic split, while a reverse rollout moves traffic back safely. The platform is designed to prevent traffic from reaching unready capacity, preserve sufficient replicas for each traffic share, respect autoscaling policies, and avoid declaring a regressed step successful. In a demonstrated Qwen2.5-7B to Qwen3.5-9B migration, a p95 latency gate detected a 137% regression at 10% traffic, paused the rollout, and enabled reversal to the original model without failed requests across 6,800 live requests.

A global fintech uses Together’s Dedicated Model Inference platform to run its internal AI coding assistant on GLM 5.2, supporting unpredictable engineering-hours demand that made traditional capacity planning ineffective. Because the workload has relatively low request volume but large prompts and sharp concurrency bursts, the deployment prioritizes concurrency headroom across dozens of B200 GPUs rather than maximum raw throughput. DMI gives the company self-service endpoint provisioning, scaling, configuration, model upgrades, blue/green testing, and programmatic metrics access, reducing dependence on support tickets and central platform-team coordination. During a production queuing issue, metrics identified a pending-prefill backlog rather than insufficient compute capacity, allowing the team to adjust cache-aware routing and worker limits through a live, zero-downtime configuration change. The company migrated from GLM 5.1 to GLM 5.2 on the same account, chose 256K or 512K contexts over a 1M-context option to preserve concurrency, and is now planning a separate regional deployment for a customer-support NLP workload.

Migrating from closed-source AI models to open-source models can be faster and less complex than traditional technology migrations, particularly when managed services reduce operational burdens and existing model gateways and harnesses support multiple providers. A proposed strategy involves discovering candidates by defining the workload, using relevant benchmarks to narrow options, comparing practical measures such as cost per task, token use, latency, and verification time, and testing representative tasks. Evaluation should assess both model accuracy and runtime performance, with replayed production traffic providing more useful evidence than generic benchmarks when available. If a model falls short, organizations can iteratively adapt prompts, inference settings, context and tool integrations, or, when necessary, fine-tune or distill model weights while using modular evaluations to isolate improvements. After technical validation, teams should document migration effort, risks involving compliance, scaling, tooling, and downstream services, and expected return on investment, which may include substantial cost reductions and comparable quality. Production rollout can use a phased roadmap and canary deployments, such as initially routing a small share of traffic to open-source models, to validate real-world performance before broader adoption.

Together Fine-Tuning has expanded its platform to support a broader range of recent open-weight models, including GLM, Kimi, DeepSeek, Qwen, and Gemma variants, while adding tools intended to improve visibility, control, and cost efficiency throughout fine-tuning workflows. New experiment-tracking features expose live training and validation metrics through the API, CLI, SDK, and dashboard, enabling teams to compare runs and monitor model behavior during training. The release introduces expert-layer LoRA for Mixture-of-Experts models, early stopping based on validation loss, gradient accumulation for larger effective batch sizes, and training-price reductions of 30% to 70% for many models. Data-processing improvements include pre-flight JSONL validation, previews of tokenized and packed training data, per-example sample weights, and configurable sequence packing. Together also highlights Adaption’s use of its infrastructure for automated training and evaluation loops on models up to one trillion parameters, and plans to allow deployment of intermediate LoRA adapters before training completes, initially for GLM-5.3 and later Kimi K3.

Together has launched a public preview of preemptible compute for Kubernetes-based GPU Clusters in all regions, offering interruptible NVIDIA GPU capacity at a fixed 50% discount from on-demand pricing with sub-hourly billing. Preemptible nodes join existing clusters rather than requiring a separate cluster type, use unused capacity, and may be reclaimed with up to five minutes’ notice through Kubernetes events and SIGTERM signals, allowing workloads time to checkpoint and exit before removal. The service automatically replenishes the requested preemptible capacity when resources become available, although allocated capacity may remain below the target during shortages. It is intended for retryable or checkpointed workloads such as experiments, training sweeps, evaluations, batch inference, and temporary demand bursts, while critical services, coordinators, login pods, and strict-SLO serving should remain on standard non-preemptible nodes. Users can configure preemptible GPU targets through the Together Cloud console, CLI, or API and schedule workloads using a compute-class node label; clusters must retain at least one standard node, and Together plans future Slurm support, expanded regions, and in-place conversion between compute types.

Together AI’s kernels team evaluated NVIDIA’s Vera Rubin NVL72 platform and extended ThunderKittens to support optimized NVFP4 and FP8 GEMMs, finding that Blackwell-era kernels initially reached only about 42–44% of Rubin’s performance roofline because the faster tensor cores were insufficiently supplied with data. Vera Rubin retains Blackwell’s core programming model but adds a doubled 64-byte K-step for tensor-core MMA instructions, expanded tensor memory of up to 576 columns through exclusive allocations, 328 KiB oversized shared memory, B-side collector support for reducing operand reads, and early A-operand release to accelerate pipeline reuse. By widening MMA instructions, shifting from 1x1 to 2x1 tile layouts to reuse B operands, deepening shared-memory pipelines, tuning CTA cluster configurations and rasterization, using the B-side collector, and applying early A release for large workloads, the team raised its NVFP4 kernel performance to more than 22 PFLOPS and made it competitive with cuBLAS and CuTE DSL. Results were measured on a qualification-sample GPU with CUDA 13.4, and the authors note that software and baseline performance may continue to evolve as Vera Rubin releases mature.

Open-weight AI models are increasingly viable alternatives to closed systems for software development, offering developers greater ownership, flexibility, and potential cost savings without requiring model-training expertise or local GPU infrastructure. The proposed “MIGHT” stack separates the model, inference provider, gateway or router, harness, and tools such as skills and Model Context Protocol servers, allowing each component to be selected or replaced independently. Large models are presented as better suited to complex, ambiguous, multi-step tasks such as architecture changes and reviews, while smaller models can deliver faster, cheaper performance on narrowly defined work; model selection should therefore depend on the task rather than a single benchmark ranking. Cloud providers, gateways, and local routers enable access to diverse models and simplify switching among providers, while harnesses manage conversations, codebase context, tool calls, permissions, and file changes. Effective use also depends on managing context through fresh sessions and potentially dividing work among planning, implementation, and review models. This composable approach lets developers retain a stable workflow while continuously testing and adopting models, providers, and tools that best fit their requirements.