November 2026 Summaries

Together AI describes “Polymorphic Inference” as an adaptive approach to AI serving designed for rapidly changing open-source model releases, architectures, workload patterns, and hardware platforms, contrasting it with older “Static Inference” configurations optimized for relatively fixed models and traffic. The company argues that hand-tuned kernels and fixed inference-engine settings often fail to remain optimal across diverse real-world conditions, particularly for evolving workloads and architectures such as mixture-of-experts models. Its proposed response is Continuous Compilation, led by the Together Compiler, which uses AI agents and iterative performance testing to search for and apply workload-specific optimizations in production. Together AI says the compiler is already used on selected serverless endpoints and plans to offer customers self-service optimization for dedicated deployments. Its broader research agenda includes adaptive speculative decoding, dynamic routing across heterogeneous infrastructure through ThunderAgent, post-training and customization of open-source models, and workload-aware routing among models, all intended to improve performance and expand access to AI systems.

Together Compiler (TKCC) is an agent-driven continual optimization system designed to adapt AI inference engines to rapidly changing combinations of models, hardware, traffic patterns, and user workloads. Together AI argues that manual tuning and fixed compiler heuristics cannot scale across diverse architectures, including mixture-of-experts and sparse-attention models, or across hardware platforms with different performance characteristics. TKCC accelerates experimentation through persistent inference engines that hot-reload candidate optimizations instead of repeatedly restarting, multiple collaborating agents, and a two-tier measurement system that balances rapid testing with validation on realistic data distributions. It also uses a cost model combining hardware information, microbenchmarks, and measured forward passes to choose parallelism configurations as traffic shifts. The company reports that TKCC achieves up to 2.6 times higher throughput than optimized open-source inference baselines, reaches reported low-latency benchmark results of 1,111 tokens per second for MiniMax M3.1-Flash and 1,205 for DeepSeek v4.1-Flash, and outperforms open-source megakernels by 1.7 times on a Llama 1B workload. Beyond language-model serving, the system is presented as applicable to weather modeling, partial differential equations, time-series analysis, and biology workloads, reflecting Together AI’s broader concept of “polymorphic” systems that continuously reoptimize as conditions evolve.