May 2025 Summaries

FLUX.1 Kontext is a suite of generative flow matching models that enable in-context image generation and editing without the need for fine-tuning. It allows users to generate and edit images through text prompts, seamlessly extracting and modifying visual concepts to produce new, coherent renderings. The model combines character consistency, context understanding, and local editing capabilities with leading text-to-image synthesis, enabling iterative editing capabilities, step-by-step refinement, and interactive speed. FLUX.1 Kontext is available now on Together AI, with premium and maximum versions offering improved performance and features, and a developer version coming soon in private beta. The model is designed to preserve unique elements of an image, make targeted modifications, generate novel scenes while preserving styles, and edit text within images for updating marketing materials and social media content.

The Mixture-of-Agents Alignment (MoAA) framework harnesses the collective intelligence of open-source large language models (LLMs) to improve post-training performance. MoAA generates high-quality supervised fine-tuning data by combining responses from multiple open-source models, referred to as proposers in the Mixture-of-Agents (MoA) framework. This approach enables smaller models to achieve performance levels comparable to those of models up to 10x their size, while retaining the efficiency and cost advantages of small models. MoAA also enhances model alignment through direct preference optimization, which significantly improves upon the baseline models across all benchmarks for both Llama-3.1-8b-Instruct and Gemma-2-9b-it. The framework enables a self-improving pipeline to continuously enhance model performance, providing strong evidence that collective intelligence from multiple models can advance Large Language Models without relying on supervision from more powerful LLMs.

Together Code Sandbox provides a robust infrastructure for building AI-powered coding tools at scale, offering fast and secure sandbox spin-up times, customizable environments, and scalable resources. The platform is designed to support the growing demand for AI-driven coding solutions, from intelligent coding assistants to fully autonomous agents, and empowers developers to rapidly create innovative AI products without managing infrastructure. With its launch, developers can access cutting-edge technologies such as memory snapshotting and a suite of mature developer tools and APIs, enabling them to execute complex AI-generated workflows confidently. The platform is refined through extensive collaboration and feedback from leading AI developers and offers unmatched reliability and scalability, making it an ideal solution for powering AI pioneers.

Together Code Interpreter (TCI) is an API that allows developers to execute code generated by Large Language Models (LLMs) seamlessly with a simple API call. TCI solves the limitation of LLMs in executing code, enabling developers to build smarter apps with richer responses. By taking LLM-generated code as input, creating a session to execute it in a secure sandbox environment, and outputting the result, TCI simplifies agentic workflow development and opens new possibilities for reinforced learning operations. The API has been integrated into various applications, including reinforcement learning (RL), where it enables automated evaluation through rigorous unit testing, scaling to handle hundreds of concurrent sandbox executions, and providing secure environments that isolate execution, expose standard input/output interfaces, and integrate seamlessly into existing RL workflows. TCI is priced at $0.03/session, with a concept of "sessions" as the unit of measurement for usage and billing. The API also supports MCP (Microsoft Cognitive Platform) integration, allowing users to add code interpreting abilities to any MCP client. With its introduction, developers can now start using TCI today by using the Python SDK or API, and explore its benefits in lower cost, faster training, and network compression.

Together Code Sandbox provides customizable, instantly bootable microVM environments powered by robust, mature infrastructure, enabling developers to build full-scale development environments for AI and scale them effortlessly. It offers a suite of dev tools, including fast and scalable snapshotting, suite of dev tools, VM scaling, persistent storage, and more. The product is designed to bridge the gap between code generation and execution, allowing developers to focus on building solutions without manual setup. Together Code Interpreter is a session-based code execution in isolated Sandboxes via API calls that stream results, offering instant code execution for Python code generated by LLMs, secure and isolated sandbox environment, simple pricing, and more. The two products cater to different use cases and needs, with Together Code Sandbox ideal for AI IDEs or SaaS platforms and Together Code Interpreter suitable for agentic apps and RL training pipelines.

Together AI has acquired Refuel.ai, a company that specializes in cleaning and structuring data at scale, to help enterprises build production-grade AI applications. The acquisition aims to unlock the full potential of AI by providing a platform for developers to easily clean and transform unstructured data. With Refuel.ai's technology, Together AI will deliver an unmatched combination of speed, data quality, and scalability, enabling developers to rapidly take more sophisticated AI applications from concept to production. The integration of Refuel LLM-2 models into the Together AI Platform will make it easier for developers to build, deploy, and improve the quality of their agents over the entire lifecycle. This acquisition marks a significant step forward in the mission to accelerate the development of production-grade AI applications.

We leverage speculative decoding, a technique that optimizes speed and cost of our serverless and dedicated inference endpoints. This method uses a smaller, faster "speculator" model to speculate the next few tokens, which are then verified in parallel by the larger model. A strong speculator has two key properties: speed and alignment with the target model. By fine-tuning our speculators on specific domains of interest, we can gain higher speedups. Our state-of-the-art Base Speculator already provides 1.44-2.27x speedups over conventional next-token prediction for DeepSeek-R1 inference workloads. However, customizing our speculators further leads to additional speedups of 1.23-1.45x. This translates to a total speedup of 1.85-2.97x and a reduction in overall cost by ~25% compared to the Base Speculator. Furthermore, using custom speculators can increase throughput per GPU, thereby lowering overall inference costs. By training our Custom Speculators with data from specific workloads, we can achieve significant improvements, such as a 23%-26% reduction in GPU hours needed for generating 1B tokens. This demonstrates the effectiveness of our speculative decoding techniques and highlights the potential benefits of customizing speculators for individual workloads.

Arcee AI has simplified AI adoption by creating efficient, smaller language models that help enterprises integrate advanced AI workflows. The company transitioned its specialized small language models (SLMs) from AWS to Together Dedicated Endpoints, unlocking significant improvements in cost, performance, and operational agility. Arcee AI's focus on training SLMs optimized for specific tasks has produced high-performing models, including seven models available on Together AI serverless endpoints. The company's software layer, Arcee Conductor, uses a unique 150 million parameter classifier to intelligently route queries to the most suitable model, reducing latency and costs. Arcee Orchestra enables enterprises to automate tasks through seamless integration with third-party services and data sources, while simplifying infrastructure management and reducing costs. By migrating to Together Dedicated Endpoints, Arcee AI simplified its infrastructure and achieved performance improvements, including reduced latency and increased throughput. The company remains committed to continuously optimizing its GPU infrastructure, enabling effortless scaling on Together Dedicated Endpoints with superior performance, flexibility, and cost-efficiency.