June 2025 Summaries

Fireworks has introduced a new OpenAI-compatible Responses API with support for the Model Context Protocol (MCP), which is designed to connect large language models (LLMs) with proprietary APIs, tools, and data sources, solving the challenge of integrating these models into business operations. This open protocol facilitates seamless interaction between models and external systems, allowing developers to connect any model on Fireworks to a variety of tools, freeing them from vendor lock-in and enabling more flexible and powerful applications. The MCP handles the entire agentic loop—reasoning, tool selection, and execution—on the server side, simplifying the integration process and enabling rapid development of sophisticated applications. This innovation allows businesses to build applications such as e-commerce automation, internal operations bots, and data-driven assistants, all while maintaining control over their unique business logic and data connections. Fireworks encourages developers to explore this capability, which represents a step toward better-connected AI systems, and invites feedback on this preview feature.

Fireworks Virtual Cloud is a platform designed to simplify the management of GPU deployments for AI applications, offering a scalable and reliable solution that supports over 18 global regions across eight cloud providers. It abstracts the complexities of handling hardware failures and scaling workloads, enabling users to focus on creating exceptional product experiences. The platform leverages the latest hardware from NVIDIA and AMD, including cutting-edge GPUs like the NVIDIA B200s, and employs a workload-aware infrastructure to optimize performance. Fireworks also provides flexible global scheduling and high reliability, with features like prompt caching and multi-tiered traffic routing, ensuring efficient and uninterrupted service despite potential hardware failures. Additionally, the platform supports a bring-your-own-cloud (BYOC) option for enterprises needing to maintain data security and control over their hosting environment.

Fireworks has developed the 3D FireOptimizer as part of its FireOptimizer toolkit to assist in optimizing LLM serving by automatically navigating complex tradeoffs between speed, cost, and quality, tailored to unique workload requirements. This tool examines various factors such as model architecture, hardware selection, quantization, speculation strategies, and parallelism to determine optimal configurations without requiring users to manually sift through an extensive array of possibilities. By leveraging performance data, rule-based heuristics, and customer-specific tuning, 3D FireOptimizer identifies configurations that enhance quality, throughput, and latency, thereby offering significant improvements over baseline setups. Through case studies, Fireworks demonstrates the tool's ability to improve performance in different scenarios like code completion and chatbot applications, emphasizing its practical utility in real-world production environments.

Fireworks introduces Supervised Fine Tuning V2 (SFT V2), an enhanced version of its fine-tuning service designed to optimize data and models, which are seen as core assets for companies aiming to deliver premium user experiences and strong product differentiation. SFT V2 is a complete rewrite offering improved quality and faster training speeds, supporting a broader range of models including the Qwen, Phi, Gemma, and Llama series, as well as open-source MoE models like Deepseek. Key features include longer context lengths, quantization-aware training with FP4 and FP8 options to maintain inference quality, and multi-token prediction for faster generation speeds. The service also supports multi-turn function calling and offers training speeds twice as fast as its predecessor, with options to use multiple GPUs for even faster processing. Multi-LoRA allows for loading multiple LoRA addons onto a single model deployment, enhancing flexibility and reducing cold-start times. This release aims to transform data into high-quality customized models, thereby creating a self-improving data flywheel effect.

Fireworks is a vision model platform that offers enterprises advanced tools and capabilities for processing unstructured visual data, such as scanned documents and product images, to unlock new business opportunities and enhance digital experiences. By integrating vision-language models (VLMs) with large language models (LLMs), the platform supports various innovative applications across industries, including healthcare for eHR integration, e-commerce for product catalog management, and insurance for claims processing. Fireworks provides an OpenAI-compatible API, enabling users to perform tasks like generating product descriptions and language localization from images. Recent updates include the addition of new models like Llama 4 Scout & Maverick, InternVL3, and RolmOCR, which enhance image comprehension, reasoning ability, and OCR accuracy, respectively. The platform also offers prompt caching to improve latency and supports LoRA uploads for customizing vision models to specific application patterns. Fireworks aims to deliver real-time visual intelligence with high efficiency, enabling enterprises to break down data silos and create intelligent systems that leverage both visual and textual data.

The Fireworks Experimentation Platform and Build SDK are designed to accelerate AI development by significantly reducing iteration times from weeks to hours. The platform facilitates rapid prototyping and experimentation with open models by offering features like on-demand flexible capacity, LoRA add-ons for parallel fine-tuning experiments, and smart resource management to optimize deployment. The Build SDK, now in beta, enables developers to manage AI workflows programmatically using Python, with a focus on object-oriented, declarative, and customizable design principles. This infrastructure aims to eliminate traditional complexities and allow developers to focus on creating innovative AI solutions, offering seamless transitions from training to production and supporting extensive experimentation across multiple models and configurations.

Reinforcement Fine-Tuning (RFT) is a new technique announced in beta, aimed at enhancing expert models for complex tasks like agentic reasoning, function calling, and coding by leveraging Reinforcement Learning with Verifiable Reward (RLVR). RFT allows for improved model quality with minimal examples and can outperform closed frontier models in both quality and speed, as evidenced by its application in customer service AI agents and code generation with partners like Vercel. It simplifies the traditionally complex setup of reinforcement learning by automating infrastructure and training management, requiring only a Python evaluator function to grade model outputs. This approach extends to creative writing by using large language models as judges for tasks that require subjective evaluation. The Fireworks platform facilitates training without the need for complex infrastructure, and it is currently offering free access to train open models like Llama and DeepSeek for two weeks, encouraging users to explore various applications and contribute their ideas.

Fireworks AI has developed an innovative synthetic data pipeline designed to streamline the creation and fine-tuning of machine learning models by automating synthetic data generation, quality control, and iterative fine-tuning processes. This pipeline reduces the time typically required for model development from weeks to just hours by leveraging large language models (LLMs) for orchestrating generation logic, applying dynamic constraints, and driving intelligent iteration through automated evaluation loops. It includes five interconnected stages, from task definition and configuration generation to dataset customization, automated fine-tuning, and synthetic data cleaning. The system enhances model performance by using synthetic data to train models without relying on real-world data, thereby ensuring compliance with data privacy regulations. Future enhancements will incorporate interactive YAML builders, model jury consensus mechanisms, and batch APIs, positioning this pipeline as a foundational tool for efficient AI model development.