July 2025 Summaries

Fireworks' Batch API enables users to run large volumes of asynchronous requests on over 1,000 open or fine-tuned models, offering significant advantages such as no rate limits, a 50% reduction in costs, and a maximum 24-hour turnaround for results. This service is particularly beneficial for tasks like model benchmarking, bulk data generation, data augmentation, and large-scale ETL processing. Users can start by preparing their dataset in JSONL batch format, launching a batch job by selecting from a wide array of models, and then downloading the results once the job is complete. The Batch API is designed to handle datasets up to 500 MB in size, facilitating efficient and scalable processing for various AI-driven applications.

Fireworks AI addresses the challenge of selecting the best open-source model from the rapidly expanding landscape by introducing real-world benchmarks tailored to specific tasks. Their Real-World Leaderboard evaluates models based on practical applications rather than broad academic benchmarks, enabling developers and businesses to choose the most suitable models for tasks such as customer support classification, e-commerce search, and complex agent workflows. Initial findings highlight the Qwen Instruct model's superiority in knowledge-heavy tasks, Qwen3 Coder's competitiveness for simple tool-calling scenarios, and Claude Sonnet 4's dominance in complex, multi-step reasoning tasks. Fireworks AI's approach aims to eliminate guesswork, offering model recommendations that balance proprietary and open-source options according to user preferences and task requirements.

Fireworks AI has introduced supervised fine-tuning for Vision-Language Models (VLMs) with the Qwen 2.5 VL family, allowing users to adapt these state-of-the-art models to specific visual domains for enhanced accuracy in specialized tasks. This platform addresses the limitations of generic models by enabling enterprises in sectors like healthcare, finance, and e-commerce to leverage their domain-specific visual data, improving applications such as automated document processing and multimodal workflows. Fine-tuning VLMs on Fireworks AI is geared for production, offering optimized training speeds, extended context support, and low latency deployments, ensuring efficiency and cost-effectiveness. The platform simplifies the fine-tuning process, allowing users to format their datasets, upload them, launch training, and deploy custom models with ease. With comprehensive monitoring and the ability to handle complex visual documents, Fireworks AI provides the tools needed to transform visual data into a competitive advantage.

Notion's collaboration with Fireworks AI has enabled it to transform from a tool for individual users to an enterprise powerhouse, serving over 100 million users, including nearly 70% of Fortune 100 companies. Facing the challenge of scaling AI beyond simple chat functions to more complex, enterprise-grade workflows, Notion aimed to create AI agents that could seamlessly integrate into tools like Slack, Jira, and GitHub. By fine-tuning smaller, efficient models provided by Fireworks AI, Notion was able to reduce latency from 2 seconds to 350 milliseconds, significantly enhancing performance and facilitating the launch of AI features at scale. This partnership allows Notion to deliver a natural and intuitive AI-powered experience, aligning with their vision of "vibe working," and supports their expansion by empowering hundreds of engineers to develop and maintain AI-powered workflows. As Notion continues to scale its AI engineering, the company is poised to adapt and evolve its AI capabilities, ensuring rapid iteration and reliable scaling for future use cases.

VibeRL is an AI agent designed to simplify the fine-tuning process of Reinforcement Learning (RL) by automating complex setups, allowing users to merely provide a dataset and a prompt. This innovation eliminates the need for intricate manual adjustments, making RL more accessible. Preliminary tests demonstrate VibeRL's efficacy in diverse tasks such as grade school math and coding challenges, with notable improvements in accuracy. For example, in solving math problems, accuracy increased from 52% to 85%, and for LeetCode challenges, it rose from 34% to 63%. VibeRL's automation of RL workflows holds promise for making model customization easier, with plans to develop it into a product that requires less engineering and more intuitive interaction, enabling users to leverage data effectively without deep technical expertise.

The text explores the Multi-Head Latent Attention (MLA) mechanism used in large language models (LLMs) like Kimi K2, highlighting its efficiency in handling memory and computational demands during inference by compressing keys and values into a lower-dimensional space. A technical exchange on a blog led to the revelation of the inherent challenges faced during training and inference phases, specifically the structural differences in how keys are formed. This difference leads to difficulties in applying normalization techniques like RMSNorm during inference, which can result in unstable outputs. The solution, termed "QK-Clip," proposed by Kimi's researchers, addresses this issue by clipping specific weights during training, ensuring stable performance during inference without compromising efficiency. This innovation is significant for developers using LLMs in generative AI applications, as it enhances model reliability and performance in memory-constrained environments.

Sentient, a decentralized AI company, partnered with infrastructure provider Fireworks to handle the rapid scaling and high concurrency demands of their viral AI products. Sentient, backed by significant venture funding, aims to democratize AI through blockchain and open-source models, with notable products including Sentient Chat and Open Deep Search. These products quickly attracted massive user interest, leading to infrastructure challenges like concurrency bottlenecks and unpredictable traffic spikes. Fireworks addressed these challenges by offering high-performance infrastructure that provided up to 50% more throughput per GPU, enabling Sentient to scale efficiently without cost inflation. This partnership facilitated the rapid development and deployment of Sentient's AI models, including Dobby-70B, ensuring consistent performance and user satisfaction even under extreme loads. The collaboration enabled Sentient to achieve significant milestones, such as a viral launch with over 1.8 million waitlisted users and stable system performance during high-demand periods, ultimately positioning Sentient as a competitive challenger to established AI platforms like ChatGPT.

The blog post explores MuonClip, an advanced optimization technique designed to address the issue of attention score explosions in large-scale transformer training, particularly within the Kimi-K2 model. As transformers scale to handle billions of parameters and trillions of tokens, traditional optimizers like AdamW struggle with stability, leading to problems such as NaNs and gradient issues. MuonClip enhances the Muon optimizer by implementing a mechanism called qk-clip, which resizes query and key weights (W_q and W_k) to prevent attention scores from becoming excessively large, thereby maintaining stability during training. This method retains the efficiency and balance of Muon's updates while ensuring that attention scores remain within a manageable range, effectively preventing training crashes. Interactive visualizations and examples are provided to illustrate how MuonClip operates in practice, highlighting its ability to democratize large-model training by making it more accessible to startups, thus enabling them to manage big datasets without the computational pitfalls associated with traditional methods.

Fireworks AI has introduced Amazon SageMaker as a Bring Your Own Compute (BYOC) deployment option, enhancing its platform for enterprise AI teams by enabling seamless model training within SageMaker and leveraging Fireworks’ efficient inference system. This integration allows developers to use SageMaker for model training and experimentation while benefiting from Fireworks' high-performance, low-latency model serving within their existing AWS environment. The platform supports various model types, including LLMs, VLMs, and other multimedia models, offering fast response times, cost efficiency, and comprehensive AWS resource governance. The initiative is designed to meet enterprise compliance and security requirements, providing a scalable, reliable AI infrastructure that retains data governance and integrates with AWS billing systems. Currently available in private preview, this launch aims to make advanced AI infrastructure more accessible and scalable, bridging the gap between model development and production-grade inference.

Function calling is transforming large language models (LLMs) from static knowledge generators into dynamic, interactive agents capable of real-world tasks by enabling them to detect when a user request requires external data or action and produce a structured output specifying which function to call. This capability allows LLMs to integrate real-time data, such as weather information, into their responses by triggering external APIs and updating the conversation context with the retrieved information. The Model Context Protocol (MCP) further enhances function calling by standardizing the connection between AI models and data sources, simplifying integration, and establishing a framework for AI interoperability. This advancement is paving the way for agentic AI systems that can plan multi-step workflows, adapt their decisions based on intermediate results, and maintain contextual memory across interactions. As function calling evolves, it holds the potential to enable autonomous agents that collaborate with humans to tackle complex challenges, marking a significant shift in the landscape of human-AI interaction.

Leveraging a large language model (LLM) as a judge can significantly enhance the performance of policy models in domains that are challenging to quantify, such as creative writing. Using the Fireworks Reinforcement Fine Tuning (RFT) API, the Qwen2.5 32B base model was fine-tuned to achieve a 93.8% win rate on creative writing tasks against its original version. This process involved using the open-source Qwen3 235B model as a judge, which was managed with the Fireworks Build SDK to automatically allocate optimal compute resources. The evaluation methodology employed pairwise comparisons of different rollouts of the same prompt to assign rewards, using a rule-based reward function to assess dimensions like style and coherence. The Arena Hard Auto dataset, which includes creative writing, mathematics, and software engineering tasks, served as the testing ground, and the RFT methodology also showed improvements in more objective domains like mathematics and programming. The study highlights the potential of using LLMs for nuanced evaluation in creative tasks, offering significant improvements over base models.

FLUX.1 Kontext, developed by Black Forest Labs, is a suite of generative flow matching models designed to enhance image creation and editing by integrating text and image prompts. Available through the Fireworks API, FLUX.1 Kontext allows for in-context image generation, enabling users to seamlessly extract and modify visual concepts in images while preserving unique styles and character consistency across different scenes. The suite includes FLUX.1 Kontext Pro, which supports local editing and full-scene transformations, and FLUX.1 Kontext Max, an experimental model that enhances prompt adherence and typography generation. Users can explore these capabilities through the Fireworks API, a UI playground, and a Discord community, offering a collaborative environment for developers to experiment and innovate with these advanced image models.