March 2024 Summaries

The blog post discusses the construction and optimization of a Retrieval Augmented Generation (RAG) application using MongoDB Atlas and Fireworks AI, aimed at improving the development of Generative AI applications. RAG combines retrieval and generative components to enhance Large Language Models (LLMs) by allowing them to access and utilize up-to-date information from a data store, making them more efficient and flexible compared to traditional AI models. The guide illustrates building a movie recommendation system using MongoDB Atlas for indexing and vector search, and Fireworks AI for embedding generation and recommendation. It emphasizes the benefits of RAG architectures, such as data efficiency and ease of updating knowledge bases, while also providing insights on optimizing architecture for cost reduction, improved throughput, and enhanced scalability. The blog concludes with an introduction to more advanced RAG optimization techniques, including storage cost reduction and dynamic function calling, to further tailor the architecture to specific needs.

Fireworks has introduced a new fine-tuning service designed to enhance model accuracy and deployment speed by leveraging the LoRA technique, which allows for improved performance without extensive data or speed reductions. This service, which includes a competitive pricing structure starting at $2 per million training examples for models like Mixtral, enables users to fine-tune, deploy, and iterate on models rapidly, offering seamless integration with Fireworks' serverless inference platform at no extra cost. Fireworks' platform supports up to 100 fine-tuned models ready for immediate use, facilitating quick comparisons and live service integration while maintaining fast inference speeds, with rates for training and deployment lower than those of competitors. Users can easily initiate the fine-tuning process using the "Firectl" command line interface and manage settings such as epochs and learning rate, with future enhancements planned to support conversational formats and function calling.

Since its inception in 2022, Fireworks has emerged as a leading platform for serving generative AI models, initially catering to high-volume enterprise clients but now expanding its offerings to a broader audience of developers and businesses. The platform is rolling out key updates to enhance scalability and flexibility, including dedicated deployments that allow users to run models on private GPUs with reduced costs, improved speeds, and flexibility in model and hardware configurations. For those continuing with serverless models, Fireworks has optimized speeds and pricing, introducing a simpler, more competitive flat rate for token usage and increasing model rate limits to support higher production demands. Additionally, the platform is transitioning to a post-paid billing system to alleviate user concerns about credit management. The introduction of a new Business tier aims to bridge the gap for startups and developers scaling their AI model usage, providing custom support and features tailored to their needs. Fireworks is committed to democratizing AI access and invites feedback from its community to refine these offerings further.