October 2023 Summaries

Fireworks.ai reports that its inference platform has achieved SOC 2 Type II and HIPAA compliance, milestones intended to demonstrate its security controls and support customers developing generative AI products. The SOC 2 Type II audit involved third-party validation of controls related to safeguarding and processing customer data, while the company also states that it conducts routine third-party security reviews and penetration tests to identify and prioritize issues. HIPAA compliance positions the platform for healthcare and life sciences applications, including question answering, large-scale medical data analysis, personalized chatbots, automated patient conversations, and accessibility-focused services. Fireworks.ai says the compliance process was supported by Vanta and Advantage Partners and describes these certifications as part of an ongoing commitment to security, transparency, continuous improvement, and responsible AI infrastructure.

Fireworks.ai offers a high-performance LLM inference platform that significantly enhances code completion speed and quality, providing a valuable tool for developers seeking efficient AI-powered coding assistance. By integrating with Sourcegraph's Cody, the Fireworks platform has notably improved code autocomplete, doubling the Completion Acceptance Rate and halving the latency for both single and multi-line code completion. These improvements are achieved through advanced optimization techniques like multi/group query attention and PyTorch runtime optimization, resulting in latencies that are 3.5x to 7x lower than other open-source offerings. The platform is cost-effective, offering up to 120x lower serving costs, and supports developers with a free tier for easy access to its models, enabling a seamless and productive coding experience.

Fireworks.ai has partnered with LangChain to integrate its open-source models into the LangChain Prompt Playground, enhancing accessibility for developers by allowing them to use these models without an API key and free of charge, provided they are signed into the LangSmith platform. With the increasing popularity and improved performance of large language models (LLMs), this collaboration aims to streamline the process for developers to run, fine-tune, and share LLMs, offering cost-efficient solutions with optimized performance. The integration allows users to efficiently test and optimize prompts using models like Mistral 7B Instruct and Llama 2 13B, facilitating productivity in building LLM workflows. The LangChain Prompt Hub simplifies experimentation with various prompts, models, and parameters, contributing to a more efficient and iterative development process.