January 2024 Summaries

FireLLaVA is the first commercially permissive open-source multi-modality model, based on the LLaVA framework, which has been released under the Llama 2 Community License. It allows for processing and analyzing data from multiple sources, such as text and images, enabling a more comprehensive understanding of input data. FireLLaVA is derived from LLaVA, a Vision-Language Model (VLM) that combines the Vicuna language model and the OpenAI CLIP-Vit vision component. The model was developed by Fireworks.ai, which recreated the LLaVA model using only open-source models for data generation and training, addressing licensing restrictions associated with the original LLaVA model trained on GPT4 data. This new version performs comparably to the original model and even surpasses it in some benchmarks. FireLLaVA is available for use through Huggingface, a fast API, and a playground, and it enables the development of vision-capable applications with APIs that are compatible with OpenAI Vision models.

Fireworks AI has introduced the Fireworks LLM serving stack, featuring FireAttention, which aims to serve open-source models four times faster than existing alternatives by employing quantization techniques such as FP16 and FP8 without significant quality tradeoffs. This initiative focuses on the Mixtral model, the first open-source model trained on trillions of tokens to support the 'mixture of experts' (MoE) framework. The platform demonstrates improved efficiency in serving MoE models, with particular emphasis on optimizing for long prompts and short token generation scenarios. Fireworks AI's FP8 implementation is highlighted for its ability to shrink model size and enhance deployment efficiency, surpassing existing integer quantization methods. The performance analysis reveals that Fireworks AI's FP8 implementation offers a superior trade-off between accuracy and performance compared to other frameworks like vLLM. Fireworks AI invites individuals interested in advancing AI system optimization to join their team as they continue to innovate in the field of foundation model optimization.