February 2024 Summaries

Ollama's preview release on Windows allows users to pull, run, and create large language models with a native experience that includes built-in GPU acceleration, full access to the model library, and compatibility with the Ollama API and OpenAI. The software utilizes NVIDIA GPUs and modern CPU instruction sets for hardware acceleration, eliminating the need for configuration or virtualization. Users can run vision models, such as LLaVA 1.6, by simply dragging and dropping images into the interface. The always-on Ollama API runs in the background, enabling easy connectivity for tools and applications, and supports existing OpenAI-compatible tools, allowing integration with local models. The setup process involves downloading the installer, and users are encouraged to provide feedback or report issues through a Discord server.

Ollama has integrated compatibility with the OpenAI Chat Completions API, enabling users to leverage more tools and applications locally using Ollama. Users can start by downloading Ollama and pulling models like Llama 2 or Mistral, and then utilize the OpenAI API format with a local hostname for operations such as chat completions. This compatibility extends to various programming languages, including Python and JavaScript, facilitating the integration with libraries like the OpenAI Python library and Vercel AI SDK for building conversational applications. Additionally, Ollama can be used with Microsoft's open-source framework Autogen to build multi-agent applications, such as using the Code Llama model for coding tasks. Future enhancements may include support for Embeddings API, function calling, vision support, and log probabilities, with ongoing development and community feedback welcomed.

LLaVA 1.6 introduces an updated collection of Large Language-and-Vision Assistant models, now supporting higher image resolution with four times more pixels and enhanced text recognition and reasoning abilities, thanks to additional training on document, chart, and diagram datasets. The models, available in parameter sizes of 7B, 13B, and a new 34B, are distributed under more permissive licenses such as the Apache 2.0 license and the LLaMA 2 Community License. These models can be utilized through the Ollama CLI, Python, and JavaScript libraries, and the REST API, facilitating tasks like image description, object detection, and text recognition. The models' training details and benchmark results, comparing them to other leading models, are accessible on the LLaVA website.