May 2024 Summaries

Ty Dunn, Co-founder of Continue, discusses how to effectively set up and use the Continue coding assistant alongside Ollama, both of which can be integrated into Visual Studio Code and JetBrains using open-source large language models (LLMs). The post outlines the process of installing Continue and Ollama on various operating systems, recommending the use of different models such as Mistral AI’s Codestral 22B, DeepSeek Coder 6.7B, and Llama 3 8B for tasks like autocomplete and chat, depending on the user's VRAM capacity. The article also explores utilizing nomic-embed-text embeddings with Ollama for efficient codebase querying, fine-tuning StarCoder 2 on development data for better suggestions, and leveraging the @docs feature to access documentation for further learning. Dunn encourages users to join the Continue or Ollama Discord channels for support and to enhance their understanding and usage of these tools.

At Google IO 2024, Google introduced Firebase Genkit, an open-source framework designed to help developers build, deploy, and monitor AI-powered applications, now featuring support for running Google's open-source Gemma model locally through Ollama. Compatible with MacOS, Windows, Linux, and Docker containers, Firebase Genkit simplifies the development process by enabling integration with Node.js projects and providing installation via npm. This framework facilitates the creation of production-ready applications by allowing developers to easily initialize and run Genkit on a local server. Additional resources and tools, such as the Ollama plugin and optimization for NVIDIA RTX GPUs, are available to enhance the functionality and performance of applications built with Firebase Genkit.