Why and How to Run Local Models in Zed

Local models offer several advantages over cloud-hosted models, such as enhanced privacy, cost-effectiveness, and control over the system, making them attractive to developers who prefer to keep their data on personal hardware and avoid dependencies on SaaS platforms. Despite the benefits, local models are generally less powerful than cutting-edge cloud-based solutions and may require significant hardware resources to run efficiently. Tools like LM Studio, Ollama, and llama.cpp facilitate the use of local models, and Zed supports these tools, allowing developers to experiment with different configurations and models, such as the Qwen 3.6 35B A3B model, which employs a "Mixture of Experts" approach for faster performance. While local models demand more discipline and attention to be used effectively, especially due to smaller context windows and lower "cleverness," they offer a reliable and customizable alternative for those wary of changing cloud provider conditions.