The road to better completions: Building a faster, smarter GitHub Copilot with a new custom model

GitHub Copilot's code completion feature has undergone significant enhancements aimed at improving the overall developer experience by providing faster, more relevant, and higher-quality suggestions. The team behind Copilot has refined their custom models based on developer feedback, resulting in a 20% increase in accepted and retained characters, a 12% higher acceptance rate, and a 3x increase in token throughput with a 35% reduction in latency. These improvements are supported by a comprehensive evaluation process involving offline, pre-production, and production evaluations to ensure the model aligns with real developer workflows. The model training process includes mid-training on a curated corpus of modern code, followed by supervised fine-tuning and reinforcement learning, which have been optimized for accuracy, relevance, and helpfulness. This approach addresses challenges such as cursor misalignment and formatting fidelity, leading to better fill-in-the-middle (FIM) performance. Lessons learned emphasize the importance of aligning metrics with real-world impact, avoiding over-optimization, and refining training data to improve real-world outcomes. Looking ahead, the team plans to expand into domain-specific areas and refine reward functions to further enhance the precision and utility of completions.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.