The road to better completions: Building a faster, smarter GitHub Copilot with a new custom model

GitHub Copilot's code completion feature has undergone significant enhancements aimed at improving the overall developer experience by providing faster, more relevant, and higher-quality suggestions. The team behind Copilot has refined their custom models based on developer feedback, resulting in a 20% increase in accepted and retained characters, a 12% higher acceptance rate, and a 3x increase in token throughput with a 35% reduction in latency. These improvements are supported by a comprehensive evaluation process involving offline, pre-production, and production evaluations to ensure the model aligns with real developer workflows. The model training process includes mid-training on a curated corpus of modern code, followed by supervised fine-tuning and reinforcement learning, which have been optimized for accuracy, relevance, and helpfulness. This approach addresses challenges such as cursor misalignment and formatting fidelity, leading to better fill-in-the-middle (FIM) performance. Lessons learned emphasize the importance of aligning metrics with real-world impact, avoiding over-optimization, and refining training data to improve real-world outcomes. Looking ahead, the team plans to expand into domain-specific areas and refine reward functions to further enhance the precision and utility of completions.