GitHub Availability Report: March 2024
Blog post from GitHub
In March, GitHub experienced two service degradation incidents impacting various services. The first incident on March 15 lasted 42 minutes and was caused by a regression in the permissions system following a framework upgrade that introduced incompatible MySQL query syntax with the database proxy service. GitHub addressed this by rolling back the deployment and fixing misconfigurations in development and CI environments. The second incident on March 11 lasted over two hours due to a network configuration error deployed to the wrong environment, affecting multiple services like API requests and GitHub Copilot. Although a rollback was initiated quickly, a failure in one data center prolonged the impact for some users until manual corrections were made. GitHub has since implemented measures for safer configuration changes and faster issue detection to prevent future occurrences. For ongoing updates and insights, users are encouraged to follow the GitHub status page and Engineering Blog.