How we made Git fetch performance improvements in 2021, part 1

GitLab's Scalability team implemented a series of improvements to enhance server-side efficiency in handling Git fetch traffic, resulting in a 9x reduction in CPU utilization, primarily through the introduction of the Gitaly pack-objects cache. This cache has significantly reduced server load, particularly in managing highly concurrent CI pipelines, and while the changes are not visible to users, they enhance the stability and availability of GitLab.com. A prior solution, the CI pre-clone script, effectively minimized data transfer per CI job but required manual setup, lacked integration, and was prone to maintenance issues, leading to the development of the more robust pack-objects cache. A benchmark demonstrated that after enabling the pack-objects cache, server CPU utilization during a simulated CI pipeline decreased from 100% to 40%, and the benchmark run time improved from 27 seconds to 7.5 seconds. This advancement, incorporated in GitLab version 14.6, underscores a broader effort to optimize Git HTTP efficiency, which is central to GitLab CI operations, in comparison to Git SSH.

No tracked trend matches for this post yet.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.