Counting Objects

Over the past two years, GitHub's Systems Team has been focused on improving the performance of Git operations, such as cloning and fetching, by addressing inefficiencies in object counting. Traditionally, Git operations involved a time-consuming "Counting Objects" phase due to the need to track the Directed Acyclic Graph structure of repositories. To tackle this, GitHub explored caching strategies but found them impractical due to the dynamic nature of active repositories and high storage costs. Instead, they adopted a bitmap index approach, inspired by Google’s JGit engineers, which significantly reduced CPU time for network operations by marking reachable objects without traversing the entire graph. Although initial implementations faced challenges, such as slower compression due to Git's aggressive delta storage strategy, GitHub developed a heuristic that optimized the way objects were delta’ed, leading to a dramatic performance boost. This innovation, which reduced average CPU time for Git operations by over 90%, was eventually upstreamed in Git 2.0, benefiting the broader Git community and saving users significant time in network operations.

No tracked trend matches for this post yet.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.