Git compresses files by storing only the differences between versions using a rolling hash method and heuristics to pick files for compression. The rolling hash works by comparing chunks of files and finding similarities, while the heuristics consider factors such as file name similarity, growth over time, recent versions being accessed more frequently, and keeping deltas shallow. These insights help Git compress files efficiently, making it a subtle yet interesting topic that can be applicable to other caching systems.