Build less, merge faster: avoiding diamond merges with a merge queue

Plaid's Developer Efficiency team faced challenges as their monolithic repository expanded to hundreds of commits per day, leading them to implement a merge queue to avoid costly diamond merges. Their initial attempt to build everything in the merge queue was impractical due to resource constraints and long processing times, prompting a shift to a more focused approach. By concentrating on the Go ecosystem, which constituted the majority of their codebase and was the primary source of diamond merges, they optimized the process by excluding unnecessary builds like container images and focusing on compile-only validation. This allowed for significant reductions in disk space usage and merge queue latency, stabilizing at an average of 1.8 minutes per merge. The team also contributed a bug fix to the rules_go project to improve compilation error detection, reinforcing their commitment to enhancing reliability and efficiency in their development workflows.