April 2025 Summaries

Bazel, an open-source build system developed by Google, is particularly favored for its speed, correctness, and reproducibility, and is used by many large engineering teams for constructing multi-language builds. This guide offers a practical walkthrough for integrating Bazel into Python projects, showcasing the setup and configuration of a basic Python app and library using Bazel's MODULE.bazel files. It highlights Bazel's use of Starlark for defining builds and its action graph for mapping dependencies, ensuring efficient and hermetic builds. The guide explains the process of creating Python build targets using BUILD.bazel files, covering definitions for libraries, executables, and tests, and demonstrates Bazel's capabilities in managing dependencies, including Python versions and third-party libraries through extensions like rules_python. Additionally, the document touches on the advantages of using Bazelisk for managing Bazel installations and emphasizes the benefits of using Buildkite in conjunction with Bazel for advanced build and deployment workflows, offering tools for uploading libraries, identifying flaky tests, and more. The guide encourages further exploration of Bazel's functionalities and its integration with Buildkite for enhanced project management and CI/CD processes.

The text discusses the gradual decline in the effectiveness of continuous integration (CI) pipelines, describing how initially efficient systems can become slow and unreliable over time, leading to a phenomenon known as "CI apathy." This apathy stems from the normalization of dysfunction, where slow builds and flaky tests become accepted as the status quo, impacting team dynamics, innovation, and overall productivity. The narrative highlights the consequences of this complacency, such as reduced creativity, compromised quality, and talent attrition, while emphasizing the competitive disadvantage it creates. The text suggests that addressing CI apathy requires both technical improvements and cultural shifts within an organization, advocating for a "Delivery First" mindset that prioritizes software delivery as a pivotal aspect of engineering culture. It encourages teams to recognize and address CI inefficiencies, celebrate improvements, and transform their engineering practices to enhance delivery speed and reliability, citing examples of successful transformations by companies like Uber and Rippling.

Bazel builds can be significantly accelerated by using remote caching, which reuses build outputs and reduces build times on CI runners. This is facilitated by Bazel’s remote cache, which operates via simple HTTP requests and stores two main types of data: action cache metadata and a content-addressable store for output files. The blog post outlines how to deploy a self-hosted Bazel remote cache using bazel-remote, a Go-based service that supports writing to various storage backends, including local disks and cloud storage services like Amazon S3 and Google Cloud Storage. The post details how to deploy bazel-remote on AWS using infrastructure-as-code tools like Terraform and Pulumi, which set up necessary AWS components, such as ECS clusters and load balancers, and emphasizes using HTTPS and compression to optimize performance. It also explores advanced configurations, like partitioned and tiered caching, and highlights the advantages of using cloud object storage for hosting the cache. The post concludes by suggesting Buildkite as a suitable platform for managing Bazel's scale and complexity, noting that the Bazel team itself uses Buildkite for its CI system.

Bazel projects typically start with the command `bazel build //...`, which builds everything in the workspace, but this approach can become inefficient as projects scale. To enhance efficiency, selective builds are recommended, focusing only on the necessary targets for each commit. This hands-on guide demonstrates how to use Bazel with Buildkite to create a dynamic and adaptable CI pipeline that leverages Git and Python to determine which Bazel targets to build, based on changes in the repository. By integrating Bazel's querying capabilities with a Buildkite pipeline defined in Python, users can achieve more granular control over builds, dynamically generate pipelines based on code changes, and improve build visibility through detailed annotations. This approach moves away from static YAML configurations, offering a more flexible and efficient solution for managing complex build processes.