March 2022 Summaries

Bazel is a build tool that manages dependencies and fetches them as needed, ensuring consistency across developers' environments by downloading only what is required for specific build targets. However, "eager fetches" can occur, where unnecessary dependencies are downloaded during the analysis phase, leading to delays, especially on the first build or when repositories get invalidated. Eager fetches are problematic when they include slow install steps, and Bazel's repository cache only helps for certain network fetches but not for computations needed to install dependencies. To address eager fetches, users can refactor the WORKSPACE file, vendor pre-computed results, or defer repository rule work to a later action in the build graph. For BUILD file eager fetches, it's crucial to design the file graph efficiently and minimize the size of external repositories. Reporting issues to maintainers and creating CI tests outside of Bazel can help prevent regressions, as eager fetches are often unnoticed until they cause significant build delays.

Bazel, a build and test tool popularized by Google, presents challenges in understanding and integration for developers and DevInfra teams, but its customization capabilities offer significant benefits. The aspect CLI acts as an enhanced wrapper around Bazel, incorporating bazelisk and supporting plugins that address local developer workflow issues, such as customizing error messages, deploying, linting, and applying auto-fixes. These plugins can be utilized on Continuous Integration servers to streamline processes like reporting build failures and managing flakiness. The article introduces how to write a plugin using a template repository and Hashicorp's go-plugin library, with a focus on a sample plugin, HelloWorldPlugin, which demonstrates basic command integration and event handling. Additionally, the fix-visibility plugin exemplifies a practical application by addressing common visibility errors in Bazel, offering auto-fix suggestions interactively, and reducing the support burden on DevInfra teams. The article encourages the development of more plugins to enhance Bazel's usability and plans to create a plugin gallery for broader discovery and adoption.

Determinism in build systems ensures identical outputs for the same inputs, crucial for optimizing performance in tools like Bazel. Non-determinism, often introduced through npm packages during post-install steps by embedding non-stable elements like timestamps or absolute paths, can lead to inefficient builds, long CI times, and increased costs due to unnecessary rebuilds. This issue is prevalent as npm dependencies often serve as inputs for NodeJS targets, and different npm or yarn installs can result in divergent input hashes, preventing cache sharing across machines. Checking for non-determinism involves comparing outputs from separate runs of npm repository rules, which can reveal differences in generated files that affect build reproducibility. To resolve non-determinism, developers might remove unnecessary non-deterministic files post-install or patch npm dependencies. Regular checks on CI for non-determinism, even after fixing known issues, can prevent regressions and maintain efficient build processes.

Bazel aims to ensure repeatable and deterministic builds by declaring a hermetic build environment, where all tools used are specified with pinned versions, allowing consistent results across different machines and times. However, this consistency has been challenging for Python due to varying interpreter versions in different environments. The Aspect team has addressed this issue by introducing toolchain support in the latest 0.7.0 release of rules_python, enabling developers to use Bazel's toolchain system to discover and utilize the correct Python version automatically. The approach involves specifying the Python version and utilizing a workaround to load the resolved interpreter for repository rules, which execute before toolchain resolution. This advancement, facilitated by contributions from the community, is expected to enhance build reliability and consistency for Python projects using Bazel.

Writing developer documentation within source control is advocated as an effective method to integrate it seamlessly with code, ensuring versioning, easy accessibility, and simultaneous updates with related code changes. This approach allows for the use of familiar tools for reviewing and editing, such as diffing and bisecting, and leverages existing CODEOWNERS setups for appropriate review processes. However, presenting the documentation attractively is essential, with options like GitHub Pages or static rendering tools suggested for enhancing visibility. While this method offers several advantages, it might not be suitable when non-engineers require easy editing access, as they might find tools like git and Markdown cumbersome. Therefore, the presentation and editing ease must be optimized to encourage the maintenance and updating of documentation, and exceptions should be made for non-technical documentation.