November 2021 Summaries

Aspect is actively enhancing the Bazel ecosystem by addressing the tool's limitations and improving user experience through several initiatives. They have launched a Bazel rules documentation site that centralizes, searches, and versions documentation for various rulesets, making it easier for users to find and link specific attributes. Aspect is also investing in automating BUILD file maintenance with Gazelle, reducing the need for engineers to learn build systems. They are contributing to the maintenance of key rulesets and have initiated a Special Interest Group (SIG) for rules authors to foster collaboration and resource sharing among corporate contributors. Additionally, Aspect has developed a new command-line interface, aspect, to simplify Bazel's complex error messages and terminology, offering a more user-friendly experience for engineers who are not build system experts.

Bazel's built-in downloader plays a crucial role in managing network fetches but operates separately from the repository cache, ensuring that even if some repository rules are re-run, fetching from the internet is unnecessary. While configurable, this downloader's documentation is limited, and some repository rules that run external programs like yarn install are not aware of it, conducting their own network fetches. This can introduce non-hermeticity and security vulnerabilities when dealing with untrusted third-party code. To mitigate these risks, using a local read-through cache, such as Artifactory, is recommended for enhanced resilience against network outages and improved security with a scanner to identify vulnerabilities. Redirecting all downloads through this cache on continuous integration (CI) systems is advisable, using a configuration file that defines URL rewrites to ensure security and efficiency by prioritizing matches and directing fetches to a trusted source.

Continuous Deployment (CD) with Bazel presents unique challenges, particularly in the release of artifacts built by Bazel, due to its limited awareness of version control systems like Git. Stamping in Bazel allows embedding of metadata, such as git SHAs, into binaries to facilitate monitoring, although the process is complex and can lead to inefficiencies like cache misses when stable status values change. The document outlines a method for identifying releasable artifacts through Bazel rules, emphasizing the importance of deterministic builds by avoiding stamps in CI builds and ensuring artifacts reflect only relevant version changes. It highlights the use of a key/value store like Redis to manage artifact versions efficiently, ensuring only new or changed artifacts are released. The text notes the challenges of consistent stamping due to varied rule implementations within Bazel, and suggests using Aspect Workflows, an integrated solution developed since the writing of the post, to streamline the CI/CD process with detailed guidance available in the Bazel Delivery Guide.

Bazel, an open-source build tool developed by Google, is often misunderstood due to internal idioms that are inadvertently associated with it, leading to misconceptions such as the belief that Bazel cannot write to the source folder. While intermediate outputs in Bazel are typically used as inputs for other build targets and stored in the 'bazel-out' directory, there are scenarios where outputs need to be placed in the source folder, such as for editor plugins, test files, or documentation. Although standard commands like 'bazel build' and 'bazel test' cannot modify the source tree, 'bazel run' can, as it sets the environment variable BUILD_WORKSPACE_DIRECTORY, enabling source updates. This leads to the "Write to Sources" pattern, where 'bazel run' updates files and 'bazel test' ensures they don't drift from Bazel-generated outputs, although it may cause issues if the source files are outdated before tests are run. Existing rules like Aspect's 'write_source_files' or Chuck Grindel's 'updatesrc' can facilitate this process, while custom solutions can be implemented in a BUILD.bazel file to maintain synchrony between generated and source files, with many core Bazel rulesets adopting this approach to manage API markdown files.