June 2022 Summaries

Building with Bazel can sometimes lead to caching issues, particularly when dealing with non-deterministic repository rules that involve package managers like pip, npm, or cargo, which might result in varied outputs due to "post install" scripts. This blog post explores how such inconsistencies, particularly with Python distributions, can lead to cache misses by affecting the build graph inputs. It highlights the importance of understanding repository rule determinism, where deterministic rules like http_archive are contrasted with non-deterministic ones due to their reliance on package managers that may execute additional processes during installation. The post provides a method to identify file differences by hashing and comparing dependencies' shasums over different executions, suggesting that solutions can range from excluding certain files from the build inputs to more complex fixes, which will be covered in subsequent posts in the series.

Aspect's rules_ts is an enhancement of the @bazel/typescript package, providing a ts_project rule that is layered on top of Aspect's high-performance rules_js, resulting in significant performance improvements for TypeScript builds. The benchmarks compare the build times of this new ts_project with its predecessor and other TypeScript tools like ts_library and the vanilla tsc compiler, using a large generated TypeScript codebase. The tests were conducted on a MacBook Pro and involved both full clean builds and "devserver" builds, which mimic typical development workflows by excluding real-time type-checking. Results show that while ts_library from Google leads in full clean build times due to its deep integration with the TypeScript compiler, ts_project from rules_ts offers a competitive performance with added worker mode support. In "devserver" builds, ts_project configured with swc as the transpiler demonstrates a significant speed advantage, offering nearly instantaneous build times by separating transpilation from type-checking. The benchmarks suggest that rules_ts, with its improved build times and efficient npm dependency management, can enhance developer productivity and encourage more web developers to adopt Bazel for their JavaScript projects.

Bazel, a build and test tool, often confuses users about when to use custom rules, macros, or genrules due to differing practices, especially among those familiar with Google's internal system. The text emphasizes the practicality of using genrules, which allow developers to specify commands, inputs, outputs, and dependencies in a straightforward manner within BUILD files, without delving into complex Bazel internals. Despite genrules' limitations, enhancements like the run_binary from bazel-skylib and Aspect bazel-lib offer improvements, such as compatibility with Windows, better handling of output directories, and custom progress messages. Furthermore, rules from package managers like rules_python and rules_js can automate the creation of genrules for third-party developer tools, simplifying the process. While macros can help streamline the use of these tools in BUILD files, they can introduce complexities if not used carefully. Ultimately, the text suggests that writing custom Bazel rules is rarely necessary unless there's a need for advanced interoperability, performance optimization, or platform-specific toolchain management.

rules_js is a Bazel rule set designed for efficient management of npm dependencies, emphasizing performance through fast fetching and linking. It is compared to other npm tools like yarn, npm, pnpm, and existing Bazel rules like rules_nodejs, using a benchmark based on a large Node.js monorepo. The benchmark evaluates both the initial fetching from the internet and subsequent runs using cached packages. While pnpm is noted for its fast lockfile generation due to a content-addressable store, rules_js introduces incremental and lazy fetching and linking capabilities, which can significantly reduce build times in common workflows. Unlike rules_nodejs, which doesn't support incremental linking, rules_js leverages pnpm lockfiles to enable more granular linking of node_modules, enhancing performance for large projects and monorepos. This capability is particularly beneficial for development environments where frequent changes to package.json files occur. Furthermore, rules_js plans to incorporate support for npm and yarn post-1.0, aiming to ease migration from rules_nodejs while maintaining the benefits of incremental and lazy linking.

Aspect has been developing support for the Bazel build tool within the frontend/JS ecosystem over the past five years, based initially on the internal workings of Node.js at Google. However, the initial approach faced several challenges, such as slow package manager installs, separate source and output folders complicating Node.js tooling, lack of workspace support, and performance issues related to npm package handling. To address these, Aspect introduced rules_js, which simplifies the build process by potentially needing to install only a single npm package in specific scenarios. The first beta release, v1.0.0-beta.0, includes rulesets for popular tools like TypeScript and Webpack, and Aspect encourages prototyping with these resources while providing a migration guide. They are preparing for a release of the final version and are actively engaging with the community and partners, such as Adobe and contributors from Rush.js and Google, to refine and enhance their offerings.