February 2023 Summaries

The term "reproducible build" refers to a build process that produces identical results on every run, allowing for reliable connection between source code and build artifacts, verification of build outputs, and improved software supply chain security. Creating reproducible builds is challenging due to the large number of variables involved in the usual build process, including host operating system, timezone, environment configuration, compiler, and file timestamps. Dagger, a tool that executes pipelines entirely as standard OCI containers, aims to make reproducible builds easier to implement by isolating the build environment from the host filesystem and environment, reducing build variance, and providing mechanisms to audit the build environment. Additionally, Dagger addresses part of the biggest source of reproducibility issues related to file timestamps, allowing for deterministic build paths and eliminating variation in the build process. The future development of Dagger may include support for the SOURCE_DATE_EPOCH variable, disabling internet access during container execution, verifying files via checksums, fine-grained source policies, and tools that identify non-deterministic aspects of builds.

The author has adopted Ruff, a fast and feature-rich Python linter, for their Dagger Python SDK. The switch was made due to Ruff's growing popularity among popular open-source projects like Pandas, FastAPI, Pydantic, Sphinx, and Hatch. Ruff offers several benefits, including speed, reduced dependencies, and improved code quality. The author notes that Ruff is built in Rust, which contributes to its speed, and has near feature parity with existing tools like flake8 and isort. Adopting Ruff has resulted in a significant reduction in linting time, from over 3 seconds to approximately 0.03 seconds per 6500 lines of code. The author appreciates the fast execution and reduced dependencies brought by Ruff, while also recognizing that it limits contribution by Python enthusiasts. The future steps include reviewing updates on a weekly schedule and exploring the possibility of using Ruff for both linting and formatting code.

Dagger 0.3.10 has introduced a new pipeline API that enables developers and operators to programmatically group related operations together for status reporting and visualization purposes, improving visibility of specific operations or events. The release also includes support for container metadata management via the Dockerfile LABEL instruction, allowing developers to add and manage labels for containers. Additionally, all SDKs now have an improved Build() method that allows users to build a particular target in a multi-stage Dockerfile, and the Python SDK provides more verbose error messages when a Dagger session connection cannot be established. The release also includes new documentation and tutorials, including an end-to-end tutorial on using Dagger with GitHub Actions.