September 2022 Summaries

KubeCrash is a fully virtual, free, and open-source event designed to provide hands-on learning opportunities in preparation for KubeCon + CloudNativeCon North America in Detroit, Michigan. Scheduled for October 5 and 6, the event is organized by seven leading open-source companies in the cloud-native space and offers sessions tailored for SREs, developers, platform engineers, and security engineers. Participants can engage live with project maintainers and learn about popular cloud-native tools such as Pulumi, Goldilocks, cert-manager, Linkerd, CockroachDB, and Polaris. The event aims to enhance skills in utilizing open-source tools, a critical aspect of modern tech stacks, with keynotes, workshops, and case studies spanning different time zones to accommodate a global audience.

Pulumi is a versatile infrastructure automation tool that integrates both imperative and declarative paradigms, utilizing imperative languages like Python, JavaScript, or C# for coding while the Pulumi engine processes these codes declaratively. This dual approach allows users to define their desired infrastructure state with imperative programming, which does not directly execute resource provisioning. Instead, the Pulumi CLI and deployment engine interpret these models, compute necessary actions, and ensure the actual infrastructure aligns with the intended state, leveraging a declarative engine that uses a directed acyclic graph (DAG) to manage resource dependencies and order of operations. Providers, which interact with cloud APIs, operate imperatively to execute the CRUD actions needed to align the infrastructure with the desired state. Pulumi's architecture, therefore, blends imperative coding for defining infrastructure with declarative execution to offer a flexible solution for infrastructure automation.

Pulumi's watch mode enhances infrastructure as code development by providing a hot reload-like experience, enabling rapid prototyping and iteration. Initially, it faced challenges due to its dependency on a Go-based file-watching library that struggled with cross-compilation, especially on Apple Silicon Macs. To address these issues, Pulumi transitioned to a Rust-based implementation, capitalizing on Rust's active maintenance and cross-platform capabilities, while eliminating the need for cgo and ensuring efficient performance across all supported platforms. This shift not only resolved the compilation issues but also maintained Pulumi's mission of leveraging the best programming languages and tools to optimize infrastructure development. The new implementation, known as pulumi-watch, reduces deployment iteration times and simplifies the developer workflow by automatically running deployments without requiring manual commands. This improvement demonstrates Pulumi's commitment to innovation in engineering practices, offering a more consistent and efficient user experience.

Pulumi's engineering team often creates and distributes tools as native binaries, primarily using Go for its support in building self-contained binaries without additional dependencies. For Node.js projects, Pulumi employs the vercel/pkg command line tool, which packages Node.js applications into executables that can run on devices without Node.js installed. The process involves setting up pkg by installing it via npm or yarn, specifying the entry point in the package.json file, and executing the pkg command with appropriate targets and output paths. Challenges encountered include dealing with the "Inspector Not Available" warning due to the Node.js Inspector API and issues with running MacOS ARM binaries, which were resolved by adjusting the creation of singleton instances and installing the 'ldid' tool. Additionally, the requirement for static binaries in Nixos environments was addressed by changing Linux targets to static versions, and multi-platform builds were managed using a makefile to automate the process for various operating systems. The use of vercel/pkg replaces a similar process with the nexe project, which is now considered unmaintained.

Creating a static website on AWS using an S3 bucket involves more than just uploading content; it requires associating a domain name and utilizing a CDN like Cloudfront for optimal performance. The AWS Static Website Package simplifies this process by enabling easy domain association and CDN setup with minimal configuration. However, updating content in place can cause issues such as unavailable assets, so the package supports versioning through atomic deployments, where a new S3 bucket is provisioned for each deployment to ensure all assets are present before associating it with the CDN. This method maintains user experience by avoiding deployment-time issues and allows for manual rollbacks by exporting the bucket name to preserve the previous version. Additionally, a Cloudfront Function can be used to update the cache by adding a unique build identifier to the CDN's cache key, ensuring it serves the latest content. The package ultimately streamlines the deployment and management of static websites, allowing users to focus on content creation rather than technical details, with more information available in the documentation or the GitHub repository.

Pulumi has introduced a range of updates across its platform, enhancing its cloud infrastructure management capabilities. These updates include the addition of 145 new resources across AWS, Azure, and Google Cloud providers, improvements to Pulumi YAML and CLI functionalities, and expanded Automation API features, such as parallel execution of Python inline programs and enhanced logging and tracing controls. The Pulumi Service has been upgraded with a new console and dashboard for better user experience, alongside the introduction of Team Access Tokens for managing role-based access controls and a new Billing Manager role for handling billing operations. Furthermore, the Stack Favorites feature has been launched to help users easily access frequently used stacks within the service.

Pulumi operates as a desired state engine, aligning the actual state of infrastructure with a user-defined desired state by making targeted changes. However, discrepancies can arise when Pulumi's understanding of the current state becomes misaligned due to factors such as out-of-band changes—where resources are modified outside Pulumi's control—and interruptions during operations. To address these issues, Pulumi provides the `pulumi refresh` command, which reconciles its state with the actual state of resources by checking with underlying providers like AWS. While the tool can automatically resolve most discrepancies for managed resources, it requires user input for pending CREATE operations, guiding users to either import existing resources or clear pending states to allow recreation. Additional command flags, such as `--import-pending-creates` and `--clear-pending-creates`, offer further customization to streamline these processes.