September 2023 Summaries

On August 10, 2023, HashiCorp changed its license from MPL to BUSL for their products, including Terraform. This change raises questions about the safety and long-term use of these tools under a BUSL license. Spacelift supports previous and current versions of Terraform without any change, but users may consider migrating to OpenTofu due to its advantages in continuity, stability, and progress. OpenTofu is currently 100% backward compatible with Terraform and has a growing community of support. Spacelift is one of the founding members of the OpenTofu initiative and supports its development.

YAML (YAML Ain't Markup Language) is a human-readable data serialization format that supports comments denoted by the # character. Single-line, inline, and block comments can be used to document intentions, explain values, and provide links to relevant material. Comments are useful for temporarily disabling parts of a YAML file or adding authorship and licensing details. Following best practices such as breaking long comments into multi-line sections, maintaining consistency in comment format, and using appropriate comment types can enhance the readability and maintainability of YAML files.

Terraform maps are data structures used to represent collections of key-value pairs in the HashiCorp Configuration Language (HCL). They are commonly used to store configuration data, define variables, and pass information between different parts of your infrastructure code. Maps can be defined with specific types such as string, number, bool, list, set, or object, each with distinct purposes and use cases. The main difference between maps and objects lies in their intended purpose and where they are typically used in Terraform providers. Maps are used to represent structured data with multiple attributes for each key, while objects are mainly used to manage resource attributes in Terraform providers. Terraform provides various functions such as toset(), flatten(), map(), and tomap() to convert between different data types and maps. These functions can be used to create, manipulate, and convert maps in your Terraform code. Additionally, Terraform offers Spacelift, a tool that makes it easy to work with Terraform by providing features such as policy as code, programmatic configuration, context sharing, drift detection, and more.

The OpenTofu initiative, an open-source IaC toolset under the Linux Foundation umbrella, is worth considering for organizations planning to adopt infrastructure as code (IaC). While BUSL licenses can be restrictive and vague, OpenTofu's MPL license ensures a company's usage without extensive legal checks. The initiative has strong community support, which drives its growth and direction. As CTO, embracing OpenTofu means joining a new community eager to grow fast and impact the future of IaC.

Terraform's `for_each` meta-argument allows for dynamic instance creation based on the elements of a map or set, making configurations more concise and maintainable. It offers benefits such as enhanced readability, consistency, simplified data source iteration, and managing dynamic workloads. However, it requires caution when dealing with indexing issues and potential re-creation of resources due to changes in the input variables. The `for_each` meta-argument can be used with various types of inputs, including sets of strings, maps of strings, lists of objects, and even Terraform data sources. It provides flexibility and scalability compared to traditional `count` approaches, but its unique characteristics require careful design and planning to accommodate the nuances of this approach. Additionally, Spacelift is a tool that makes it easy to work with Terraform, including automating deployments, managing AWS credentials, and building complex workflows using policy as code and programmatic configuration.

OpenTofu, a fork of Terraform, has been created by the initiative of OpenTF supported by various companies in response to HashiCorp's switch from an open-source license to BSL. OpenTofu is an open-source version of Terraform and offers improvements and enhancements while retaining all features that made Terraform popular among developers. It has been forked from Terraform version 1.5.6 and is now a part of the Linux Foundation, with the ultimate goal of joining the Cloud Native Computing Foundation (CNCF). OpenTofu's manifesto presents HashiCorp with two options: switching Terraform back to an open-source license or creating a fork of Terraform that will be placed in a foundation. The community behind OpenTofu initiative includes almost 144 companies, over 10 projects, and over 718 individuals.

OpenTofu, a fork of legacy MPL-licensed Terraform, has joined the Linux Foundation and will be going through the process of joining the Cloud Native Computing Foundation. OpenTofu is community-driven, impartial, and ensures that valuable features and fixes are accepted based on their value to the community, regardless of their impact on any particular vendor. By joining the Linux Foundation, OpenTofu remains truly open source and neutral, free from the influence of any single company. This move provides a dependable path for users to maintain the status quo without worrying about BSL complexities or potential alterations in Terraform's license terms.

Crossplane and Terraform are both popular Infrastructure as Code (IaC) tools used for infrastructure provisioning and management. While they share the same objective, their features and design principles differ. Crossplane is an open-source tool that extends Kubernetes to manage cloud infrastructure via its API and kubectl command. It offers seamless integration with Kubernetes, allowing infrastructure deployments to be composed in the same way as application deployments. Terraform, on the other hand, is a standalone IaC tool developed by HashiCorp that uses a declarative configuration language called HCL. Terraform has a large ecosystem supporting over 3k providers and offers fine-grained control over infrastructure creation and management. It also has strong community support and extensive resources and documentation. Crossplane, being relatively newer, has a smaller but growing community and is still at the maturity level of an incubating project. The choice between Terraform and Crossplane depends on various factors such as scope, complexity, integration, scalability, maturity, community support, cost, and licensing model. For teams heavily invested in Kubernetes or those looking for a unified approach to manage both their infrastructure and applications, Crossplane could be the better choice. However, for those seeking a mature tool with strong community support and extensive resources, Terraform is recommended. In addition to these tools, using an IaC orchestrator like Spacelift can greatly enhance your workflow by effectively managing Terraform state, supporting policy as code, programmatic configuration, context sharing, drift detection, resource visualization, and more.

Docker Compose is a tool that simplifies the process of creating and running multi-container applications by automating the management of multiple Docker containers simultaneously. It allows users to define their application's containers as code inside a YAML file, which can be committed to a source repository. This makes it quicker, easier, and more repeatable than manually starting and linking containers. Compose also automatically creates a Docker network for the project and manages storage volumes. It is particularly useful for applications with multiple services that have dependency relationships or require specific configurations before deployment.

This article discusses the use of GitHub Actions to manage infrastructure as code (IaC) using Terraform, highlighting its advantages and limitations. It provides a step-by-step guide on how to create an infrastructure pipeline with GitHub Actions, including pull request integration and delivery to AWS. The author also compares the use of Spacelift for IaC management over generic CI/CD tools like Jenkins. While GitHub Actions is praised for its deep integration with GitHub, flexibility, and wide community support, it has limitations when managing large-scale infrastructure projects. Spacelift, on the other hand, offers features specifically designed for IaC management, such as native state file management, stack dependencies, variable management, built-in drift detection, and resource visibility. The author concludes by recommending Spacelift for organizations seeking optimal orchestration of their infrastructure due to its granular control over access and executions, time-saving features, and dedicated IaC focus.

A .gitignore file is a text file used by Git to specify files and directories that should be ignored and not tracked by the version control system (VCS). This article provides examples of how to use a .gitignore file with Terraform projects. The steps include creating a new .gitignore file, running git init and terraform init commands, and adding patterns to ignore local Terraform directories, files, state files, log files, sensitive data, and more. Using a .gitignore file can help maintain security and efficiency in managing source code with Git.

This article discusses the creation of an IAM (Identity and Access Management) role in AWS (Amazon Web Services) using Terraform. An IAM role is a security principal that enables delegation of permissions to AWS resources within an account without sharing long-term credentials such as access keys or passwords. The roles are typically used for granting permissions to AWS services, applications, or other accounts. The article provides a step-by-step guide on how to create an IAM role and assign it to a newly created EC2 instance that allows read-only access to a newly created S3 storage bucket. It also covers the configuration of the AWS provider, obtaining Access key ID and secret access key for your AWS user, running 'aws configure', creating a new main.tf file, setting the IAM role, and more. The guide concludes with steps on how to validate, apply, and clean up the Terraform configuration. It also encourages readers to explore Spacelift as a tool for managing Terraform infrastructure and working with AWS credentials per run.

This article discusses the use of three different tools - GitHub Actions, Azure DevOps, and Jenkins - to create CI/CD pipelines solely for deploying Terraform code. It explores how these pipelines can be constructed and their limitations. The value of a specialized IaC tool like Terraform is highlighted, focusing on its flexibility and support from the community. The article then delves into designing the CI and CD sections of the pipeline using each tool, emphasizing the challenges faced with generic CI/CD tools when managing infrastructure at scale. Finally, it introduces Spacelift as a solution to these limitations, offering features such as state file management, orchestration options, policy enforcement, and more.

The -target option in Terraform is used to target a specific resource for action instead of performing the action on all resources. It can be used with the terraform plan, apply, or destroy commands. This option is useful when you want to apply changes to a single or multiple resources and ignore all other resources. However, it should be used with caution as it can create resource dependencies and cause unexpected changes to your infrastructure. Terraform has built-in dependency mapping that should be taken advantage of wherever possible.

The OpenTF repository has been made public, with over 33k stars received in less than a month. Companies like Spacelift have pledged to support the project by covering costs of full-time engineers. Contributions are welcomed through opening issues or preparing RFCs for major changes. Code contributions can be done by forking the repository and creating pull requests, with all commits needing to be signed. OpenTF's in-progress items include deciding on a name, creating an initial registry, and working towards its first stable release as a drop-in replacement for Terraform. Support can also be shown through joining OpenTF's Slack or Reddit communities.

StatefulSets and Deployments are two Kubernetes API objects used to manage sets of identical Pods. While both can orchestrate multiple Pod replicas, they have different features suited for separate use cases. Deployments offer declarative configuration to automate Pod updates and scaling operations, whereas StatefulSets include additional features that help you orchestrate stateful workloads in your cluster. The difference between StatefulSets and Deployments reflects the divide between stateful and stateless systems. StatefulSets are designed to run your app's stateful components, while Deployments are used for stateless ones. Stateless applications don't need to store any data within themselves, whereas stateful applications require persistent storage that outlives the lifecycle of individual container replicas. StatefulSets solve the challenges of running stateful services in Kubernetes by creating a set of identically configured Pods from a spec you supply, but each Pod is assigned a non-interchangeable identity. They also facilitate graceful scaling operations and rolling updates, where Pods are added and removed in a predictable order. Deployments use a declarative management model to automate Pod updates and support rollouts and rollbacks. They allow you to change the number of Pod replicas by adjusting the value in your Deployment's manifest, ensuring that Kubernetes will automatically add or remove Pods as required. The decision whether to use a StatefulSet or a Deployment should be based on the characteristics of the service you're deploying in your cluster. In general, choose a StatefulSet for stateful applications requiring stable, persistent storage and non-interchangeable Pods with specific roles. Select a Deployment when your service is stateless, doesn't require persistent storage, and won't be affected by changes to Pod identities.

Spacelift is a comprehensive CI/CD platform designed specifically for infrastructure-as-code, offering deep insight, full control, and unlimited flexibility. It supports various technologies such as Ansible, AWS CDK, Kubernetes, Terraform, and Terragrunt. Users can customize the execution environment by bringing their own Docker image or using hooks to download necessary executables. Spacelift aims to be a flexible alternative to homegrown solutions on top of generic CI tools, addressing common state management issues and offering additional infrastructure management capabilities.