May 2022 Summaries

Matthew Casperson's guide from May 31, 2022, provides a detailed walkthrough on deploying an Octopus Worker onto a new EC2 instance using AWS CloudFormation. The process involves using a CloudFormation template to set up an EC2 instance within a Virtual Private Cloud (VPC), which includes specifying instance types and configuring network settings to ensure SSH access is restricted to a specific IP. The template installs an Octopus Tentacle as a Worker, enabling the delegation of deployment tasks from the Octopus Server to the EC2 instance. Casperson highlights the importance of configuring security groups to manage network traffic and the installation of necessary tools for Kubernetes deployments. Additionally, the guide addresses setting up a Polling Tentacle to facilitate communication between the Worker and the Octopus Server, thereby easing configuration with firewalls. The end result is an EC2-hosted Worker ready to handle deployment tasks efficiently by operating in proximity to AWS resources.

The blog post by Matthew Casperson provides a guide to creating a CloudFormation parameter with a comprehensive list of EC2 instance types, which can be directly integrated into CloudFormation templates to streamline the deployment process. The article explains the need for the jq tool to execute a script that generates a YAML block, including all available instance types in a sorted list, making it easy to copy and paste into templates. By using this method, users can access a dropdown list of instance types when deploying through Octopus, eliminating the need to consult external documentation. The post aims to simplify the process of selecting EC2 instance types and offers additional resources in the Runbooks series for further learning about CloudFormation.

The DevOps Research and Assessment (DORA) metrics offer a standardized way to measure the performance of software development teams, focusing on four key areas: Deployment Frequency, Lead Time for Changes, Change Failure Rate, and Time to Restore Service. The text discusses how Octopus Deploy, in conjunction with GitHub Actions, can be used to calculate these metrics through a Python script that queries both Octopus and GitHub APIs. The script is designed to capture build information and track issues, enabling teams to assess their deployment performance and make data-driven improvements. Octopus Deploy version 2022.3 and later includes built-in support for DORA metrics, providing insights at both the project and space levels, which helps organizations make informed decisions regarding their DevOps practices. The document further details how these metrics can be calculated and interpreted, emphasizing their importance in benchmarking a team's performance against global standards.

AWS services, including Amazon Machine Images (AMIs), are region-specific, necessitating unique AMI IDs for each region, which complicates the creation of flexible CloudFormation scripts. To address this, the text outlines a method to generate a mapping of up-to-date regional AMI IDs using a Bash script, which can be incorporated into CloudFormation templates to maintain their relevance across different regions. This process involves using the `jq` tool to automate the retrieval and mapping of AMI IDs based on their regional availability and ensures that templates are not tied to a specific region. Additionally, the text suggests using AWS Systems Manager Parameter Store as an alternative method for automatically referencing the latest Amazon AMIs, thereby streamlining the management of CloudFormation templates. Overall, the article provides practical guidance for maintaining up-to-date CloudFormation templates in the ever-evolving AWS environment.

Bob Walker, the Technical Director of Customer Success at Octopus Deploy, shares insights into his role and the functions of the Customer Success team, highlighting how they facilitate customer success through relationship building, technical support, and license management. Walker, who transitioned from being an enthusiastic user of Octopus Deploy to joining the company in 2017 as one of the first US-based employees, emphasizes the importance of continuous learning and adapting to industry changes through a practice known as "sharpening time." This approach allows team members to explore new topics and technologies, such as Kubernetes and AWS, enhancing their ability to support customers effectively. The team measures success through a variety of indicators, including customer renewal rates, feature usage, and feedback, striving to ensure that customers not only renew their licenses but remain satisfied and successful with the product. Frequent updates and examples, like those related to the recent release of Config as Code, demonstrate their commitment to addressing evolving customer needs and enhancing documentation as part of their comprehensive customer support strategy.

Cloud computing offers the flexibility to scale resources on demand, and AWS's Instance Scheduler is a tool that efficiently manages this process by automatically shutting down and restarting EC2 and RDS instances, thereby reducing operational costs for resources not needed 24/7. This resource management is facilitated through the use of custom periods defined in a DynamoDB database and managed via a Python-based Scheduler CLI, which allows for the automation of these tasks through scripts. Users can tag resources with specific periods to control when they are active, such as shutting down overnight and on weekends, to optimize cost savings. The Instance Scheduler is deployed using a CloudFormation template within Octopus, and any potential errors in deployment, such as duplicate parameter values, are addressed with straightforward adjustments. The setup requires Python 3 and other tools installed on a Linux Worker, and it ensures idempotent deployments with the ability to update or create periods as necessary.

Jenkins, a popular open-source Continuous Integration platform, can be configured to automate code building and testing processes, and it integrates seamlessly with Octopus Deploy for managing releases and deployments. This guide details the steps for setting up Jenkins and linking it with Octopus Deploy to deploy a web application to Azure. The process involves configuring Jenkins, installing necessary plugins, generating API keys, and setting up Azure accounts and permissions. Users can create and manage deployment projects through Jenkins, leveraging the Octopus Deploy Jenkins plugin for creating releases and automating deployments. The integration facilitates efficient management of continuous integration and deployment workflows, showcasing the potential of combining Jenkins' automation capabilities with Octopus Deploy's release management functionalities.

In response to the critical Log4j vulnerability, many engineering teams faced the challenge of quickly identifying and addressing affected code bases to mitigate exposure. This involved understanding the application's structure and dependencies, often requiring a detailed examination of code bases at specific git commits. The article outlines a method using GitHub Actions to capture dependencies during the build process, storing them as artifacts for easy access and verification. By integrating this with Octopus Deploy, teams can track metadata, including links back to CI builds, and automate the process of querying dependencies through a custom Python script. This approach allows for rapid identification of vulnerabilities in deployed applications, enabling prompt responses to potential threats and reducing the pressure on support teams. The process highlights the importance of maintaining an organized and automated workflow to efficiently manage and respond to future vulnerabilities.

Runbooks are essential tools for capturing valuable business knowledge in an automated and testable format, enabling level 1 support teams to efficiently diagnose issues and handle customer requests in enterprise environments. These environments often feature custom applications with complex infrastructure, where understanding the system's design and functionality can be challenging. The blog post provides an example runbook for smoke testing a typical microservice application in AWS, detailing steps for DNS verification, MySQL database testing, HTTP endpoint status checking, and load testing using tools like dig, curl, and hey. By implementing these smoke tests, support teams can quickly identify potential problems in the application's underlying services and infrastructure, ensuring they respond to support requests effectively.

Matthew Casperson's article, published on May 10, 2022, explores the creation of a Virtual Private Cloud (VPC) on AWS using CloudFormation, focusing on a configuration that includes both public and private subnets. It explains that public subnets connect to the internet via an internet gateway, allowing resources with public IP addresses to be accessed externally, while private subnets do not connect directly to the internet and can only communicate within the VPC. The article provides a CloudFormation template to set up a VPC with one public and one private subnet, detailing how instances in the public subnet can be accessed via the internet gateway, and instances in the private subnet can access the internet through a NAT gateway, although they cannot be accessed from the internet. This setup offers flexibility in deploying instances that need public internet access or enhanced security by being shielded from public traffic.

Matthew Casperson's blog post, published on May 9, 2022, elaborates on creating an Amazon Web Services (AWS) Virtual Private Cloud (VPC) with public subnets using a CloudFormation template. It explains the differences between public and private subnets, highlighting that public subnets connect to the internet via an internet gateway, while private subnets do not. The post provides a step-by-step guide to building a VPC with two public subnets, detailing the necessary components such as the internet gateway, route tables, and their associations to direct traffic through the gateway. The process allows instances within the VPC to access the internet and receive public IP addresses for external communications. The post concludes by hinting at the next topic in the series, which will cover creating a VPC with both public and private subnets and a NAT Gateway to enable internet access for private subnet instances.

Virtual Private Clouds (VPCs) are essential for infrastructure on AWS, providing resource segregation and connectivity, but creating them can be complex. AWS offers two types of subnets within VPCs: public and private. Public subnets connect to the internet via an internet gateway and host resources with public IPs, while private subnets do not connect to the internet and only communicate with other subnets in the same VPC. The text provides a CloudFormation template to create a VPC with two private subnets, illustrating how to define subnets using AWS::EC2::Subnet resources with distinct CIDR blocks and placing them in different availability zones. The network connectivity between these subnets is managed by a route table, allowing seamless communication without additional routes, and the process of deploying this configuration is simplified through the use of CloudFormation templates, emphasizing the ease of creating VPCs with private subnets.

Octopus Runbooks can automate various infrastructure management tasks and provide alerts to support teams when issues arise. By configuring SMTP settings, users can set up automated email alerts that notify teams of problems, using simple runbooks like a 'Hello World!' script to trigger emails upon failure. For more complex issues, advanced runbooks can gather detailed information from sources like Kubernetes clusters using PowerShell scripts, allowing support teams to quickly identify and resolve deployment issues. These runbooks automate the extraction of deployment details, pod logs, and descriptions, embedding this information in email alerts to facilitate faster troubleshooting. By utilizing Octopus Runbooks, organizations can centralize key information dissemination, helping both technical and non-technical staff address issues efficiently.

Shadow IT in AWS refers to ad hoc resources created by DevOps teams that often lack proper management and documentation, leading to challenges in maintaining and securing infrastructure. These resources can include virtual machines, S3 buckets, and virtual private clouds, typically created through the AWS web console without standardized naming or tagging conventions. The lack of information about these resources, such as ownership, purpose, and application details, poses security risks and complicates troubleshooting and management. To address these issues, it is essential to implement a consistent tagging strategy that identifies the team responsible, the deployment project, and the environment associated with each resource. Additionally, using declarative templates like CloudFormation for resource creation helps manage infrastructure by allowing for easier recreation and detection of unauthorized changes. Scripts can be utilized to identify resources lacking the necessary tags or those not created by CloudFormation templates, thereby helping to manage and bring shadow IT resources under control. This approach not only enhances the visibility and accountability of resources but also facilitates the effective scaling of infrastructure.