June 2020 Summaries

Bob Walker shares his experience of creating a PowerShell script to clone an Octopus Deploy project from a sample instance to Redgate's instance, emphasizing the limitations of the existing Octopus Data Migration tool and the Octopus Deploy API. The default migrator tool has several constraints, such as requiring database access and potentially overwriting custom changes, which prompted Walker to develop a script tailored to his specific needs. His script, shared on GitHub, allows selective cloning of project components and supports multiple runs without overwriting everything, focusing on simplicity and specific use cases. Walker's approach includes using dummy data for sensitive information, filtering data for cloning, and handling variables and deployment processes with care to avoid unnecessary overwrites. He highlights potential use cases for his script, such as syncing projects between different environments and breaking large spaces into smaller ones, and encourages others to fork and adapt the script for their requirements.

Octopus Deploy offers over 400 step templates, many of which are written in PowerShell, enabling a wide range of deployment tasks across different environments, including Linux. With the advent of PowerShell Core, these templates can now be used on Linux without needing to convert them to Bash. This guide demonstrates how to execute Octopus Deploy step templates on a Linux Worker using PowerShell Core, particularly within an Azure environment. By utilizing Azure Resource Manager (ARM) templates and Octopus Deploy's Runbooks, users can automate the creation and management of Linux VMs, including tasks such as installing and configuring the Octopus Tentacle for deployment purposes. The process includes setting up a resource group, converting cloud init scripts to Base64, and performing health checks on the newly created workers. The ability to use PowerShell Core on Linux significantly broadens the scope of templates available to Linux users, enhancing flexibility and functionality in deployment operations.

Amidst the COVID-19 pandemic, Shawn Sesna embarked on a project to create a Raspberry Pi cluster using Docker Swarm, driven by a desire to explore cluster computing and the new Raspberry Pi 4. The project aimed to shift some computing resources from a home hypervisor server to the energy-efficient Raspberry Pi cluster, while also deepening Sesna's understanding of Docker Swarm. The endeavor revealed several insights, such as the limited availability of ARM-compatible Docker images and the nuances of using Docker Swarm for service creation. Sesna also encountered challenges related to persistent storage, leading to the adoption of network storage solutions like CIFS for seamless container operations. Despite some troubleshooting, particularly with permissions and database compatibility issues, Sesna successfully configured a functioning system, including transitioning to a MySQL database to overcome SqlLite limitations. Overall, the project provided a platform for learning about Docker Swarm and Kubernetes, highlighting both the capabilities and constraints of running applications on a Raspberry Pi cluster.

Deploying .NET Core applications to a Raspberry Pi 3+ has become streamlined with Octopus Deploy, which offers various methods for integration, such as command line and web portal interfaces. This updated guide details the process of deploying .NET Core applications without Mono, leveraging the Octopus CLI tool to package and push applications to the Octopus Server. It highlights the importance of configuring your Raspberry Pi with the necessary environment settings, such as using Raspbian and ensuring the installation of .NET Core 2.0 or later. The guide outlines setting up the necessary infrastructure, including creating environments, accounts, and deployment targets within the Octopus platform, and emphasizes the importance of managing deployment processes for both applications and service definitions. By following these detailed steps, users can efficiently deploy and manage .NET Core applications on Raspberry Pi, enhancing the capability of Octopus Deploy to support Linux ARM and ARM64 targets.

In the blog post, Shawn Sesna explains how to publish a package to a local instance of Octopus Deploy using a self-hosted GitHub Actions Runner, extending on a previous post by Ryan Rousseau. He describes GitHub Actions as a build server using YAML to define workflows, illustrating the process with an example workflow for building and pushing a .NET Core application to Octopus Deploy. Sesna discusses the limitations of GitHub-Hosted runners in accessing on-premises resources and introduces locally hosted runners as a solution, which operate by reaching out to GitHub Actions. He outlines the simple setup process for a local runner, including configuration in the GitHub repository settings, and explains how to modify the workflow YAML file to utilize the local runner. This approach enables the deployment of packages to a local Octopus Deploy instance and can be adapted for other build servers like TeamCity, Jenkins, or Azure DevOps.

Maven is a popular build tool for Java developers, offering a standardized project format and a wide array of plugins for creating Java applications. It utilizes static file-based repositories, such as the widely used Maven Central, to store and distribute artifacts without requiring APIs or servers beyond an HTTP host. These repositories are easy to navigate in a web browser, but search portals like search.maven.org offer convenient indexes for artifact searches. Maven artifacts are identified by a GAV (Group, Artifact, Version) coordinate and may include multiple files distinguished by classifiers and packaging types, such as JARs, WARs, and source files. While developers can download artifacts using web browsers or tools like Curl, Maven's command-line interface simplifies the process by constructing URLs and handling file downloads based on specified GAVs, classifiers, and packaging types. Maven also supports SNAPSHOT versions, which are frequently updated during development and identified by unique timestamps in metadata files, allowing Maven clients to automatically download the latest version.

The decision to move Octopus Deploy to the cloud involves choosing between Octopus Server on a virtual machine and the hosted Octopus Cloud, with key considerations being cost, security, and usage needs. Octopus Server provides more control and customization, allowing users to manage storage, backups, and security, making it suitable for organizations with specific regulatory requirements. It offers the flexibility of using various infrastructure and database hosting options, such as VM, PaaS, or Docker, and supports multiple authentication providers and IP configurations. In contrast, Octopus Cloud simplifies infrastructure management by handling maintenance, backups, and scaling, with a dynamic Worker Pool specific to each region, though it limits access to the database and external package feeds. Both options offer features like Spaces for organizing projects and environments, but while the server allows high availability configurations, Octopus Cloud relies on AKS for resilience. Ultimately, the choice depends on the business's infrastructure requirements and the level of control desired over the deployment environment.

Microservices, when deployed on Kubernetes platforms, offer a robust framework for developing and managing complex systems rapidly and reliably. This approach is complemented by service mesh technologies, like Istio, which simplify networking tasks such as routing, security, and traffic management by abstracting these concerns into the infrastructure layer. The article delves into setting up a Continuous Integration/Continuous Delivery (CI/CD) pipeline using Kubernetes and Istio, showcasing the deployment of Google's Online Boutique sample application to an Amazon EKS cluster. It emphasizes configuring high availability, zero downtime deployments, and HTTPS management, while also providing insights into deploying feature branches for integration testing without disrupting the mainline services. The process involves creating EKS clusters with tools like eksctl, managing Kubernetes targets and namespaces within Octopus Deploy, and leveraging Istio for advanced traffic routing, including HTTP and gRPC requests. Additionally, it discusses high availability strategies, such as configuring pod anti-affinity and adjusting deployment replica counts, as well as employing Octopus's blue/green deployment strategy for seamless updates. The article further explores the use of Istio in routing feature branch traffic based on HTTP headers and gRPC metadata, facilitating the testing of new features in isolated environments.

Nulls, often referred to as the "billion-dollar mistake," pose significant challenges in programming, leading to bugs when not properly managed. To address this, TypeScript's strictNullChecks compiler flag treats null and undefined as distinct types, ensuring they are appropriately handled and reducing related errors. However, enabling this feature in an existing codebase can be complex, potentially resulting in thousands of errors that need methodical resolution. Various strategies can be employed to implement strict nulls incrementally, such as using multiple tsconfig files or the non-null assertion operator, each with its pros and cons. The text discusses lessons learned from converting a codebase to be strict null compliant, such as narrowing types early, handling resource differences, and using defaults to manage optional parameters. It concludes that while starting with strict TypeScript rules is ideal for new projects, existing codebases require careful planning and effort, but the benefits of reduced complexity and improved reliability make it worthwhile.

In the journey of evolving Octopus Deploy, significant changes have been made to its codebase to enhance modularity and simplify integration with external services. Initially, the deployment logic was tightly coupled within the Tentacle agent, requiring synchronized updates between Octopus Server and Tentacle. With the introduction of Calamari in Octopus 3.0, deployment execution was delegated to this standalone engine, reducing Tentacle's complexity. Despite Calamari's success, challenges in adding new integrations led to the creation of "flavors," splitting Calamari into independent, customizable executables for various platforms and technologies. This approach, complemented by the introduction of Sashimi, modularizes Octopus Server, allowing for easier and faster integration of new technologies like AWS or Terraform without deep server-side entanglement. By employing techniques like Single-instance storage, Octopus maintains a lean installer size, ensuring efficient packaging and distribution. The ongoing refactoring with Calamari and Sashimi promises a more agile and maintainable system, streamlining the addition of new features and third-party integrations.

As data scales and becomes more complex, automation is crucial for efficiently managing SQL Server databases, and PowerShell, particularly through the dbatools module, offers a robust solution to streamline these processes. Unlike the SqlServer module recommended by Microsoft, dbatools, an open-source community-driven PowerShell module initiated by Chrissy LeMaire, provides over 500 cmdlets, significantly more than its counterpart, and is noted for its ease of use and strong community support. dbatools simplifies tasks ranging from creating databases and SQL Server logins to executing SQL scripts and migrating SQL instances, supporting best practices and standardizing procedures with simple commands. The module also enables database administrators (DBAs) to focus more on scripting and source control rather than traditional SSMS wizards, integrating them more closely with engineering teams and making them indispensable in the evolving landscape of database management. As DBAs are encouraged to embrace automation, those adept at using tools like dbatools will be in high demand, while opportunities may dwindle for those who cannot adapt, emphasizing the evolution rather than the obsolescence of the DBA role.

Bob Walker reflects on his transition to Git in 2013 and the challenges he faced with feature branch testing in static environments traditionally structured as Dev, Test, Staging, and Production. He discusses the limitations of this model, where new feature testing required merging into the master branch, increasing the risk of unfinished code reaching production. Walker explores alternative approaches like using feature flags and creating sandboxes for feature branches, which help in testing without affecting the production-ready code. He also suggests rethinking the typical Dev and Test environments by combining them into a single, more dynamic Test environment, which would allow for more efficient testing and deployment processes. This approach includes spinning up new infrastructure for feature branches and leveraging tools like Octopus Deploy to manage deployments more effectively. Walker emphasizes the importance of regularly updating feature branches with changes from the master branch to avoid complex merge conflicts. He concludes by proposing a shift in mindset towards treating testing environments like short-lived branches and production environments like a long-lasting trunk, and hints at detailing this new workflow setup in a future guide.

Bob Walker discusses setting up a database deployment process for feature branches, emphasizing the challenges and solutions of adapting traditional static workflows to dynamic environments facilitated by Git. The static workflow of {Dev, Test, Staging, Production} is inefficient for feature branch deployments due to unfinished code merging into the master branch, unclear paths for bug fixes, and potential merge conflicts. Walker suggests combining the Dev and Test environments into a single dynamic Test environment with individual sandboxes for each feature branch, which are created by restoring a backup of Staging. The process includes the use of Octopus Deploy for managing deployments, AWS RDS for database management, and Infrastructure as Code (IaC) for automating the spin-up and tear-down of resources. Involving DBAs early in the process is crucial to ensure smooth production deployments, and Walker highlights the use of runbooks and step templates to facilitate database management tasks. The article underscores the importance of a flexible deployment strategy in agile development environments, allowing for the accommodation of multiple simultaneous feature developments while maintaining stability in Staging and Production environments.

In this blog post, Bob Walker explores using GitHub Actions to build a Redgate SQL Change Automation package and deploy it to Octopus Deploy, addressing the challenges encountered during the process. Walker highlights the necessity of using Windows for Redgate's SQL Change Automation, leveraging GitHub Actions’ ability to run on multiple operating systems, and installing required tools like Chocolatey and Octopus CLI. He discusses setting up environment variables for consistent versioning and package management, utilizing PowerShell for scripting, and the benefits of Octopus Deploy in ensuring a controlled and predictable deployment process. Despite initial challenges, Walker successfully implements the workflow, emphasizing the utility of GitHub Actions as a robust CI tool, while advocating for Octopus Deploy's capability to manage complex deployment needs without compromising simplicity and transparency.

Octopus 2020.2 introduces several updates aimed at enhancing deployment processes and release management. Key features include execution containers for workers, which allow isolated deployment work without managing dependencies, and integrated change management with Jira Service Desk, enabling automatic change requests during deployment. The release also improves run conditions, offering more flexibility with rolling deployments and machine-level variable conditions. A new code editor feature simplifies variable lookups, and the release enforces updated minimum requirements for running Octopus Server. This is the second of six planned releases for 2020, each with six months of long-term support, and users can look forward to future features like Config as Code.

Octopus has introduced execution containers in version 2020.2 to streamline the management of deployment tool dependencies, addressing the challenges of outdated bundled tools and the burden of manually installing dependencies on deployment targets. By utilizing containers, Octopus allows users to execute deployment actions within containerized environments, ensuring that the latest tool versions are used without being tied to the Octopus Server updates. This approach leverages Docker images, including those provided by Octopus on DockerHub, to offer a flexible and controlled environment for managing deployment processes. The use of containers simplifies the setup of Octopus Workers and enhances the deployment of various applications across different platforms and clouds. While currently an Early Access Preview feature, execution containers promise to enhance the deployment experience significantly by providing a more efficient and maintainable solution for handling dependencies.