February 2020 Summaries

Octopus Deploy integrates seamlessly with popular build servers like Azure DevOps, Jenkins, and TeamCity, offering plugins to package artifacts, create releases, and initiate deployments, while also allowing the submission of build information programmatically via its API for unsupported build technologies. The process is illustrated using a GitHub example where, in the absence of a build server, the API is used to push build information such as commit details and versioning to Octopus Deploy. Despite some gaps in the documentation regarding the JSON payload structure, it is crucial to include elements like BuildEnvironment and VcsCommitNumber within the OctopusBuildInformation hash table for successful integration. Troubleshooting tips are provided for common issues such as commit links not showing due to incorrect VcsType settings and clarifications on work items that are handled dynamically rather than through the payload.

Bob Walker's blog post offers a comprehensive guide on setting up an automated database deployment pipeline using Redgate's SQL Change Automation and Octopus Deploy. He emphasizes the benefits of automation, such as improved auditing and version control, while outlining the necessary tools and configurations needed to execute a successful proof of concept. The process involves using a state-based approach to compare the database state stored in a NuGet package with the target database, generating delta scripts for deployment. Walker details each step of the configuration for both Octopus Deploy and build servers like TeamCity and VSTS/TFS, including setting up the developer workstation, configuring the CI/CD pipeline, and implementing a jump box for enhanced security. The tutorial also includes practical advice on managing schema changes and ensuring manual verification for added reliability during initial implementations. Overall, the piece underscores the long-term advantages of automating database deployments, despite the initial setup effort required.

Kubernetes is commonly used to orchestrate containers and manage traffic between them, but it operates without understanding the specific configurations or relationships of the deployed resources. To address this, the operator pattern has emerged, allowing Kubernetes clusters to encapsulate business knowledge and operational workflows by defining Custom Resource Definitions (CRDs) and controllers that automate tasks based on domain-specific logic. This approach extends Kubernetes with higher-level concepts like test web servers, as demonstrated in an example using Kotlin. The operator consists of three key components: the CRD, a controller responding to Kubernetes API events, and application-specific logic, allowing for automated management of resources and tasks such as security scans and load testing. This encapsulation not only standardizes configurations but also enables the automation and management of complex operational tasks within the Kubernetes environment.

Kubernetes Ingress resources enable efficient management of incoming HTTP traffic by allowing multiple services to be exposed through a single public IP address, with NGINX being a popular choice for ingress controllers. However, standard Ingress resources often required customization through annotations or configmaps for more complex routing needs. The introduction of version 1.5 of the NGINX ingress controller, featuring two custom resource definitions (CRDs)—VirtualServer and VirtualServerRoute—addresses these limitations by offering advanced networking capabilities. The blog details how to set up a Kubernetes cluster using Docker Desktop and deploy a sample application, describing how VirtualServer resources can be configured for tasks like traffic splitting, load balancing, and defining timeouts and retries, all of which were previously managed through annotations. Additionally, VirtualServerRoute allows for more granular routing, enhancing flexibility and reducing the complexity of managing Ingress resources. This approach not only simplifies the implementation of sophisticated network rules but also promotes better portability and maintainability within the Kubernetes ecosystem.

Database deployment automation offers a strategic approach to managing complex database changes efficiently, reducing friction in adopting new processes. The text discusses two primary methods: state-based and migration scripts. The state-based approach focuses on defining the desired database state, which is then compared to the deployment target to generate delta scripts, offering integration with IDEs like SQL Server Management Studio. However, it can be complex and may generate unexpected changes. On the other hand, the migration scripts approach involves manually writing delta scripts, providing more control and flexibility for complex changes, though it requires discipline and expertise in SQL. The text suggests that the choice between these methods depends on project needs and team experience, recommending dedicated databases for developers to facilitate risk-free changes and branching. The adoption of automated processes, like using Octopus Deploy, requires building trust and implementing proper permissions to ensure security and effective change management. Communication and collaboration are emphasized as crucial for successful deployment automation, with a suggestion to start with a pilot team to iterate and refine the process before a broader rollout.

Infrastructure as Code (IaC) has significantly advanced cloud infrastructure management by allowing for programmatic definitions of infrastructure setups, ensuring consistency and predictability. However, working with multiple cloud providers often requires learning different tools tailored to each provider, such as AWS CloudFormation or Azure Resource Manager templates. HashiCorp's Terraform addresses this by offering a unified tool that supports multiple providers. This detailed guide demonstrates using Terraform and AWS autoscaling to dynamically create worker pools for Octopus Deploy, which shifts deployment tasks onto other machines, enhancing scalability and reducing the load on the Octopus server. The process involves setting up Terraform configuration files to define resources, such as AWS autoscaling groups, security groups, and virtual private clouds (VPCs), alongside scripting tasks for both Windows and Linux environments. The implementation facilitates efficient deployment handling and cost management by automatically scaling resources based on demand, exemplified by tasks like database deployments and cloud infrastructure management.

Automating database deployments is driven by the need for reliability, traceability, and speed, but teams often hesitate initially due to concerns over maintaining quality and the visibility of production changes. However, failed deployments and human limitations can shift teams towards automation, recognizing that a high-quality process actually facilitates development. Automation can be implemented through different approaches: top-down, incremental changes from the developer's side, or a combination of development and production focus. While automating deployments can be challenging, particularly in aligning environments and processes, those who adopt it consistently find it invaluable, refusing to revert to manual processes even when changing jobs. Elizabeth Ayer from Redgate, a leading provider of database software, emphasizes that investing in the deployment process enhances reliability and efficiency, making the initial efforts worthwhile.

Bob Walker discusses the challenges and solutions associated with database deployment automation, emphasizing the stress and risks involved compared to code deployment due to the lack of flexibility in rolling back changes. He reflects on his experiences as a lead developer where manual database deployments led to discrepancies, time-consuming processes, and frequent errors during major releases. The absence of a source of truth for database schema and manual tracking of changes created significant difficulties. In response, Walker describes how automating database deployments using tools and integrating database changes into source control dramatically improved efficiency, reduced deployment times from hours to minutes, and increased confidence in the deployment process. This automation allowed for more frequent releases, giving teams the ability to deliver features and bug fixes more promptly, while also freeing up DBA and operations teams from after-hours work. This article is the first in a series aimed at guiding readers through setting up database lifecycle management and deployment automation, providing real-world examples and highlighting common pitfalls.

Shawn Sesna's post discusses the use of project triggers in Octopus Deploy to facilitate the deployment of applications to dynamically created targets, particularly useful in environments with scaling capabilities such as Kubernetes on Azure. The post explains a scenario where an Azure Resource Manager template is used to create a Kubernetes cluster, deploy the application, and then dismantle the cluster to save costs. The challenge arises when deployment targets are created as part of the deployment process, which can lead to steps being skipped since Octopus Deploy selects targets at the start of deployment. To resolve this, Sesna suggests adding a post-deployment script to register the Kubernetes cluster with Octopus and incorporating a Health Check step to ensure new targets are included. This allows the deployment process to recognize and deploy successfully to newly created targets.

Kubernetes offers two native deployment strategies, recreate and rolling update, which allow administrators and developers to manage software or configuration deployments with varying approaches to downtime and concurrency. The recreate strategy deletes old pods before creating new ones, ensuring no concurrency but causing downtime, which is useful for backend datastores that cannot handle simultaneous client versions. The rolling update strategy, on the other hand, incrementally replaces old pods with new ones, maintaining some active pods to prevent downtime but requiring datastores to support concurrent client versions. Additionally, Octopus provides a blue/green deployment strategy, not natively supported by Kubernetes, which involves creating a new deployment, switching traffic to it once ready, and then deleting the old deployment. This method ensures the new deployment is fully operational before traffic is switched, offering a safety net by allowing for rollback if the new deployment fails, but also requiring datastores to handle concurrent versions temporarily. Understanding these strategies is crucial for selecting the appropriate method based on specific deployment needs and minimizing potential downtime.

Octopus Cloud, initially launched on Amazon Web Services in Portland, Oregon, has transitioned to Microsoft's Azure platform to enhance performance and reduce costs, with the new default hosting region being Azure West US 2 in Washington, USA. Now, Octopus Cloud has expanded its availability to include a new region in Azure West Europe, located in the Netherlands, allowing customers to choose their preferred hosting location. The company is open to migrating existing customers to the new region upon request and plans to add more hosting regions based on customer demand.

Mark Harrison's article from February 3, 2020, delves into the concept of rolling deployments, a software deployment strategy that allows updates without downtime, thus keeping applications available 24/7. Unlike big-bang deployments that take the whole application offline, rolling deployments incrementally update software across multiple targets, ensuring at least one instance remains online to serve users. This method offers several advantages, such as the ability to perform health checks and rollbacks seamlessly. The article provides practical examples of implementing rolling deployments using popular tools like Docker, Kubernetes, and Octopus Deploy, highlighting Docker's native support for rolling updates and Kubernetes' capability to manage pod updates incrementally. Additionally, the article touches on the potential challenges of database rollbacks during deployments and emphasizes the importance of backward-compatible database changes. Harrison concludes by recommending rolling deployments for their minimal disruption and compatibility with continuous integration and delivery (CI/CD) pipelines.