September 2021 Summaries

Bob Walker's detailed exploration of Octopus Deploy highlights the significance of variable run conditions and output variables in optimizing deployment processes, particularly when dealing with Azure virtual machine scale sets. The post outlines a specific customer scenario where Azure's default over-provisioning led to inefficiencies during scale-outs, causing deployments to take longer than expected. To address this, Walker developed a deployment process incorporating the new "Check VMSS Provision Status" step template, which ensures all new virtual machines are provisioned before deployment begins, while skipping already existing machines. This approach reduces deployment time by synchronizing the provisioning and deployment processes, utilizing the new Contains filter in Octostache to streamline condition checks, thus avoiding unnecessary redeployments and ensuring a smooth scaling process. The post concludes by emphasizing the robustness and maintainability of Octopus Deploy when enhanced by these functionalities, inviting users to reach out for further guidance.

Octopus 2021.2 introduces several new features to enhance deployment processes on the Google Cloud Platform (GCP), complementing its existing support for AWS and Azure. Key updates include the introduction of a Google Cloud Account type that securely stores service account keys, and the ability to inherit service accounts from Google Compute Engine (GCE) virtual machines, which simplifies operations by removing the need to store credentials in Octopus. Additionally, support for Google Container Registry (GCR) has been integrated into the Docker feed type, enabling image queries with activated Cloud Resource Manager API. A new script step allows running gcloud scripts efficiently, and Terraform support now includes the capability to manage Google credentials within the deployment step, with state persistence facilitated by Google Cloud Storage. The update seeks user feedback for continuous improvement and encourages sharing experiences and suggestions related to GCP deployments.

Alex Yates' blog post, part of a series on safe schema updates, explores the advantages of a loosely coupled software and database architecture over traditional monolithic systems. This approach promotes resilience by isolating database administration concerns, allowing each service to manage its own data and communicate through APIs, thereby simplifying dependency management and reducing the complexity of deployments. The post highlights the potential for continuous integration, as smaller and independently managed services make it easier to handle concurrent tasks and mitigate failures. Yates emphasizes that while the technical benefits are significant, human and cultural factors also play a crucial role in achieving these improvements, hinting at further exploration of these aspects in the next installment of the series.

A new Workato connector for Octopus Deploy enhances workflow automation possibilities by enabling seamless integration between Octopus Deploy and various platforms through a low-code/no-code visual designer. Workato, an enterprise Integration Platform as a Service (iPaaS), allows users to construct workflows and automate processes across hundreds of systems by using Ruby-based connectors defined as JSON configurations. This connector supports a multitude of actions, such as creating and deploying releases, managing resources, and triggering events, thus facilitating quick implementations needed by IT, product, and business operations teams. Users can initiate workflows in Workato by installing the Octopus connector, establishing an API key, and setting up a connection to their Octopus Deploy instance. Examples of possible workflows include notifying a Slack channel of Octopus events and integrating ServiceNow, Jenkins, and Octopus Deploy for approval and release processes. The flexibility of Workato allows for extensive customization of workflows to suit team needs, enhancing post-deployment updates to systems like Slack, Microsoft Teams, and Jira.

Octopus Deploy has introduced built-in support for Google Cloud Platform (GCP), joining its existing support for AWS and Azure, to address the needs of customers using GCP Kubernetes clusters and running Tentacles on GCP virtual machines. This integration, available in Octopus Deploy 2021.2 and newer, allows users to connect and authenticate with GCP through a dedicated account type, utilize the gcloud command-line tool in custom scripts, and manage GCP infrastructure with Terraform. Users can deploy and scale containerized applications on GCP using Octopus and Kubernetes, with the process facilitated by Octopus managing GCP credentials, allowing pre-authenticated gcloud scripts. The integration enables the execution of scripts on GCP targets, such as creating Kubernetes clusters, and provides options for quick setup using Octopus Cloud's dynamic workers or custom provisioned tools on worker machines. The integration streamlines authentication processes and supports the deployment of infrastructure to GCP, encouraging user feedback through community engagement.

Continuous Integration (CI) is often misunderstood as merely involving automated builds and tests, but its core purpose is to reduce work in progress and avoid large, complex merges by integrating changes frequently and incrementally. This practice emphasizes breaking down broader goals into smaller, independently deliverable tasks, ensuring minimal risk upon release by validating integrated changes as a whole. While automated builds and testing are essential, CI is fundamentally a project management issue that involves managing development, testing, and deployment efforts to eliminate integration challenges. The article argues that CI should prioritize merging over diverging, advocating for trunk-based development and frequent, small integrations, which contrasts with traditional approaches that involve lengthy integration phases fraught with the risk of "load-bearing bugs." It also highlights the importance of understanding CI in its broader sense to properly address challenges related to database architecture, including loose coupling and Domain-Driven Development (DDD), which facilitate the creation of safe and resilient IT systems.

The Octopus 2021 Q3 release introduces enhanced integration with Google Cloud, providing built-in support for Kubernetes and facilitating deployment and runbook automation on Google infrastructure. This update also features a no-code Workato connector for integrating Octopus Deploy with business workflows, LDAP authentication for self-hosted servers, and new variable substitution filters for creating dynamic run conditions. Additionally, the release includes breaking changes requiring the OctopusUseBundledTooling property for steps using execution containers, impacting users with custom Windows execution containers. With these updates, Octopus aims to simplify complex deployments and improve user experience, while also offering long-term support for Octopus Server releases.

In this article, Alex Yates explores the concept of resilience versus robustness in IT systems, emphasizing the importance of building systems that are prepared for failures rather than solely focusing on preventing them. Through the lens of complex systems theory, Yates highlights the distinction between complex and complicated systems, noting that complex systems, which include human elements, are inherently unpredictable. Drawing on insights from various fields like DevOps and Safety 2.0, the article argues for a shift from traditional Safety-I approaches, which aim to prevent failures, to Safety-II approaches that focus on ensuring systems succeed despite failures. Yates uses Netflix as an example of a resilient IT system that can withstand failures by designing for them, incorporating redundancy, and practicing failure scenarios to maintain core operations. He advocates for frequent, smaller updates and effective containment of failures through strategies like loose coupling and the Strangler Pattern, which can improve database reliability and reduce risk. The article sets the stage for future discussions on continuous integration and other best practices for enhancing IT system safety and reliability.

Octopus has enhanced its support for Kubernetes deployments and runbooks, enabling users to manage tasks and conduct chaos engineering experiments independently from system deployments. The integration of Octopus Runbooks allows for the execution of management tasks in parallel with deployments, making it easier to automate processes like chaos engineering, which tests system resilience by simulating failures. By offering a unified interface and shared context for deployments and runbooks, Octopus enhances the management and tracking of infrastructure states. Runbooks provide a flexible framework, allowing for tasks to be scheduled or executed manually, thus empowering continuous validation of systems, as advocated by experts like Sylvain Hellegouarch. The system's ability to seamlessly integrate tools like the Chaos Toolkit and offer extensive features such as logging, auditing, and reporting, further supports the scalability and reliability of DevOps practices.

In this blog post, Alex Yates explores the complexities and challenges associated with database schema updates within IT systems, describing a metaphorical descent through various levels of "hell," such as data, dependency, and global failure issues. These challenges often stem from databases becoming shared back-end services for multiple systems, leading to difficulties in managing dependencies, testing, and deployment coordination. The post highlights the risks of inadequate testing environments and poor source control practices, which can result in unplanned work and technical debt, eventually overwhelming IT teams with maintenance tasks. Yates emphasizes the need to recognize and address these systemic issues before they become insurmountable, advocating for a shift in perspective and methodology to improve software architecture and delivery processes. The article serves as the first part of a series aimed at re-evaluating safety within complex IT systems and suggests that future posts will delve into concepts like resilience, robustness, and loose coupling to foster a safer and more efficient development culture.

Matthew Casperson's blog post discusses the complexities of Kubernetes and how Octopus's Runbook feature can simplify support and maintenance tasks. Runbooks allow users to manage routine maintenance and emergency operations without deploying, offering advantages over manual scripting and ad-hoc debugging. The post provides a simple Kubernetes runbook example, illustrating how it can enrich diagnostic procedures by automating the listing of pods and identifying those not in a running state. By executing Kubernetes CLI tools like kubectl from a runbook, local installation is unnecessary, and infrastructure credentials are securely managed. Runbooks also integrate business intelligence into scripts and provide detailed audit trails for better troubleshooting and tracking. The Octopus tool supports deployments across various cloud providers and on-premises infrastructures, offering a streamlined, consistent approach to managing deployments and support tasks in complex environments.

Octopus Deploy offers a comprehensive solution for managing Kubernetes deployments by integrating with various cloud services and providing robust features for modeling development environments, managing application variables, and versioning Docker containers. It allows teams to progress code through development, test, and production environments seamlessly, while offering a top-to-bottom view of infrastructure by combining Octopus and Kubernetes dashboards. Octopus supports microservices deployment strategies, whether promoting individual services or entire ecosystems, and facilitates cloud migration by incorporating best practices for secure and manageable deployments. The platform's capabilities include centralized cluster administration through a Script Console, enabling developers to execute commands without sharing credentials, and maintaining an audit log for accountability. By utilizing Octopus, teams can efficiently manage complex deployments across multiple environments and technology stacks, ensuring a cohesive and reliable deployment process.