March 2020 Summaries

Bob Walker discusses automating database deployment approvals to streamline the process and reduce the need for manual intervention by Database Administrators (DBAs). Initially, DBAs play a crucial role in approving changes to prevent potential failures in production and build trust in the automation process. However, Walker suggests a multi-layer approach to automate approvals, where initial checks for standard violations occur on the build server using tools like SQL Enlight and tSQLt. This ensures scripts meet standards before reaching Octopus Deploy. Despite automation's capabilities, some scripts may still require human review; thus, Walker recommends manual interventions in a QA, Testing, Staging, or UAT environment, not in production or development, to balance oversight and efficiency. PowerShell scripts are used to inspect delta reports and trigger manual interventions when necessary, with a focus on maintaining visibility and trust in the deployment process. Through this strategy, Walker outlines a method for incorporating logic into deployment processes using output variables and run conditions in Octopus Deploy, enabling auto-approvals while ensuring essential review points remain intact.

Bob Walker discusses updates to DbUp, a free and open-source tool used in conjunction with Octopus Deploy for automating database deployments. The article highlights how DbUp executes SQL scripts in a controlled sequence, offering features like script approval for DBAs and HTML report generation to track changes. DbUp's new capabilities include running scripts multiple times through the AlwaysRun feature and grouping them for better organization. Walker also elaborates on creating a .NET Core console application for deploying to SQL Server, emphasizing the seamless integration with Octopus Deploy to automate the deployment process. The article concludes by addressing integrated security concerns and setting up dedicated worker pools for different environments, further enhancing the database deployment pipeline's robustness and approval processes.

Jim Burger discusses the importance of enhancing the command-line interface (CLI) experience with features like tab completion, which can significantly improve productivity and ease of use for developers. He shares his frustration with having to switch between the shell and browser to remember commands, leading to the implementation of new features in Octopus CLI that support tab completion in popular shells such as zsh, bash, and pwsh. By installing specific scripts, users can benefit from automatic suggestions and completion of commands, thus streamlining their workflow. Burger explains the mechanics behind tab completion, which involves registering commands that provide suggestions when the tab key is pressed, and highlights how Octopus CLI and other tools like dotnet can simplify this process by offering subcommands that handle suggestions. Ultimately, the article advocates for the integration of shell completion features in CLI tools to save time and improve efficiency for developers.

With the release of Octopus 2020.1, Octopus Server now operates on .NET Core 3.1, allowing installation on Linux, Docker Containers, and Kubernetes, which brings various benefits such as a modern development environment, cross-platform support, and improved performance and reduced costs for Octopus Cloud. The transition required addressing challenges like platform-specific differences between Windows and Linux, particularly in configuration settings and database performance, and necessitated adjustments in authentication providers. The team enhanced their debugging skills for Linux and Docker, and the shift to .NET Core enabled the use of self-contained packages, simplifying installation by reducing dependencies. This progression supports innovation and rapid software delivery, although it also means dropping support for older operating systems.

Octopus Deploy 2020.1 introduces several enhancements, including the addition of dark mode, which is designed to reduce eye strain and improve user experience. This release also expands cross-platform compatibility, making the Octopus CLI available on Linux and macOS through Homebrew, APT, and YUM package repositories, and enabling AWS and Azure deployments to be executed on both Windows and Linux machines. Additionally, the update introduces environment and tenant-specific worker pools, which allow dedicated worker pools for different deployment environments, enhancing deployment flexibility. The release raises the minimum requirements for running Octopus Server and includes modifications to library variable set permissions, offering more granular access control. With long-term support for the release, Octopus aims to continue delivering robust updates, and the team is already planning future enhancements such as deep git integration.

Octopus Deploy 2020.1 introduced a dark mode feature to enhance the user experience by catering to the preferences of developers and operations professionals who often favor this mode for its eye comfort benefits. This update aligns with the company's commitment to maintaining a modern and user-friendly interface, which is crucial for appealing to its developer-centric audience. The implementation involved using CSS custom properties and React Context to manage theming, ensuring a seamless transition between light and dark modes, and leveraging the prefers-color-scheme media feature to detect system settings. Despite challenges with IE11 compatibility, a ponyfill was utilized to ensure basic functionality in that browser. This move reflects Octopus Deploy's focus on design and accessibility, aiming to make software deployment a more enjoyable task.

Bob Walker discusses the use of a model-based approach for database deployments with Redgate's tooling, emphasizing its ease of use for teams new to automated database deployments. This approach simplifies the process by comparing the desired state of a database with its current state and generating a delta script. However, it struggles with complex changes like renaming tables or moving columns, which require workarounds such as breaking deployments into multiple steps to ensure non-breaking changes. Walker illustrates the migration of a build server from TeamCity to Jenkins, detailing the configuration of Jenkins with Octopus Deploy for seamless deployments. He highlights the importance of post-deployment scripts in managing data transitions and suggests maintaining discipline in script management to avoid repetitive executions. The process is flexible enough to support blue/green deployment strategies by allowing databases to support the two most recent code versions. Walker shares insights on how this methodology has been successfully implemented in the past, enabling efficient data migrations and rapid environment setups, thus paving the way for advanced deployment strategies.

WebDriver has become a standard for simulating user interactions through web browsers, making it easier to conduct browser-based tests, with platforms like Cypress offering specialized interaction methods. While headless browsers, initially popularized by PhantomJS and now supported by Chrome and Firefox, allow automated tests in non-interactive environments, they are not suitable for all applications, particularly desktop interfaces and screen recordings. Windows' historical approach to interactive services, which allowed services to interact with users, has been restricted due to security concerns, leaving headless browsers or solutions like Azure DevOps interactive agents as viable alternatives. Linux users benefit from more flexible headless testing environments with tools like xvfb that emulate a display in memory, while off-site testing services such as Browserstack or Sauce Labs offer automated testing without the need for users to configure operating systems. For Windows, using auto-logon and running agents on startup remains a practical, albeit risky, solution for testing applications that require a real desktop environment.

Alex Yates discusses various database source control and deployment tools for SQL Server users of Octopus Deploy, highlighting the differences between model-based and migration-based solutions. Model-based tools focus on scripting the desired end state of a database, while migration-based tools execute an ordered list of scripts against the target database. Yates advises against creating home-grown solutions due to their complexity and cost, suggesting instead the use of popular tools like SSDT, Redgate's SQL Change Automation and SQL Source Control, Entity Framework Migrations, Flyway, and DbUp, each with its own advantages and challenges. He emphasizes the importance of choosing a tool that fits the team's specific needs, considering factors like IDE preferences, budget, and complexity. Yates, with his extensive experience in the database DevOps space, encourages teams to avoid reinventing the wheel and to utilize existing tools that have been refined by the community.

Bob Walker shares his experience of transitioning from manual to automated deployments for Oracle databases, using a combination of TeamCity, Octopus Deploy, and Redgate's Oracle Toolset. He outlines the creation of a CI/CD pipeline that automates the deployment process, reducing the complexity and time associated with manual deployments. Walker details the setup process, including creating a source database, tying Oracle to source control, configuring a build server, and setting up Octopus Deploy to manage deployments. He emphasizes the importance of generating delta scripts and having human oversight to build trust in the deployment process. While acknowledging the pipeline's current limitations, Walker encourages further enhancements, aiming for more efficient and reliable database deployments in the future.

Initiating a new IT project can be overwhelming due to the complexity of integrating various tools and platforms, especially in environments where deployments using Kubernetes, NGINX, WildFly, or Tomcat are common. To simplify this process, Octopus has introduced the TestDrive project, which offers virtual machines available on Vagrant Cloud that encapsulate self-contained CI/CD workflows with popular open-source platforms like Jenkins, Docker, and Kubernetes. These virtual machines, built for Hyper-V and VirtualBox, provide users with a complete CI/CD pipeline environment that is easy to set up with just a few commands and offers a secure space to experiment with Octopus and related tools. The TestDrive VMs aim to reduce frustration and speed up the learning curve for developers exploring Octopus's capabilities within a CI/CD pipeline.

Octopus Deploy is revising its release strategy by providing six feature releases annually, each receiving critical patches for six months, and eliminating the Long Term Support (LTS) designation to expand support uniformly across all releases. New versions will initially be deployed to Octopus Cloud instances, allowing for immediate issue resolution, before being offered for download to self-hosted users, which enhances stability and user confidence. This approach addresses the previous challenge of balancing new features with stability and aims to offer a seamless experience where users no longer have to choose between the latest features and stable releases. The shift is expected to benefit both cloud and self-hosted customers by ensuring more stable releases and equitable access to new features.

Bob Walker's article delves into the complexities and advantages of automating database deployments, emphasizing the transformative impact on continuous delivery processes. He highlights the challenges associated with manually running ad-hoc queries, such as reliance on specific personnel and lack of immediate notification, which can delay necessary data fixes. To address these issues, Walker introduces the use of Octopus Deploy to automate the execution of ad-hoc queries, providing features like auditing, artifact management, approvals, and automation to streamline the process and enhance efficiency. By implementing a structured workflow with Octopus Deploy, including the use of YAML files for metadata, automated notifications through Slack, and manual interventions for specific conditions, the process becomes more reliable and less dependent on individual availability. He acknowledges that while the process may not suit every organization, it offers a flexible framework that can be adapted to various company needs, reducing manual intervention and enhancing deployment reliability.