November 2019 Summaries

In a 2019 announcement, Octopus Server outlined upcoming changes to its system requirements, which include raising the minimum required operating systems to Windows Server 2012 R2 and SQL Server 2017+, while ending support for Internet Explorer 11. The changes aim to streamline their codebase by focusing on .NET Core 3.x, which is compatible with Windows Server 2012 R2, and to leverage advanced JSON features in SQL Server 2017 for improved performance and security. While these updates may initially present challenges, Octopus believes they will enhance the overall functionality and security of their services, encouraging users to adopt modern operating environments. The company provided guidance on upgrading both operating systems and database servers, and assured users that deployments would remain compatible with older systems. They also acknowledged the declining relevance of Internet Explorer 11 and advised users to transition to modern browsers for a better experience. Additionally, Octopus is preparing to support hosting on Linux, further expanding their platform's flexibility.

Octopus Deploy aims to streamline automated application deployments, which often include database management, by utilizing PowerShell and SQL Server modules for tasks such as installation, connection testing, and database creation. The SqlServer module, recommended by Microsoft, offers cmdlets that facilitate SQL Server administration, including creating logins and running SQL scripts. Users can test database connectivity using basic SqlClient classes and create databases via SQL commands or SQL Server Management Objects. Additionally, PowerShell can execute SQL scripts from files, manage database ownership, and conduct inline SQL queries to ensure data integrity. The post encourages community contributions and feedback through GitHub and highlights the utility of PowerShell in automating database configurations and operations within deployment processes.

Jim Burger's blog post discusses the versatility and adaptability of using Microsoft .NET Core for development on Linux, offering insights and tools for .NET developers transitioning to Unix-like operating systems. He emphasizes the use of Docker and docker-compose to streamline the setup of databases and logging tools, highlighting the efficiency of short YAML configurations and the security benefits of using environment files. Burger shares his personal preferences for Linux-friendly development tools, recommending IDEs like Jetbrains Rider and Visual Studio Code for their cross-platform support and seamless integration. He also touches on the use of Vagrant for managing isolated virtual environments, particularly for testing untrusted binaries, providing a sample configuration for setting up a lightweight Arch Linux environment. The post encourages developers to explore these tools and methods to enhance productivity and flexibility in their development workflows.

Octopus Cloud underwent a significant transformation journey to optimize costs and improve performance, starting with its initial launch in July 2018 on AWS, where hosting costs far exceeded revenue due to a dedicated VM architecture per customer. Confronted with the need to reduce costs and enhance provisioning speed, the company explored various compute options, ultimately deciding to rebuild its platform from the ground up by transitioning to Kubernetes, Linux, and .NET Core, and switching from AWS to Azure. This shift allowed Octopus Cloud to reduce per-customer hosting costs by over 50% and cut provisioning times from over 10 minutes to less than 30 seconds, demonstrating marked improvements in performance and efficiency. The decision to embrace Kubernetes and .NET Core was driven by the desire to leverage existing internal expertise, avoid vendor lock-in, and align with the broader industry trend towards containerization, despite the challenges of migrating from a Windows-based system to a Linux-based, containerized environment.

The article by Bob Walker explores the challenges and inefficiencies of performing SQL rollbacks in database deployments, suggesting that rolling forward is typically a more effective strategy. Rollbacks can be complex and risky, often leading to data loss or corruption, and are rarely justified except in specific, limited scenarios. The article emphasizes that the effort required to roll back a database change often surpasses that of rolling forward with a fix, especially once users have begun interacting with the updated system. Walker argues for prioritizing fast and safe deployments and making database changes backward compatible to reduce deployment risk. He also highlights the pitfalls of relying on backups and rollback scripts, noting that they often have a limited lifespan and can complicate the deployment process. Ultimately, the article advocates for focusing on improving deployment processes rather than on elaborate rollback strategies, as rolling forward can be more efficient and less disruptive.

Bob Walker discusses the iterative journey of implementing an automated database deployment process using TeamCity, Redgate, and Octopus Deploy, highlighting the challenges and solutions encountered along the way. Initially, the process involved manual script generation and lacked consistency across environments, which posed difficulties in tracking changes and testing. The first version of automation introduced some improvements but still had significant limitations, such as different procedures for various environments and manual interventions. The introduction of Octopus Deploy in the second version streamlined the process by providing a centralized deployment server, enabling consistent deployment practices, and improving auditability. This version resolved most issues by automating script generation and execution and allowing database developers and DBAs to review changes at different stages. Further refinements in version 2.1 reduced the need for DBA involvement during production deployments by enabling post-deployment approvals and enhancing test data availability for developers. Despite initial complications, the final deployment process became more efficient and reliable, demonstrating how tailored solutions can evolve to meet specific organizational needs.

Bob Walker recounts his experience automating database deployments, reducing deployment times from hours to minutes by overhauling the existing process rather than merely automating it. Initially, a small work group comprising developers, DBAs, and a database architect collaborated with Redgate to design a streamlined process after recognizing the inefficiencies of their "wild west" style deployments. The revised process involved switching from a shared to a dedicated database model, leveraging tools like Git, TeamCity, and Octopus Deploy, and setting clear roles and permissions to prevent unauthorized changes. The implementation led to reduced deployment times and a significant drop in errors, prompting more frequent and efficient deployments. The transition to this new process initially met with resistance, but it eventually proved beneficial by freeing up database developers to focus on more strategic tasks. Walker highlights the importance of iteration and adaptation in refining the process, which ultimately transformed production deployments into seamless, routine operations.

Bob Walker discusses the journey of automating database deployments to significantly reduce deployment time from hours to minutes, highlighting the need to redesign fundamentally broken processes. He emphasizes the importance of establishing a small, empowered work group, including representatives from each stage of the deployment process, to define and refine the ideal deployment process through collaboration and regular feedback. The initial focus should be on drafting the process without getting bogged down by specific tooling, while considering features like state-based vs. migration-based development and common tools like SQL Server Management Studio or Visual Studio. Walker advises starting with a pilot team to implement the process and tools, iterating based on feedback, and gradually rolling it out to early adopters and then more broadly, while building trust through manual verification and pilot applications. He underscores the necessity of continuous iteration to address unique needs and pain points as the process is adopted more widely, aiming to create a reliable, automated database deployment system.

Matthew Casperson discusses the limitations and current methods for using custom plugins with Terraform, as there is an ongoing discussion about potentially supporting custom plugin repositories. Currently, only official Terraform plugins are automatically downloaded, while custom plugins must be manually distributed. Casperson demonstrates how to incorporate a custom plugin, specifically the Octopus plugin, into a Terraform template by outlining three methods for making these plugins accessible: saving the plugin globally in specific directories for Windows, Linux, and MacOS; placing the plugin alongside the template files in the .terraform/plugins/<arch> directory; and using the --plugin-dir option to specify a custom directory. These methods are necessary until Terraform possibly implements support for custom plugin repositories, allowing end users to deploy plugin executables themselves.

NGINX Unit is a modern solution for managing web applications that emphasizes simplicity and flexibility, using JSON for configuration and an HTTP API for ease of interaction. Unlike traditional NGINX, which relies on more complex and less standardized configuration files, NGINX Unit offers a declarative model that simplifies common networking tasks like routing and security, making it particularly suitable for environments that leverage multiple programming languages. It allows for reconfiguration without restarting applications, supports polyglot development, and is quick to install and deploy, making it a compelling alternative to traditional methods such as PM2 or systemd. While it lacks some advanced features like complex routing rules and health checks, and is limited to Linux and MacOS, its design reflects a back-to-basics approach that is expected to evolve with regular updates. For those whose needs align with its current capabilities, NGINX Unit presents a lightweight and straightforward option for application deployment and management.