January 2022 Summaries

Effective artifact repository management is essential for optimizing CI/CD pipelines by ensuring security, versioning, and dependency management, which enhances software delivery performance and consistency. Tools like Apache Maven, Bazel, and Docker Image Repositories are crucial in streamlining these processes, while Harness CI/CD integrates with such tools to automate and secure artifact management. An artifact repository securely stores artifacts, such as container images and software packages, which are distinct from source code and serve as the backbone of the software delivery process. Key considerations for artifact repositories include centralized versus distributed models, versioning support, permissions, auditability, and scanning for security vulnerabilities. Artifact management tools like Maven, Bazel, and Docker Image Repositories cater to different aspects of artifact management, emphasizing security, automation, and efficiency in modern DevOps practices.

Harness has introduced a GitOps workflow for its Feature Flags, allowing developers to manage their feature flag pipelines entirely through code, thus simplifying the release process while ensuring immediate updates in the Harness UI and compliance with governance rules. This new feature addresses the challenge developers face with multiple tools and interfaces by enabling them to work within their preferred coding environment, using YAML syntax, and synchronizing changes with Git. The system retains the flexibility of both UI and code-based management, ensuring that updates are instantly reflected and validated against predefined governance standards. The integration of GitOps into Harness Feature Flags, which builds on existing capabilities for managing release pipelines, offers developers a streamlined, in-code approach to release management without bypassing governance rules.

Harness's adoption of a micro-frontends architecture significantly enhanced its application performance, allowing for independent development, deployment, and efficient resource sharing, akin to microservices for the backend. By transitioning from a monolithic web application to distributed individual applications, Harness achieved improved initial load times, independent module deployments, and reduced cognitive load on developers. The company explored various micro-frontends solutions, including Webpack Module Federation, Import Module As an App, Managed Wrapper, and iframes, each with its own pros and cons related to bundling, dependency sharing, data/resource sharing, and security concerns. Ultimately, Harness opted for a solution that utilized Webpack Module Federation coupled with a wrapper to facilitate two-way data sharing and independent service operation of child modules, enabling seamless integration and flexibility without reliance on a single third-party library. This approach allowed for minimal changes to the existing parent application and fostered an environment conducive to experimentation while supporting open-source contributions.

Feature flags, also known as feature toggles, offer a strategic approach to managing software features by enabling conditional activation of code sections, thereby facilitating controlled feature releases and enhancing delivery velocity while minimizing risks. The practice helps in testing features with live data in production, allowing for rapid iteration and improved incident management. Effective feature flag management requires careful consideration of naming conventions, governance processes, and operational toggles to avoid system bloat and technical debt. Organizations are encouraged to use feature flags by default and adopt trunk-based development to streamline code deployment and feature releases. By separating deployment from release, feature flags empower various teams within an organization to manage features independently, providing a first line of defense against potential system failures. Testing in real-world conditions with feature flags enables teams to gather valuable data for iterative improvements, ensuring features align with customer needs and business objectives. Implementing enforceable governance processes is crucial for maintaining system hygiene and ensuring compliance with best practices, ultimately maximizing the benefits of feature flags in a software delivery lifecycle.

Migrating from Jenkins to Harness CD significantly streamlined deployment processes by eliminating the need for scripting and enabling advanced deployment strategies like Canary and Blue-Green without custom logic. Harness, designed specifically for Continuous Deployment (CD), improved efficiency and reliability with features such as Continuous Verification, which aids in ensuring deployment health and recommending rollbacks if necessary. The transition highlighted the limitations of using Jenkins, a Continuous Integration (CI) tool, for CD tasks, which often required relying on Helm for managing deployed versions and rollbacks. Harness's configuration-based approach, along with its templating engine, allowed for the easy recreation of pipelines, reducing complexity and enhancing deployment awareness compared to traditional methods. This shift provided a more modern and integrated solution, emphasizing the benefits of using a tool built from the ground up for CD tasks, ultimately offering a more robust and seamless deployment experience.

Harness CI now integrates with GitHub Actions, enabling users to utilize over 10,000 pre-built actions from the GitHub marketplace in their continuous integration pipelines, thereby enhancing automation, efficiency, and flexibility. This integration allows GitHub Actions to be used as plugin steps within Harness CI, facilitating tasks such as code cloning, Docker image building, and vulnerability scanning. The process involves specifying attributes like 'name', 'with', and 'env' in the plugin step settings, and running the step in privileged mode due to the use of Docker in Docker (dind). Examples provided include using the Trivy scanning action for vulnerability detection, a GCS upload action for uploading files to Google Cloud Storage, and a Git checkout action for cloning repositories. The integration is supported by nektos/act, which runs a Docker container to execute the GitHub action workflow locally, emphasizing the pluggability and extensibility of Harness CI.

Dashboarding, as described by the author, is a method of transforming scattered data into actionable insights through visualizations such as pie charts and bar graphs, which can help companies improve their Software Development Life Cycle (SDLC). By capturing and analyzing data from various sources like source code management systems, project tracking systems, and CI systems, organizations can monitor metrics such as commits per pull request, time to merge, and lines of code to identify bottlenecks and inefficiencies. This process allows for a deeper understanding of internal team performance, coding quality, and deployment frequency, ultimately aiding in strategic decision-making and process improvement. Dashboarding offers the advantage of developing tailored dashboards in-house without sharing data with third-party entities, ensuring full control over key information.

Harness Intelligent Cloud AutoStopping Rules offer an automated solution for managing idle cloud resources, helping businesses reduce cloud costs by up to 70% by ensuring resources run only when needed. These rules address the issue of cloud cost overruns, often caused by idle resources, by dynamically detecting idle times and shutting down or terminating resources accordingly. They allow workloads to run on fully orchestrated spot instances without interruptions and provide a user-friendly dashboard for managing rules. Unlike traditional resource schedulers, AutoStopping Rules allow for better prediction of idle times and access to stopped machines. They can be integrated seamlessly with existing infrastructure and are particularly useful for non-production workloads, offering significant savings and operational efficiency. The rules support Kubernetes clusters on AWS, GCP, and Azure, and can be configured with advanced settings like ECG heartbeat agents and fixed schedules, providing flexibility and control over cloud resources. Additionally, AutoStopping Rules complement autoscaling tools by focusing on cost savings and resource management at a granular level, making them a powerful tool in cloud financial operations.

Lessonly by Seismic transitioned from Jenkins to Harness CI, granting over 30 developers access and ownership of the build process, which led to enhanced self-sufficiency and a better developer experience without needing intervention from the Site Reliability Engineering (SRE) team. Previously acquired by Seismic in 2021, Lessonly is a training and enablement solution used by over four million learners at more than 1,200 companies. The change to Harness CI Enterprise provided a unified CI/CD solution, allowing developers to follow artifacts from creation to deployment across environments, thus easing the development process with improved logging and visibility. This transition enabled developers to independently handle build failures, a task that was previously difficult with Jenkins due to limited access. The SRE team praised Harness for its user-friendly interface and features like the YAML editor and Visual Pipeline Builders, which facilitated easy onboarding and setup, empowering developers to manage the entire application delivery process autonomously.

Harness, a company committed to advancing Continuous Delivery, highlights the achievements of employees Sony Priyadarshini and Tathagat Chaurasiya, who were honored with December's Canary Award for their efforts in enhancing product quality and overcoming industry challenges. Sony, a Staff Software Development Engineer in Test, values the company's culture of transparency and collaboration, which fosters trust and motivates her to maintain high-quality standards. She cherishes the memory of achieving significant automation coverage on the Pipelines team. Tathagat, part of the Continuous Delivery Platform team, appreciates the opportunity to tackle real-world challenges and values the supportive, diverse team environment. His favorite memory includes a company offsite in Jaipur, and he advises persistence and curiosity for career growth. Harness is actively recruiting and encourages interested individuals to join their talent community on LinkedIn.

Trunk-based development (TBD) is a practice that enhances continuous integration by merging small, frequent changes into the main branch, using feature flags to manage the release of these changes and ensure incomplete features remain inactive. This approach differs from traditional branching strategies, such as gitflow, where feature branches are long-lived, and code merges occur less frequently, often causing complexity and merge conflicts. TBD encourages constant integration, ideally daily, which reduces merge complexities and accelerates continuous integration/continuous deployment (CI/CD) processes. While the method presents challenges, such as the potential for code collisions and difficulties in managing production releases, feature flags provide a solution by allowing changes to be merged without activating unfinished features, thus maintaining safe and frequent deployments. The integration of feature flags with TBD enables teams to increase their velocity and release quality by minimizing the risks associated with incomplete or untested code, positioning feature flags as a crucial component for successful trunk-based development.

Harness has launched a source-available Community Edition of its Continuous Delivery (CD) platform, designed to simplify and expedite the deployment of pipelines for developers, reducing the need for enterprise contracts and minimizing bureaucratic procedures. This offering is particularly beneficial for developers at large enterprises and smaller teams, allowing them to build and deploy pipelines in about 15 minutes. The Community Edition operates under the Polyform Shield License, permitting free use for most users, and initially supports Kubernetes, with plans to extend compatibility to AWS, Azure, Google Cloud Build, VMware Tanzu, and serverless deployments. It includes features such as automated deployments, infrastructure provisioning, integrated approval flows, and a pipeline-as-code experience. Available for download on GitHub, this edition requires Docker Desktop or a Kubernetes cluster for installation. As a first general-availability release, Harness is seeking user feedback to refine this self-managed software offering, complementing its existing SaaS products.

Harness is focusing on empowering developers by addressing the need to separate continuous integration (CI) and continuous delivery (CD) processes, enhancing efficiency through AI/ML, and integrating cloud cost visibility into the software development life cycle (SDLC) to avoid unexpected expenses. During a discussion at KubeCon + CloudNativeCon NA 2021, Nick Durkin, Field CTO at Harness, emphasized the importance of these strategies, noting that the integration of cloud cost management is pioneering in the industry. Harness's expansion includes new offices in Belfast and Belgrade, and the release of new features such as Continuous Integration and Continuous Delivery (CIE) and Feature Flags, with plans to incorporate Cloud Cost Management. The organization aims to streamline operations by allowing developers to focus on high-level tasks rather than mundane ones, thus optimizing productivity.

Feature flags enable developers to safely experiment in production by controlling the extent of changes, which fosters innovation and quicker iteration cycles. This approach enhances the developer experience by supporting data-driven decision-making and cost optimization, allowing for a more flexible and effective development process. Experimentation, a common practice in business, allows developers to iterate and improve without risking large-scale negative impacts on production environments. Understanding what constitutes normal performance in applications is challenging, as performance optimization requires input from various stakeholders, including developers and performance engineers. The dissemination of performance and cost information, often locked away by finance departments, is crucial for effective optimization, highlighting the importance of methodologies like FinOps. Platforms like Harness support a positive Developer Experience by enabling efficient development cycles and improving key metrics such as deployment frequency and change failure rate, ultimately nurturing innovation and productivity among engineers.

Cloud cost management is a collaborative effort requiring input from developers, finance teams, and stakeholders, where effective practices involve attributing costs, optimizing expenses, and governing resources. Harness CCM offers tools like Custom Cloud Cost Perspectives and Custom Dashboards, which allow users to organize and visualize multi-cloud cost data in ways that align with business objectives, providing insights into specific resources or workloads. Perspectives enable detailed cost analysis and reporting, facilitating the identification of wastage and offering actionable recommendations to reduce expenses while maintaining governance. Custom Dashboards enhance this capability by offering advanced data visualization and analysis, allowing users to create custom fields and bucket costs, thereby providing a comprehensive view of cloud inventory and usage. Harness CCM plans to introduce further features, such as Business Mapping, to improve cost allocation and management.

Implementing the Builder Factory Pattern in test setups can significantly enhance the maintainability and refactor-friendliness of codebases by simplifying the process of writing and maintaining tests. This approach reduces setup complexity and maintenance costs, encouraging developers to write more tests by making the process easier than manual testing. The Builder Factory Pattern involves using a factory to create builders of objects with default properties, while still allowing property overrides for specific tests, thus combining the benefits of shared default objects with the flexibility of customization. This method supports easier refactoring, promotes immutability, and minimizes the impact of changes in validation logic on existing tests. By using Java with tools like Lombok for builder generation and Guice for dependency injection, the pattern ensures that tests are concise and focus only on relevant fields, contributing to a more efficient and reliable testing process.

Implementing Test Intelligence in Continuous Integration (CI) pipelines can significantly reduce testing bottlenecks, potentially accelerating development cycle times by up to 52% while enhancing visibility into test cycles and identifying code coverage gaps. In a webinar hosted by DevOps.com, Shivakumar Ningappa, Principal Software Engineer, discussed the challenges posed by lengthy unit testing cycles and rising infrastructure costs, highlighting the benefits of the "fail fast" methodology, which emphasizes speed in the development process. The webinar explored how Test Intelligence was built, its benefits, and included a demo showcasing its impact on development efficiency. Shivakumar presented data illustrating the substantial time savings Test Intelligence offers, concluding with a session of questions and answers to further elaborate on its application and effectiveness.

Campspot, a company specializing in reservation software for resorts, transitioned from AWS Code Deploy to Harness for its continuous delivery needs, significantly enhancing operational efficiency and customer experience. Originally, as the company scaled, they faced challenges with AWS Code Deploy due to its lack of native support for multiple ports in their applications, leading to complex custom scripted workarounds that resulted in deployment complexities and lengthy rollbacks. These issues, including frequent deployment failures and long downtimes, necessitated late-night deployments and frustrated both developers and the DevOps team. The switch to Harness resolved these challenges by simplifying the deployment process, enabling daily deployments, and reducing rollback time from 45 minutes to 10 minutes, which allowed the team to focus on other critical projects and improve the reliability and speed of deployments, thus preventing customer-impacting issues.

Harness CI Enterprise (CIE) optimizes the continuous integration process by implementing a caching strategy that significantly reduces build times and conserves network bandwidth, particularly for UI builds. This strategy leverages Kubernetes for building infrastructure, allowing for efficient execution of build jobs in containers, and integrates smoothly with continuous deployment processes. Harness CIE's caching mechanism stores data in services like Amazon S3 or GCS, reducing redundant downloads of node modules, which can lead to substantial time savings, as demonstrated by a reduction from over five minutes to just one minute and 25 seconds in execution time in a test scenario. The platform provides a user-friendly interface with options for building directly from the UI or using a YAML editor, and it supports features like Test Intelligence to minimize test times. By utilizing these features, Harness aims to streamline development workflows, enhance pipeline performance, and reduce resource consumption, offering a robust alternative to other CI tools like Jenkins.

Harness developed a custom solution for string externalization in React applications to meet specific needs, such as ease of updates for documentation teams, templating support, developer-friendly APIs, validation, and autocomplete, which existing packages like react-intl and react-i18next did not fully address. The in-house approach utilizes YAML for string storage, mustache.js for templating due to its small size and security features, and TypeScript for validation and autocomplete, with React's Context API ensuring the strings are accessible throughout the application. The solution includes a custom hook, `useLocaleStrings`, to fetch strings, and supports nested objects using lodash functions. The integration of mustache.js allows for template-based string rendering, and the setup is further enhanced by leveraging TypeScript for static analysis, which will be explored in a forthcoming sequel to this exploration. All the code for this implementation is available on GitHub.