September 2022 Summaries

Let's Encrypt is a free, automated, and open certificate authority integrated with GitLab to help DevOps teams enhance web security by encrypting traffic and safeguarding user data on websites. The process of adding a Let's Encrypt Transport Layer Security (TLS) certificate to a GitLab Pages-hosted website is simplified into a few straightforward steps, including creating a GitLab project, adding source code, and enabling automatic certificate management. TLS, which evolved from Secure Sockets Layer (SSL), is critical in ensuring secure internet connections, protecting data from interception and manipulation. Historically, obtaining TLS certificates was complex and costly, but Let's Encrypt, established in 2013 by the Internet Security Research Group, made it easier, automated, and free. GitLab's integration with Let's Encrypt allows users to secure their websites with minimal effort, contributing to a safer internet. By implementing TLS certificates, DevOps teams not only enhance their website security but also participate in promoting internet encryption, underscoring GitLab's commitment to a more secure and privacy-respecting online environment.

Microservices architecture is a framework that divides an application into smaller, independent services, each with its own process and database, offering several advantages such as improved scalability, fault isolation, and programming language flexibility. This architecture allows developers to update or scale individual services without affecting others, making it easier to manage demand fluctuations and enhance resilience through fault-tolerant designs using tools like GitLab. Additionally, microservices enable faster deployment and time-to-market, promote data security through secure APIs, and provide outsourcing flexibility by isolating specific functions for third-party partners. However, this framework also presents challenges, including higher upfront costs, the necessity for robust interface control due to numerous APIs, and increased complexity in debugging and integration testing. While microservices share similarities with service-oriented architecture (SOA) in breaking down monolithic components, they differ in scope, with SOA being enterprise-wide and microservices focusing on standalone applications for specific functions, making them more agile and scalable.

The blog post provides a detailed tutorial on securely signing an Android application using GitLab's Project-level Secure Files feature to manage build keys. It begins by instructing how to create a Keystore file with a private signing key using the keytool command, followed by setting up necessary environment variables and updating the app's build.gradle file to include the new signing configuration. The tutorial then guides users on uploading the Keystore file to GitLab, setting CI variables, and configuring the .gitlab-ci.yml file to create a CI/CD pipeline for building the Android app. Additional insights include using prebuilt Docker images for CI systems and managing build artifacts, and it ends by encouraging readers to try the process and look forward to the next part of the series, which will cover iOS code signing.

In summer 2022, GitLab's Vulnerability Research team initiated a Google Summer of Code (GSoC) project aimed at developing a benchmarking framework for Static Application Security Testing (SAST) tools. This framework is designed to evaluate the impact and quality of security analyzers and configuration changes before they are deployed in production environments. The project leverages GitLab's integrated SAST tools, which are encapsulated in Docker images, to translate tool-native vulnerability reports into a standardized format for easier comparison. The benchmarking framework is built to assess the efficacy of these analyzers using a baseline of expected vulnerabilities, allowing for data-driven decision-making and performance monitoring over time. The framework focuses on five key characteristics: relevance, reproducibility, fairness, verifiability, and usability, and aims to be language-agnostic, enabling integration of new SAST tools with minimal configuration. The project establishes baselines for applications written in popular languages and frameworks, such as Java's Spring and Python's Flask, to ensure practical utility. The initiative marks a significant step towards creating a free and open-source software (FOSS) benchmarking framework, encouraging community involvement to expand its scope across more languages and frameworks in the future.

A single-application DevOps platform enhances organizational efficiency by streamlining software development processes, from planning to operation, within one unified system. This integration facilitates faster, more secure software delivery, improving competitiveness and adaptability. The platform offers comprehensive visibility and actionability, allowing teams to gain deep insights into project status and impact, thus empowering them to make informed decisions swiftly without needing synchronous discussions. Features like epics and issues enable effective communication and collaboration, while classification labels and dashboards provide an intuitive means to track project progress and performance metrics. Moreover, value stream analytics offer automatic data insights into workflow efficiency, enabling quick adjustments to optimize project timelines. Overall, a DevOps platform ensures that data is actionable and projects advance rapidly, fostering greater organizational agility and value creation.

Software supply chain security has become a major focus, with organizations generating software bills of materials (SBOMs) but often neglecting the critical aspect of cloud infrastructure management. GitLab and Firefly have teamed up to address this gap by creating bills of materials for cloud infrastructure, known as infrastructure bills of materials (IBoM), which are essential for understanding and securing an organization's entire cloud footprint. As cloud computing and cloud-native applications grow, managing infrastructure through infrastructure-as-code (IaC) becomes vital for maintaining stability, consistency, and security. Tools like Terraform, Pulumi, and Helm aid in codifying cloud assets, allowing them to be managed with the same rigor as application software within a DevOps framework, a practice known as GitOps. Firefly's Cloud Asset Management solution enhances GitLab's GitOps capabilities by providing governance, drift detection, and remediation across an organization's cloud services, automating the codification of cloud assets into IaC, and ensuring that changes are monitored and managed through GitLab's CI/CD processes. This integration helps organizations better secure their software supply chains by providing comprehensive visibility and control over both application and infrastructure software.

GitLab SaaS has introduced two larger compute machine types for CI/CD jobs on Linux, enhancing flexibility and performance for users. These new machine types, powered by Google Compute N2D virtual machines, include a medium option with 2 vCPUs and 8GB RAM, available to all users, and a large option with 4 vCPUs and 16GB RAM, accessible only to paid plans and GitLab for Open Source members. The new machines offer improved performance for compute-intensive tasks, as evidenced by a 41% reduction in execution time for Linux kernel builds on the medium machine compared to smaller types. Users can choose the appropriate machine type for each job by tagging their CI files accordingly, though the large machine type is restricted from free plan users. The cost of CI minutes varies by machine type, and GitLab is working on providing detailed usage reports by runner type to help users manage their resources more effectively. Feedback from users is encouraged to guide future developments.

GitLab has become an official Amazon Linux 2 Service Ready Partner, demonstrating its commitment to providing robust support for Amazon Linux 2 by meeting the rigorous AWS Service Ready program requirements, which include compatibility and security testing. GitLab 15.0 and later versions, including 15.3, support Amazon Linux 2 without any changes needed in the process to achieve this designation. The GitLab Distribution Team utilizes GitLab's own DevOps tools to manage the complex process of building, testing, and distributing packaging for Amazon Linux, involving extensive CI builds, compatibility tests, and security scans. Both x86 and ARM64 architectures are catered for, with ARM64 binaries available since version 12.6.0 and Amazon Linux 2 RPM packages now available for package-based installs. Despite the technical complexity, the announcement stresses the informational nature of the content, noting that the development and release of products and features are subject to change at GitLab Inc.'s discretion.

Mobile development teams encounter specific challenges in implementing DevOps practices due to differences in build tools, slower release cycles, and the need for specialized runners, particularly concerning secure code signing for Android and iOS projects. GitLab addresses these challenges with the introduction of Project-level Secure Files in version 15.0, which facilitates secure storage of sensitive configuration files outside version control and simplifies their management in a CI environment. This feature allows these files to be securely uploaded to a project and easily integrated into CI/CD jobs using the glab CLI tool, ensuring their availability for code signing processes without compromising security. This post is the first in a series focusing on mobile DevOps, with future posts planned to explore further aspects such as setting up code signing for Android applications.

Adopting a microservices architecture can effectively jumpstart a DevOps practice by improving scalability, enhancing fault isolation, and speeding up the delivery of new features. This approach involves decomposing applications into autonomous components, which aligns with business domains and allows for independent management by development teams. Identifying appropriate service boundaries is crucial to avoid unwanted interdependencies. Effective monitoring of metrics such as CPU and memory usage, API response time, and error rates is essential to manage microservices efficiently and prevent downtime. Infrastructure automation and consumer-driven contract testing ensure seamless deployment and version compatibility, while continuous monitoring aids in timely responses to potential issues. By applying the single responsibility principle, microservices architecture enables teams to independently handle their technical stack, deployment strategies, and release schedules, ultimately enhancing the speed of feedback loops and reducing the time to market. Building a strong foundation in system decomposition and CI/CD practices is recommended for successfully implementing a microservices architecture.

Artificial intelligence (AI) and machine learning (ML) are increasingly integrated into DevOps practices, as evidenced by the growing number of teams utilizing these technologies for tasks such as code review and software testing. According to the 2022 Global DevSecOps Survey, 24% of respondents have adopted AI/ML in their DevOps processes, a significant increase from previous years. AI/ML is particularly beneficial in automating repetitive, detail-oriented tasks, making it a valuable tool for overcoming the time-consuming challenges of software testing and code development. However, the integration of AI/ML is not without obstacles, as developers face steep learning curves and rapid technological changes. Despite these challenges, the potential for AI/ML to streamline operations and enhance efficiency makes it a promising area for continued growth and exploration within DevOps, with practices like ModelOps gaining traction as teams seek to merge data science and operations.

Git, a free and open-source version control system, is widely used in the software industry, with 93% of developers reportedly using it according to a JetBrains survey. Despite its prevalence, formal education on Git is not common, leading many to learn it independently or on the job. Various resources are available to aid in learning Git, particularly through visual and interactive methods, such as Oh My Git, a gamified learning tool; Git for Computer Scientists, which offers a deeper understanding of Git internals; Learn Git Branching, which visualizes branch changes in real-time; Explain Git with D3, which provides command-line interface simulations with graphical feedback; and a video by Michael G. Schwern using children's toys to explain Git concepts. Additionally, GitLab offers free educational licenses to institutions for learning and research purposes.

GitLab is focused on enhancing its open-source community by employing a dual-flywheel strategy that emphasizes both internal and external contributions to the platform. To improve the efficiency of handling merge requests (MRs), GitLab has implemented workflow labels like "ready for review," "in dev," and "blocked," which contributors can change themselves to better manage the review process. This initiative has significantly reduced the median time for MRs to transition from "ready for review" to being addressed, decreasing from 17 days to 5 days. Despite some initial challenges with false positives and workflow management, the introduction of these labels and community involvement has led to a record-high number of contributors and MRs merged within a month. GitLab continues to seek improvements through automation and is considering a new program called Leading Organizations to further enhance review cycle times and incentivize recurring contributors. This ongoing effort aims to accelerate innovation and maintain GitLab's competitive edge in the software development landscape.

Nearly 1,400 participants were surveyed over the past year to understand their aspirations and challenges in DevOps, with 83% expressing a desire to learn a new programming language and 15% wanting to improve automation skills. Security emerged as the predominant challenge for over 70% of teams, aligning with findings from a recent Global DevSecOps Survey, while nearly 24% cited testing as a significant issue. For 2022, the primary focus areas were increasing automation, cited by 47% of respondents, and transitioning to a DevOps platform, prioritized by 23%. The text also offers additional beginner resources for those interested in DevOps, including guides and eBooks on topics like Python basics, Rust, CI/CD, and Git.

Navigating GitLab CI/CD's decentralized structure often requires understanding the distinct roles of cache and artifacts in sharing content between jobs. While artifacts are essential when a job relies on outputs from a previous job, caches serve to speed up processes without being crucial to job completion. This guide elucidates how to implement caching in GitLab CI/CD pipelines, illustrating local cache setups using Docker volumes and bind mounts, and detailing distributed cache configurations with S3 for shared access across multiple runners and hosts. By using predefined variables like CI_COMMIT_REF_NAME and CI_JOB_NAME, developers can tie caches to specific branches or jobs, enhancing efficiency. The document aims to simplify complex configurations, offering a visual cheatsheet alongside reference documentation to tackle real-life scenarios, such as setting up caches for jobs within the same stage across different runners and hosts.

Migrating to GitLab's One DevOps Platform offers significant benefits for small and medium-sized businesses (SMBs) by replacing complex and costly DevOps toolchains with a streamlined, end-to-end solution that enhances efficiency and reduces costs. GitLab's platform enables IT teams to focus on software development rather than managing disparate tools, thus increasing productivity, speeding up software creation, and boosting competitiveness. This transition results in financial savings by eliminating licensing fees and reducing the time spent on maintenance and context switching. GitLab's platform also integrates security throughout the development lifecycle, allowing for quicker error detection and correction, thereby saving time and resources. The platform's efficiency fosters faster, more secure development and deployment, enabling SMBs to adapt swiftly to new market demands and remain competitive. Overall, adopting this single, cohesive platform can prevent toolchain complexity and facilitate more frequent and efficient software deployments, as exemplified by companies like Weave, which significantly increased their deployment frequency after making the switch.

In recent years, the unpredictability of global events, such as the pandemic, has highlighted the need for organizations to be flexible and innovative in adapting to evolving user workflows, particularly within the context of DevOps. GitLab has addressed this need by employing the Jobs-To-Be-Done (JTBD) framework, which focuses on understanding users' fundamental goals rather than just the features of their products. This approach allows GitLab to better align their offerings with user needs by examining the entire code integration, verification, and deployment process. The company has re-evaluated its vision for the Pipeline Execution group, responsible for Continuous Integration use cases, by redesigning their research methods to be more collaborative and unbiased. By co-creating an interview deck with users, GitLab identified six stages of user workflow and discovered new opportunities to enhance learnability, monitoring, and discoverability within their tool. As users remain central to GitLab's decision-making process, the company is committed to refining its research and communication practices to capture insights accurately.

Monorepos, which consolidate multiple sub-projects within a single repository, have gained traction for their ability to streamline collaborative development, despite the challenges they pose, such as slow workflows due to their large size and deep histories. Exemplified by large-scale projects like Chromium and Linux, monorepos can become cumbersome as they contain extensive histories and large files, which may be intentional or accidental. To address these issues and enhance efficiency, developers can employ techniques like git sparse checkout, which reduces the files checked out to only what's necessary, and git partial clone, which uses filters to exclude large files from being cloned. These strategies aim to optimize the workflow by minimizing local repository size and update time, thus improving overall productivity.

Adopting a comprehensive DevOps platform is essential for small and medium-sized businesses (SMBs) to alleviate the pressure on their limited IT staff, who often juggle numerous tasks from maintaining outdated systems to ensuring data security. Such platforms streamline processes by automating repetitive tasks, thus reducing human error and freeing IT professionals to focus on creating software that drives business growth. By integrating all necessary tools into one cohesive system, a DevOps platform eliminates the inefficiencies and costs associated with managing a fragmented toolchain. This transition not only enhances collaboration and the efficiency of software development and deployment but also improves employee satisfaction by reducing stress and preventing burnout. The shift to a unified DevOps approach ultimately transforms the chaotic environment of IT into a more manageable, efficient, and productive workspace, which is crucial for the retention and effectiveness of IT personnel in SMBs.

Transitioning from in-person to virtual technology training posed challenges for effective communication, especially regarding the use of whiteboarding to unpack complex concepts. Whiteboarding, as discussed in "The Back of the Napkin," enhances understanding by facilitating two main functions: "selling ideas" through clear explanations and "solving problems" via collaborative drawing. This method leverages progressive disclosure, akin to storytelling, to improve comprehension by gradually building a visual narrative that aids in understanding both detailed and big-picture elements. In the realm of solutions architecture, whiteboarding proves invaluable for fostering collaboration and overcoming language barriers, as visual aids transcend linguistic differences and facilitate collective idea development. While virtual whiteboarding presents logistical advantages, such as preserving visuals and eliminating the need for physical supplies, achieving fluidity in digital drawing remains challenging. Efforts to create a seamless virtual whiteboarding experience involve selecting the right hardware, such as tablets with active pen technology, and software that supports real-time collaboration without disrupting the flow of ideas. Despite hurdles, these tools offer a cost-effective means to enhance remote collaboration, enabling teams worldwide to maintain a shared understanding and innovate effectively.