February 2022 Summaries

As software evolves, increasing complexity often necessitates splitting projects into smaller, cohesive components, with GitLab CI/CD offering two main approaches to manage this: parent-child pipelines and multi-project pipelines. Parent-child pipelines are useful for managing distinct components within a monorepo by allowing independent execution of UI and backend jobs, thereby optimizing efficiency and control. Conversely, multi-project pipelines cater to applications spread across multiple repositories, enabling independent pipelines to be chained together to ensure comprehensive integration. While parent-child pipelines run within the same project context and offer greater control over configurations, multi-project pipelines operate in separate contexts and are treated as external logic. Both approaches have unique benefits and challenges, and GitLab is focusing on enhancing these systems by improving visibility, cascading cancellations, and variable management across pipelines.

In 2021, GitLab introduced several notable enhancements to its DevOps Platform, focusing on improved functionality, security, and integrations. Key updates included the introduction of Epic Boards, which offer a customizable interface for tracking project progress, and tighter integrations with tools like Visual Studio Code and Gitpod, enhancing developer productivity. Security received significant attention with the development of a comprehensive security dashboard and upgrades to static application security testing (SAST). Additionally, GitLab launched features like Praefect for Gitaly to ensure high availability, a visual pipeline editor for better CI/CD configuration visualization, and the GitLab Agent for Kubernetes to streamline cloud-friendly integrations. Looking ahead to 2022, GitLab plans to enhance operational capabilities with a focus on observability following its acquisition of Opstrace, promising further innovations in its platform.

Infrastructure as Code (IaC) is transforming the management of computer resources by automating processes and abstracting cloud provisioning, though it presents unique security challenges such as the risk of leaked credentials and supply chain attacks. This comprehensive guide explores the specific vulnerabilities associated with IaC, focusing on tools like Terraform, cloud providers, and deployment platforms using containers and Kubernetes. It emphasizes security best practices, including the use of scanners like tfsec, kics, checkov, terrascan, and semgrep to detect vulnerabilities in IaC configurations. The guide also discusses the integration of these tools into CI/CD pipelines and the importance of thinking like an attacker to identify potential security threats. It provides practical examples and encourages the development of more IaC scenarios to test vulnerabilities, suggesting that using multiple scanners can enhance detection capabilities. Additionally, it addresses the use of GitLab's IaC Security Scanning feature, highlighting its integration potential and the value of customizing detection methods to suit specific security needs.

GitLab employs a comprehensive approach to handling security vulnerabilities, emphasizing transparency and community involvement. As an open-core platform, GitLab allows businesses to either use GitLab.com or self-manage their GitLab instances, with security being a shared responsibility across the organization and its users. The platform encourages the reporting of security issues through its HackerOne Bug Bounty Program and has established processes for timely resolution of these issues. Security patches are automatically applied for GitLab.com users, while self-managed customers must update their instances, informed by Common Vulnerabilities and Exposures (CVEs) that GitLab assigns to each vulnerability. These CVEs, with severity scores based on the Common Vulnerability Scoring System, guide the prioritization of fixes. Despite concerns over frequent disclosures, GitLab prioritizes transparency, assigning CVEs to all vulnerabilities and publicly disclosing details within 30 days of patching. The company’s commitment to public disclosure aims to fortify supply chain security by enabling other organizations to learn from GitLab’s proactive practices, thereby enhancing overall software security.

GitLab's Dependency Scanning feature helps detect vulnerabilities in software dependencies by utilizing the GitLab Advisory Database, which is updated by the Vulnerability Research team to include security advisories with CVE identifiers and malicious packages. This feature is available in GitLab Ultimate self-managed and SaaS versions, with a free open-source edition offered as a time-delayed clone to foster collaboration and transparency. The advisory database is well-documented for easy adoption and integration, and it uses Common Weakness Enumeration and Common Vulnerability Scoring System standards to communicate vulnerabilities effectively. Integration with tools like Trivy enhances container scanning, and community contributions are encouraged to improve the database, with a community-sync flag enabling rapid updates. Notably, critical vulnerabilities like log4Shell are quickly shared with the community to mitigate widespread disruption.

Embracing the inevitability of becoming a software-driven entity, even for companies not traditionally in the software industry, is vital due to the demands from competitors, customers, investors, and employees for regular software updates. Continuous integration and continuous delivery (CI/CD) pipelines are essential for streamlining code development and delivery through automation, enhancing code quality, accelerating release rates, and reducing costs. These pipelines enable rapid identification and isolation of code faults, simplified rollbacks, and continuous feedback, offering transparency and accountability in the development process. CI/CD also improves mean time to resolution, aids in monitoring metrics, and reduces non-critical defects, ultimately saving time and money while allowing developers to focus on refining code efficiency. Notably, the success story of Goldman Sachs demonstrates the potential of efficient CI/CD pipelines, transforming their code build frequency from bi-weekly to over a thousand times per day.

The integration of the scientific process with the DevOps lifecycle is transforming research methodologies by enhancing collaboration, transparency, and efficiency. Organizations like the European Space Agency (ESA) and research teams at MathWorks and the Square Kilometre Array Organisation (SKAO) demonstrate how using GitLab and similar DevOps tools can streamline the management of complex projects, from mission control systems to model-based design testing. By adopting the iterative and collaborative nature of DevOps, researchers can better manage hypotheses, data, and scripts within source control management systems, ensuring reproducibility and integrity. This transformation shifts research from siloed, sequential efforts to concurrent, transparent collaborations, allowing for real-time documentation and review of all stages of the research lifecycle. Such integration not only reduces redundancy and development time but also improves quality assurance and audit-readiness, exemplifying a new paradigm in scientific inquiry that could extend benefits across various fields.

GitLab's blog post outlines its strategy for enhancing software supply chain security (SSCS) by providing comprehensive documentation and a framework that covers various aspects of the software development lifecycle, including source code protection, build environment security, and tool management. Emphasizing the importance of a seamless and secure end-to-end process, GitLab aims to integrate these elements within a single application, aligning with its Single Application Strategy to reduce security risks and improve traceability for audits. The SSCS framework, inspired by industry standards and customer feedback, focuses on five key areas: source, build, consumption, management process, and tool security. GitLab's platform is designed to collaborate with other security tools, offering a turnkey solution for maintaining a secure software supply chain, and encourages feedback to refine its approach while continuing to explore partnerships with leading technologies in the space.

Machine learning is increasingly being integrated into DevOps to enhance various processes, despite the technology still being in its early stages. It can aid in analyzing and interpreting test data, managing help-desk alerts by automating routing and even solving issues based on predefined rules, and enhancing security by detecting breaches in real time through analysis of network traffic and security logs. Machine learning also helps in gathering user requirements using natural language processing, optimizing project management, and providing development recommendations based on past projects. It automates testing and code reviews, improves communication across teams by reducing process complexity, and anticipates provisioning needs to save time. Furthermore, machine learning enhances software quality by identifying issues before production and integrates workflows for continuous improvement, learning from both provided and self-acquired training models to ensure better products and services over time. As DevOps teams increasingly embrace machine learning, skills related to AI or ML are becoming crucial for developers, as highlighted by GitLab's survey findings.

DevOps is a methodology that unites software development and IT operations into a cohesive team with the goal of accelerating the software development lifecycle while maintaining secure code. The core principles of DevOps include automation, collaboration, continuous improvement, and a focus on user needs, which are essential for streamlining processes like testing and deployments. Originating around 15 years ago and coined by Patrick Debois in 2009, DevOps incorporates Agile philosophies and aims to dismantle silos between development and operations teams. While DevOps offers benefits such as faster code release, improved code quality, and increased developer satisfaction, its implementation can be challenging due to issues like integrating legacy systems, managing toolchains, and overcoming communication barriers. Despite these challenges, the methodology's popularity has surged, particularly during the global pandemic, as organizations adopt advanced technologies such as Kubernetes and AI/ML to further enhance their DevOps practices.

A software engineer explores the viability of using an iPad as a primary coding device by leveraging the capabilities of Gitpod and GitLab to develop and deploy a Twitter bot. Despite initial challenges, the iPad, paired with a magic keyboard, enables the engineer to create a development environment through Gitpod, which provides browser-based access to a full suite of development tools and a VS Code-like interface. This setup allows the engineer to write, test, and preview code effectively, while GitLab CI/CD facilitates seamless deployment to platforms like Heroku. The process highlights the advantages of using Gitpod for quick environment setup, enabling collaboration, and simplifying onboarding with a .gitpod.yml file that specifies necessary tools and configurations. This approach not only eases the development and deployment processes but also promotes efficient collaboration by allowing new developers to quickly spin up environments tailored to project needs.

GitLab's recent blog post highlights the company's commitment to enhancing security, compliance, and governance within the software development lifecycle, emphasizing the importance of shifting compliance left, just as with security. It discusses the challenges posed by high-profile software supply chain attacks and the increasing demand for tighter controls, especially in cloud-native environments. GitLab's innovations aim to simplify security and compliance through an end-to-end DevOps platform by introducing infrastructure as code scanning, flexible container scanning, and API security measures. The post also outlines the benefits of a unified DevOps platform, offering new tools for continuous compliance, policy management, and vulnerability management. GitLab plans to further enhance its offerings with upcoming features like production container scanning, automated compliance checks, and intelligent code security, while encouraging community contributions to its transparent product roadmap.

Pj Metz, the education evangelist at GitLab, has embarked on a journey to learn Python to better engage with the educational community, which frequently uses the language. With prior coding experience in C#, JavaScript, HTML, and CSS, Metz is leveraging Codecademy, an online learning platform, to teach himself Python and document his progress. He highlights the differences in syntax and variable handling between Python and other languages like C# and JavaScript, noting Python's dynamic typing and readability, which initially felt unusual but became more intuitive with practice. Metz appreciates Python's syntax for its clarity, such as using colons and indentation for control flow instead of curly braces, and its ability to create multi-line strings with triple quotes. As this is his first foray into Python, Metz shares his beginner's perspective and anticipates making mistakes, emphasizing the learning process over perfection.

The blog post outlines a tutorial on using GitLab as a GitOps tool, specifically focusing on leveraging GitLab's Auto DevOps feature to manage deployments. It assumes prior knowledge from earlier tutorials, including having a Kubernetes cluster connected to GitLab. Auto DevOps, described as GitLab's approach to simplifying software delivery, offers opinionated templates that can streamline pipeline building, covering processes from testing to deployments. The tutorial provides a step-by-step guide to deploying a minimal "hello world" application using Auto DevOps, highlighting how minimal setup is required to initiate complex pipelines with GitLab CI, while also addressing the flexibility for customization based on different use cases. The article concludes by hinting at future content that will explore deploying applications using a GitOps style workflow, emphasizing the ease with which development teams can manage application code and basic testing without delving into extensive security or compliance configurations.

Contributing to open source projects offers individuals a valuable opportunity to learn, teach, and expand their technical expertise while becoming part of a community, though it can initially be intimidating. Contributions are not limited to coding; they can include designing, writing documentation, managing issues, and more, allowing individuals to align their involvement with their interests and skills. Understanding the unique culture, governance models, and potential codes of conduct within each project is crucial, as norms and terminologies vary, and projects may follow structures like BDFL, meritocracy, or liberal contribution. Collaboration, respect, and patience are key, especially given the diversity within the community and the importance of kindness and collaboration. Engaging with open source requires observing community norms, asking thoughtful questions, and ensuring support from project implementers before contributing code, all of which help in building a positive and productive experience.

In 2021, GitLab experienced significant advancements in server-side Git fetch performance, particularly for CI, through the implementation of the pack-objects cache and other efficiency improvements. The pack-objects cache emerged as a solution to address the limitations of the CI pre-clone script, offering an integrated caching layer that deduplicates identical Git fetch requests, thereby reducing CPU usage. This cache, although initially disabled by default due to increased disk write throughput, proved beneficial for optimizing Git fetch responses and was later enhanced by a new RPC transport system for Git HTTP traffic, effectively circumventing bottlenecks related to gRPC-Go's memory allocation. Additionally, improvements in Git itself, such as optimizing reference scanning and increasing buffer sizes, contributed to these performance gains, reflected in GitLab versions from 14.4 onwards. Despite challenges, these efforts resulted in more efficient handling of Git fetch data and an overall boost in GitLab's server-side capabilities.

As of July 1, 2022, GitLab requires all public open source projects on the Free tier to apply to the GitLab for Open Source Program to continue receiving GitLab Ultimate benefits. This program, created to support the open source community, provides eligible projects with access to GitLab's top-tier features including unlimited seats, 50,000 CI/CD minutes, and various security and compliance tools at no cost. To qualify, projects must meet specific criteria, such as having an OSI-approved open source license, being non-profit, and maintaining public visibility. Although free accounts do not include support, discounted support can be purchased, and program membership requires annual renewal. Interested parties must apply through a process that includes setting up a GitLab account, preparing project screenshots, and submitting an application form, with responses typically provided within 10 business days. Acceptance is at GitLab's discretion, and the program terms are subject to change.

Over the past five years, the complexity of integrating various DevOps tools has led organizations to reevaluate their approach to building internal platforms, resulting in the concept of a "Developer Self-Service Platform." Initially, companies attempted to meet DevSecOps goals by forming teams to manage tools like Jira, GitHub, and Jenkins, but faced challenges in tying these products together due to a lack of custom code expertise among operations staff. Executives now face the daunting task of leveraging 10 to 50 disparate tools to gather data and analytics for business optimization, a process that has proven costly and time-consuming, with some Fortune 100 companies investing over $10 million without yet delivering a comprehensive dashboard to executive teams. To address these challenges, businesses must conduct value stream assessments to determine platform requirements, focusing on reducing touchpoints, enhancing quality frameworks, and ensuring compliance. The article suggests that instead of continuing to build platforms in-house, organizations should consider using integrated solutions like GitLab to meet their requirements more efficiently, thereby capturing a return on investment and preventing unnecessary costs associated with the addition of each new tool.

Continuous integration and continuous delivery (CI/CD) are essential components of a successful DevSecOps strategy, streamlining software development through automation and enabling faster, more reliable deployments. CI focuses on automating code integration and builds, while CD automates testing and deployment, reducing manual intervention and tool complexity. Key best practices include utilizing a unified DevSecOps platform, automating processes, committing code frequently, and integrating security early in the pipeline for a shift-left approach. Additional strategies like feature flags, comprehensive monitoring, and maintaining pipeline as code enhance the pipeline's effectiveness. Metrics such as the DORA framework help measure CI/CD success, linking technical performance to business outcomes like faster deployment and improved customer satisfaction. Implementing CI/CD requires careful planning and collaboration across development teams to ensure an efficient, consistent process that ultimately benefits customers and internal stakeholders alike.

GitLab Ultimate offers a Vulnerability Report feature, providing insights into vulnerabilities detected in the default branch of projects, groups, and the Security Center. Users can filter, view, and manage vulnerabilities, including accessing detailed information, changing statuses, and exporting data. The GitLab GraphQL API enables further management of vulnerabilities through functions such as creating, reading, updating, and deleting, with queries for data retrieval and mutations for data modification. The API facilitates efficient data requests by allowing clients to specify the exact data needed. Additionally, the text explains how to run a GraphQL query via a curl request to create a vulnerability, detailing the steps for setting up a project, generating a personal access token, and making a POST request with the necessary authorization. The text also provides guidance on building a basic web application using the GraphQL API to report vulnerabilities, deploying it on Kubernetes, and managing it through GitLab's Kubernetes-Agent, demonstrating the integration of GitLab's security features with cloud-based infrastructure for comprehensive vulnerability management.

Many companies have yet to fully explore the potential of non-traditional paths in DevOps hiring, often relying heavily on university graduates despite the diverse talent available outside traditional educational routes. Alternative pathways such as coding bootcamps and self-taught learning are becoming more prevalent, providing intensive and cost-effective training in specific tech skills and producing candidates with unique problem-solving abilities and perspectives. Non-traditional candidates often bring valuable experiences and soft skills from previous careers, such as communication, teamwork, and project management, which are crucial for DevOps roles. Additionally, diversifying hiring practices can address under-representation in the tech industry, offering opportunities to women and people of color who may not follow conventional academic paths. Emphasizing skills and practical knowledge over formal education can lead to a more dynamic and effective DevOps team.

The CI/CD Tunnel, utilizing the GitLab Agent for Kubernetes, provides a secure method for accessing Kubernetes clusters from GitLab CI/CD jobs without exposing the cluster to the internet. This blog post details how to use generic impersonation to securely manage access to clusters by leveraging Kubernetes' RBAC rules and fine-grained permission controls. The GitLab Agent for Kubernetes, recently enhanced by GitLab, aids in monitoring and troubleshooting cluster events through its activity list. The post illustrates the practical application of these features by demonstrating the setup of a GKE Kubernetes cluster and configuring projects within GitLab to use the CI/CD Tunnel for deploying and managing Kubernetes-native applications. The use of impersonation is highlighted to restrict access at various levels, including impersonating specific users or CI jobs, ensuring that permissions align with the organization's security policies, thereby reducing the risk of unauthorized access to cluster resources.