June 2021 Summaries

The guide provides a comprehensive walkthrough on setting up a self-managed GitLab instance in the AWS Marketplace, detailing the necessary prerequisites, such as an AWS account and a functioning VPC. It covers the step-by-step process of launching GitLab, including configuring security group settings, creating a key pair, and successfully accessing the GitLab login page. Furthermore, the guide elaborates on installing the GitLab Runner and Docker Engine, connecting to the GitLab machine via SSH using the AWS console, and setting up the runner to execute CI/CD jobs. Despite noting that installing runners on the same server as GitLab is not a best practice due to security and performance issues, the guide proceeds with this approach for simplicity. It concludes by explaining how to register the runner and verify its availability under specific runners in GitLab, setting the stage for creating projects and running CI/CD pipelines in future discussions.

The blog post explores the pull-based or agent-based approach to GitOps, particularly focusing on the GitLab Agent for Kubernetes. This method involves installing an agent on infrastructure components, such as Kubernetes clusters, to pull configuration changes from a GitLab repository whenever discrepancies from the desired state are detected. This approach helps maintain security by not requiring exposure of Kubernetes clusters outside the firewall and distributes resource consumption by having agents handle drift correction. The GitLab Agent for Kubernetes consists of two components: the agentk, which is installed on the Kubernetes cluster, and the kas, which resides on the GitLab instance and manages authentication, authorization, and change monitoring. The post illustrates a GitOps workflow where users create and update Infrastructure as Code configurations, with changes automatically applied to the infrastructure by the agentk. Despite its benefits, this approach necessitates maintaining agents on all infrastructure components, which can consume resources. The article concludes by announcing the next part of the series, which will cover the push-based or agentless approach to GitOps.

GitLab 14 epitomizes the evolution of DevOps by offering a comprehensive platform that integrates all aspects of the software development lifecycle to enhance speed, trust, and visibility. This iteration of GitLab addresses the limitations of the DIY DevOps era by providing a cohesive solution that improves efficiency and collaboration, particularly in hybrid and remote work environments. It offers advancements in continuous integration and delivery, infrastructure as code, and security, with features like the Pipeline Editor and Kubernetes Agent, making it easier for teams to manage cloud-native deployments. GitLab 14 emphasizes the importance of metrics, such as DORA metrics, for tracking and improving software delivery performance, thereby driving better business outcomes. The platform’s built-in security features align with the principles of DevSecOps, ensuring that security is integrated seamlessly into the development process. GitLab's open core model has fostered significant community contributions, enriching the platform with diverse inputs from over 10,000 merge requests since 2016. With over 30 million registered users, GitLab continues to empower organizations of all sizes to deliver software efficiently and securely, underscoring its commitment to inclusivity and innovation in the DevOps landscape.

CI/CD pipelines are essential for integrating code changes efficiently and reducing potential risks in software development. By leveraging GitLab CI's features such as Directed Acyclic Graphs (DAG) and parallel jobs, developers can enhance execution speed and productivity by allowing jobs to run concurrently or with direct dependencies, thus minimizing wait times. The introduction of parallel matrix jobs in GitLab 13.3 allows multiple job instances to be created at runtime based on variable values, streamlining large pipeline management. Parent-child pipelines and merge trains further optimize pipeline performance by reducing merge conflicts and ensuring stable target branches. Additionally, GitLab's ability to configure multiple cache keys per job enhances pipeline performance, while efficient CI/CD strategies save both development time and costs, improving code quality and boosting team morale.

A recently identified configuration bug in self-managed GitLab instances impacted the ability to disable Usage Ping via text configuration files, leading to the unintended collection and transmission of Usage Ping events for versions 9.3 to 13.12.3, which was rectified in version 13.12.4. This issue does not affect users who disabled Usage Ping through the user interface or those using the SaaS version of GitLab. Users are advised to check their configurations to determine if they were affected and are encouraged to fill out a form if they suspect unintentional data sharing, allowing GitLab to purge the data and confirm the action. Additional details and instructions are available, and users can engage with GitLab's community forum for further questions and feedback.

GitLab recently discovered a security lapse involving the GPG key and other tokens used for signing and distributing GitLab Runner packages, which were not secured in accordance with their security policies. Although no unauthorized modifications or access to packages were found, GitLab has proactively rotated the GPG key and all related tokens to ensure security. The old key, identified by the fingerprint 3018 3AC2 C4E2 3A40 9EFB E705 9CE4 5ABC 8807 21D4, has been revoked, and a new key, with the fingerprint 09E5 7083 F34C CA94 D541 BC58 A674 BF81 35DF A027, is now in use for signing packages starting June 13, 2021. Users who utilize package signature verification for DEB or RPM packages must update their key to ensure continued security, while those not using verification remain unaffected. GitLab advises treating the old key as compromised and recommends using the new key for trustworthiness. New users are not impacted by this change but should ensure they have the new key if they wish to use package signature verification. Further details and configuration information can be found on GitLab's documentation site, and additional queries can be directed to GitLab Support.

The GitLab Environment Toolkit (GET) is an innovative solution designed to automate the deployment of GitLab at scale, supplementing the performance testing facilitated by the GitLab Performance Tool (GPT). Developed by the Quality Engineering Enablement team, GET simplifies provisioning and configuration by utilizing open-source tools like Terraform and Ansible to deploy GitLab Omnibus and Helm Charts across cloud providers such as GCP, AWS, and Azure. This toolkit supports various GitLab Reference Architectures, including dynamic deployment from 1,000 to 50,000 users, cloud-native hybrid architectures, and features like zero downtime upgrades, built-in load balancing, and monitoring capabilities. Emphasizing simplicity and maintainability, GET is not intended to replace existing installation methods but rather to streamline the setup process for scalable GitLab environments, with ongoing developments planned to expand its capabilities and address customer feedback.

GitLab, widely recognized as a comprehensive open-source DevOps platform, is also being used creatively by hackers and bug bounty hunters for project management and collaboration. Through GitLab's issue tracking and labeling features, hackers like Alex Chapman, Dominic Couture, and Nishant Jain streamline their bug-hunting processes, from idea conception to reporting and tracking payouts. Alex uses GitLab for meticulous note-taking, while Dominic collaborates with friends by documenting potential vulnerabilities. Nishant appreciates GitLab's friendly features and uses it for hosting blogs and collaborating on HackerOne programs. The platform's tools, such as markdown support and issue boards, are highlighted for their utility in organizing and tracking hacking projects. Despite some areas for improvement, particularly in mobile performance and markdown editing, GitLab remains a valuable resource for hackers seeking to manage their workflows efficiently.

In the digital age, organizations must embrace digital transformation to remain competitive, often transitioning to a DevOps model to enhance development and infrastructure processes. While development has long benefited from DevOps practices, infrastructure has lagged behind, particularly in speed, but the use of Infrastructure as Code (IaC) and GitOps is changing this by enabling infrastructure teams to apply application code management disciplines to deliver products faster and more predictably. Security, however, can be a bottleneck, especially when testing is tacked on at the end of the delivery cycle. To address this, the integration of Indeni Cloudrail with GitLab CI/CD allows for IaC security checks to be shifted left, integrating security controls early in the development lifecycle and automating infrastructure compliance. This shift-left approach minimizes the disruption caused by noisy security tools, as Indeni Cloudrail uses context-based analysis to reduce false positives, ensuring only critical security concerns are flagged. By combining their strengths, Indeni and GitLab support organizations' digital transformation efforts by streamlining IaC security within the familiar GitOps framework, promoting faster and more reliable deployments.

Development teams are increasingly integrating security measures into their DevOps workflows, focusing primarily on identifying and addressing vulnerabilities during the development phase. However, protecting client-side application code at runtime is crucial to mitigate additional threats not directly linked to vulnerabilities, such as tampering and reverse engineering. This tutorial highlights the integration of Jscrambler with GitLab to enhance code protection by employing techniques like obfuscation, runtime protection, and threat monitoring. The integration process involves setting up a GitLab CI environment where JavaScript-based web or mobile apps are protected using Jscrambler's layered approach. The tutorial guides users through configuring Jscrambler's protection techniques, managing API keys securely, and establishing a protect stage in the GitLab CI pipeline to ensure source code is safeguarded before deployment. This comprehensive approach not only obfuscates the code but also provides robust runtime defenses such as anti-tampering and anti-debugging capabilities, making reverse-engineering attempts more challenging.

GitLab's DevOps platform integrates continuous integration (CI) and continuous delivery (CD) through GitLab CI/CD, enhancing developer productivity by automating tests with each code commit. While setting up CI for Linux-based systems is straightforward, iOS and macOS developers face challenges due to limited access to 24/7 Mac systems. The blog post provides a tutorial for setting up CI for iOS and macOS development using Scaleway's Mac mini M1 and GitLab Runner, assuming familiarity with Xcode, GitLab, and terminal use. It covers the deployment of a Mac mini M1 for CI tasks, installation of the Homebrew package manager, and configuration of GitLab Runner, linking it to a GitLab instance via a registration token. The guide also explains creating a .gitlab-ci.yml file for specifying CI tasks, thereby streamlining the development workflow for iOS and macOS applications on Scaleway.

GitLab 14.0, the major release for 2021, introduces various new features and planned deprecations aimed at enhancing workflows, performance, and scalability. Scheduled to officially launch on June 22, the release affects both GitLab.com and self-managed users, with significant changes including the removal of deprecated GraphQL fields, a shift in the default Git branch name to "main," and updates to the GitLab Runner and OAuth flows. Several legacy features such as GitLab Managed Apps, the Web Application Firewall (WAF), and the Clair container scanning engine are being phased out, replaced by more efficient alternatives like Trivy for container scanning. Additionally, enhancements to CI/CD templates, including Auto DevOps, Terraform integrations, and SAST variables, aim to improve user experience and maintainability. The release also brings changes to API configurations to streamline processes and uphold performance standards, prompting users to review and update their existing setups to align with the new version's requirements.

An Engineering Manager at GitLab highlights the benefits and challenges of managing a globally distributed engineering team, emphasizing how such a setup can facilitate "continuous development" for rapid-response scenarios. The team, distributed across countries like India, Germany, Australia, and the U.S., tackled an abuse issue on GitLab.com, utilizing time zone differences to work around the clock, thereby reducing project delivery time. However, this approach necessitates careful management, including clear work handoffs to prevent duplication and well-documented interfaces to avoid confusion, as miscommunication can lead to delays. The manager advocates for synchronous communication to be managed by team leaders rather than forcing team members to work off-hours, suggesting that meetings be recorded for those in different time zones. While continuous development can expedite critical projects, it requires balancing trade-offs, such as increased management overhead, and should be evaluated through retrospectives to assess its effectiveness against the additional effort required.

GitLab champions the integration of security into the DevOps process, coining it as DevSecOps, to ensure security is a central and transparent element of software development. The company emphasizes the automation of security testing within the CI pipeline, allowing developers to address vulnerabilities during code iteration, thus enhancing efficiency and collaboration between development and security teams. GitLab's end-to-end DevSecOps platform consolidates tools, enabling seamless license management and integration, which reduces friction and improves communication. Recognized by Gartner as a challenger in the application security testing market, GitLab offers a wide range of scanning types and vulnerability management capabilities, positioning itself as a leader in the evolution of application security testing. The company's recent acquisitions and feature releases underscore its commitment to innovation and addressing the evolving demands of modern application architectures, particularly in light of heightened cybersecurity awareness following significant cyberattacks and governmental directives.

GitLab has been enhancing its Elasticsearch integration to enable Advanced Global Search features for GitLab.com users, addressing challenges like scalability and efficiency in data migration processes. Initially, reindexing large GitLab instances posed significant issues, prompting the development of a more efficient migration strategy inspired by Rails DB migrations. The new approach includes automated, background-run migrations with a focus on minimizing downtime and strain on system resources. This involves utilizing timestamps for migration files, storing migration data within Elasticsearch, and implementing various configuration options such as batch processing and space requirement checks. Since its implementation in the 13.6 release, the framework has improved performance by successfully migrating various document types like issues and merge requests, while also emphasizing the importance of regular updates to align with ongoing enhancements.