November 2023 Summaries

Atlassian is ending support for its Server products, including Bitbucket Server, in February 2024, prompting a need for users to migrate to alternatives such as GitLab. GitLab has enhanced its importers for Bitbucket Server and Cloud, addressing previous issues like timeouts during large project imports by introducing parallel, asynchronous importers that distribute tasks across multiple background jobs, thus avoiding rate limiting and improving error handling. These updates, implemented in various GitLab versions, ensure a smoother transition by improving data integrity and extending the types of data imported, such as merge requests, pull request approvals, and LFS objects. For Bitbucket Cloud, specific improvements include fixing issues with unassociated commits in merge requests and refining reference links to be more compatible with GitLab's system. These enhancements make transitioning from Bitbucket to GitLab more efficient and accessible for users, with detailed documentation available to guide the process.

GitLab's Red Team uses open-source command and control (C2) tools to simulate real-world threats and evaluate security measures. The team customizes these tools to bypass detections, applying professional development practices for continuous testing within the Mythic framework. They leverage GitLab CI/CD pipelines to automate testing after code changes, enabling rapid validation of updates to Mythic and its compatible agents. The public project they share includes a suite of pytest tests for integration with Mythic, supporting the iterative development of secure operations. The article also highlights the importance of continuous testing, demonstrated when a bug in Poseidon's upload function was quickly identified and resolved, showcasing collaboration with the community of C2 developers who open source their tools. GitLab encourages feedback and contributions to further enhance the security testing framework.

GitLab will be participating in AWS re:Invent 2023 in Las Vegas from November 27 to December 1, showcasing how its DevSecOps Platform on Amazon Web Services (AWS) enhances secure, enterprise-grade AI integration throughout the software development lifecycle. The event will feature various activities, including lightning talks, live demos, and customer sessions, with topics such as frictionless developer experiences, new integrations between GitLab and AWS, and secure Terraform development using GitLab's security scanning policies. GitLab will also highlight its collaboration with AWS on service integrations like AWS CodeStar Connections and Amazon CodeGuru Security, which enable streamlined development and security processes via GitLab's platform. Additionally, GitLab's single-tenant SaaS solution for regulated industries, GitLab Dedicated, will be discussed, emphasizing its deployment on AWS to meet compliance and data residency requirements. Attendees can engage with GitLab at Booth #1152 and access some sessions on-demand after the conference.

GitLab's Red Team, initially established in 2019, has evolved from conducting visible, opportunistic security exercises to executing stealth operations that emulate real-world threats, offering the organization a realistic practice environment for detecting and responding to attacks. This transition involved developing a maturity model and implementing Purple Teaming to promote collaboration between offensive and defensive security teams, ultimately paving the way for stealth operations. These operations include replicating adversary techniques to test and improve the organization's detection and response capabilities, using both transparent processes and strategic collaborations with various teams at GitLab. The Red Team's journey highlights the importance of structured planning, transparent communication, and iterative learning in building an effective offensive security practice, emphasizing the value of stealth operations in simulating realistic cyber attack scenarios.

Dunelm, a major UK home furnishings retailer, has successfully fostered a DevSecOps culture that enhances collaboration and security by integrating GitLab's comprehensive platform. This cultural shift, championed by Paul Kerrison and Jan Claeyssens, emphasizes breaking down barriers between developers and security engineers, eliminating the "us-versus-them" mindset, and embedding security into development processes. The adoption of GitLab since 2020 has facilitated increased productivity and efficiency, enabling Dunelm's teams to share responsibility for security and work cohesively towards business improvement. By including security team members in leadership roles and utilizing GitLab's automation features for compliance and vulnerability scanning, Dunelm has positioned itself to leverage AI tools in the future for tasks such as code generation and real-time feedback. This holistic approach to technology and security supports Dunelm's extensive operations, which include distribution centers, 178 stores, and a significant ecommerce platform handling over 12 million transactions annually.

GitLab has introduced a new Enterprise Agile Planning add-on designed to enhance collaboration by integrating non-technical users into the DevSecOps platform used by engineers for building, testing, securing, and deploying code. This add-on allows non-engineering team members to engage in planning workflows, track software delivery performance with Value Stream Analytics, and utilize executive dashboards without needing full GitLab licenses, available at $15 per user per month for GitLab Ultimate subscribers. The platform supports cross-team collaboration and planning, providing a unified view of project status to help identify bottlenecks and make informed decisions, which is crucial as teams scale and agile planning extends to include stakeholders like product managers and marketing. GitLab also integrates compliance checks into the development process, enhancing efficiency and transparency, and offers a streamlined approach to Agile planning tools to reduce maintenance time and boost innovation. This approach is designed to simplify the planning process and foster a collaborative, transparent, and efficient environment, facilitating faster software delivery and offering a competitive edge in the digital landscape.

HackerOne, a cybersecurity company, has significantly enhanced its developers' experience and productivity by adopting the GitLab DevSecOps Platform, which integrates source code and issues management, CI/CD, and security compliance. Since its adoption in 2018, the company has seen a 50% increase in merge requests per engineer and a 50% reduction in code release time, with quarterly releases to production rising by 73%. This transformation has been attributed to the streamlined CI/CD pipelines within GitLab, which have improved efficiency and reduced friction for developers, leading to increased job satisfaction. Additionally, the platform strengthens security by limiting system access and ensuring that changes are reviewed by experts, maintaining compliance with industry standards like FedRAMP. Looking ahead, HackerOne plans to leverage artificial intelligence to analyze vast amounts of data, aiming to extract valuable insights for their clients, reflecting a broader industry trend towards operationalizing AI.

GitLab's Dynamic Application Security Testing (DAST) scan is a tool designed to identify security misconfigurations in actively running web applications, particularly those using JavaScript and single-page applications. The scan requires precise configuration to accommodate the web application's specific attributes, such as authentication mechanisms and page load times. Users should ensure the scan targets a test environment, not a production one, and that timeouts are adjusted to allow the scan to complete. For applications requiring authentication, variables like DAST_USERNAME and DAST_PASSWORD should be set at the project level, and the scan must be configured to navigate various login forms, including multi-step processes or modals. The scan also requires specific field variable values for username, password, and submit actions, with recommendations to use resilient selectors like 'id' and 'name' attributes. Additional configurations, such as DAST_BROWSER_ACTION_STABILITY_TIMEOUT and DAST_BROWSER_MAX_RESPONSE_SIZE_MB, can be adjusted to handle slow response times or large JavaScript files. Troubleshooting tools, including authentication reports and analyzer logs, are available to assist in resolving scan issues, and users are encouraged to utilize GitLab's free trial for further exploration.

GitLab Duo Chat is an AI-assisted feature designed to enhance software development processes by providing real-time support for both technical and non-technical users throughout the entire development lifecycle. Launching in Beta on November 16, it integrates with GitLab's Web IDE and VS Code extension, offering capabilities such as code explanations, test generation, and interactive code creation. This tool aims to streamline workflows, elevate productivity, and support users in understanding and managing code, issues, and project epics without needing to switch contexts. GitLab Duo Chat emphasizes data privacy by not using proprietary inputs as training data and employs specific large language models like Anthropic Claude-2 and Vertex AI Codey for various tasks. The feature is part of GitLab's broader initiative to incorporate AI into software development, which 83% of respondents in their Global DevSecOps Report view as critical to maintaining competitiveness. GitLab Duo Chat will be available for Ultimate tier customers as part of GitLab 16.6, with future updates expected to transition other AI capabilities from Beta to general availability.

Learning a new programming language can be enhanced with the use of AI-powered tools, like GitLab Duo Code Suggestions, which offer a guided experience, particularly useful in mastering Python through hands-on examples. The tutorial emphasizes setting up a Python development environment, using tools like VS Code and GitLab Workflow extension, to explore fundamental concepts such as variable definitions, flow control, file I/O operations, and more. The process involves practical exercises to build a command-line application that parses log files, illustrating the use of regular expressions and data structures to handle different log formats, including structured logging. Additionally, the role of automation through CI/CD pipelines is highlighted, promoting dependency management and continuous code verification, while encouraging learners to experiment with various Python features, share feedback, and continue learning asynchronously through exercises like implementing log file limits, using search filters, and parsing CI/CD job logs.

GitLab has embarked on a project to enhance the GitLab Runner Fleet dashboard, aiming to improve visibility, usability, and support for managing self-managed runners in continuous integration and deployment (CI/CD) processes. Historically, the GitLab Runner, which executes CI/CD jobs, was limited in user interface capabilities, prompting a user-centric redesign focused on offering detailed insights into runner system failures, runner load, and wait times for job execution. Through problem validation research and moderated usability testing, GitLab identified key features such as the importance of quickly identifying runner failures, understanding runner load, and reducing wait times for job execution. The updated design includes visualizations for wait times, lists of the busiest runners, and summaries of recent failures, providing administrators with a comprehensive view of runner performance and potential issues. This initiative is part of ongoing plans to iterate and improve the dashboard, with feedback being solicited from an Early Adopters Program to ensure the enhancements align with user needs and contribute to a more efficient runner management experience.

Gitaly, the service that manages Git repository access in GitLab, has implemented new strategies for maintaining repositories more efficiently, especially important for large monorepos. Traditional methods of repository maintenance, such as compressing loose objects into a single packfile, were resource-intensive and often inefficient. By leveraging recent Git updates, Gitaly has adopted geometric repacking and cruft packs to address these inefficiencies. Geometric repacking, introduced in Git v2.32.0, allows multiple packfiles to be merged without rewriting them all into one, maintaining a geometric sequence to limit packfile numbers. Cruft packs, from Git v2.37.0, enable storage of objects pending deletion in a compressed form, tracking their access times without needing them to remain as loose objects. These innovations have significantly reduced the time spent on repository maintenance, notably cutting repacking times by 20% globally and by 80% in large repositories, and can be manually enabled in GitLab 15.11 or later by adjusting feature flags.

In the evolving realm of software development, the integration of cloud compatibility, faster development cycles, and AI-driven capabilities has become crucial, prompting many organizations to transition from traditional tools like Jenkins to more advanced platforms such as GitLab CI/CD. GitLab offers a comprehensive DevSecOps solution that streamlines workflows, enhances collaboration, and emphasizes security, containerization, and GitOps principles. The transition from Jenkins to GitLab, though beneficial, is often complex due to existing Jenkins implementations. The text outlines various migration strategies, including gradual adoption for new or strategic projects and comprehensive migration efforts, as exemplified by companies like Lockheed Martin, which successfully improved efficiency and productivity through GitLab's capabilities. To ensure a smooth transition, organizations are advised to conduct stakeholder communication, provide training, and undertake detailed planning while leveraging GitLab's documentation and professional support services.

The blog post discusses automating software release processes using GitLab's Changelog API and CI/CD pipelines, emphasizing the importance of effective communication about software changes through release notes and changelogs. It explains how to automate the creation of release notes and changelogs by leveraging commit messages and merge histories stored in a git repository. The process involves using commit trailers in the format `<HEADER>:<BODY>` to store change information, which the GitLab Changelog API can parse to generate changelogs automatically. The tutorial outlines setting up automated release jobs in a `.gitlab-ci.yml` file, using GitLab's API with a project access token, and semantic versioning to ensure smooth automation and adherence to best practices. The example given illustrates these concepts through a Python web app, showcasing the addition of a new feature and the removal of an old one, while automating the release of Version 2.0.0, thus streamlining the release management process by integrating automation into existing development workflows.