July 2020 Summaries

AWS Fargate does not support containers running in privileged mode, which complicates the use of Docker-in-Docker (DinD) for building and running container images within GitLab Runner's CI/CD pipelines. However, a cloud-native solution is proposed by integrating AWS CodeBuild to build containers, circumventing the limitations of Fargate. Detailed instructions are provided for setting up and configuring GitLab Runner to run on AWS Fargate, including using AWS CodeBuild to handle the container build process. The process involves creating an S3 bucket and a CodeBuild project, configuring CI containers with necessary AWS CLI tools, and setting up AWS IAM permissions for managing resources like S3, CodeBuild, and CloudWatch Logs. By adding a buildspec.yml file and a custom script, codebuild.sh, to the GitLab repository, users can trigger CodeBuild jobs that fetch build files from S3, execute build processes using Docker, and log outputs. This integration allows GitLab CI to scale efficiently on AWS Fargate while leveraging AWS services to manage container builds without requiring privileged mode.

GitLab has officially launched a GitLab Workflow Extension for VS Code, initially developed by Fatih Acet and subsequently expanded by over 25 contributors, achieving over 160,000 installations. This extension enables users to interact with GitLab directly from their code editor and has been integrated into GitLab's official offerings following Acet's departure. Upon taking over, GitLab conducted a security review and automated various processes to enhance the extension's reliability, culminating in the release of version 3.0.0, which addressed numerous longstanding issues. Moving forward, GitLab aims to improve the extension's test suite and code quality while encouraging community contributions through open-source collaboration. The company is committed to maintaining and enhancing the extension, ensuring robust functionality and fostering an inclusive environment for developer engagement.

Kubernetes, a key component of successful DevOps practices alongside containers, can be intimidating for newcomers, with only 38% of respondents using it actively according to a 2020 survey. Despite its complexity, there is high interest in adopting Kubernetes to transition from monolithic servers to microservices, as highlighted by GitLab engineers Jason Plum and Gerard Hickey during a company-wide meeting. They emphasized understanding Kubernetes' core concepts and terminology, such as containers, pods, namespaces, services, ingress, ConfigMaps, and secrets, for deploying cloud-native applications. Kubernetes serves as an orchestrator that manages containers and software-defined networking, allowing for portability and vendor-independent deployment. The platform ensures containers are started, maintained, and running efficiently, providing a replicable syntax that facilitates updates and load balancing without downtime. Understanding these elements is crucial for effectively utilizing Kubernetes in creating scalable and reliable applications.

GitLab commissioned Forrester Consulting to conduct a Total Economic Impact (TEI) study to help organizations understand the potential return on investment (ROI) from using GitLab for version control, collaboration, continuous integration, and continuous delivery. The study involved independent interviews with GitLab customers, which were analyzed to create models quantifying the benefits experienced by these organizations. Key findings indicated a 407% ROI, an 87% improvement in development and delivery efficiency, a 12-fold increase in revenue-generating application releases, an 80% reduction in code defects, and significant savings from reducing the number of tools used. While these results are impressive, GitLab emphasizes that individual results may vary and encourages organizations to use an online estimator to assess their own potential benefits. The study highlights the experiences of GitLab customers and provides tools for others to estimate their own potential outcomes.

This guide provides insights into enhancing a Continuous Integration/Continuous Deployment (CI/CD) pipeline by focusing on performance, security, and integration. It emphasizes the speed advantage of CI/CD, recommending autoscaling runners and leveraging tools like Google Kubernetes Engine and Auto DevOps for faster builds. It encourages creativity in using CI/CD by suggesting unconventional uses like building cross-project pipelines or integrating Rust with Firebase, all while ensuring security through threat awareness and using Vault for data protection. Seamless integration with other platforms, such as AWS and Firebase, is highlighted to maximize the pipeline's effectiveness, especially for specific projects like Android development. The guide underlines the importance of a well-optimized CI/CD pipeline in achieving efficient and secure software delivery.

As the number of web applications hosted in containers increases, many companies still lack a robust container security strategy. GitLab offers solutions to enhance the security of these applications, focusing on mitigating threats through features like secret detection, dependency scanning, Dynamic Application Security Testing (DAST), and Web Application Firewall (WAF). By leveraging the MITRE ATT&CK Matrix, the post explores how attackers gain initial access and execute malicious instructions by exploiting exposed credentials, supply chain vulnerabilities, and input validation flaws. GitLab's approach emphasizes shifting security left to detect vulnerabilities early in the development process and employing layered defenses such as logging, SIEM integration, network policies, and Pod Security Policies to prevent and detect attacks. These features collectively aim to secure containerized environments and support organizations in proactively protecting their cloud-native applications.

GitLab emphasizes the importance of safe deployment practices through progressive delivery, which allows for controlled exposure of changes to users and minimizes potential negative impacts. To enhance deployment safety, GitLab has introduced features such as protected environments, sequential deployment, and options to limit job concurrency, prevent deployment of outdated versions, and implement deployment freezes. Protected environments restrict deployment jobs to authorized personnel, ensuring controlled access. Sequential deployment addresses issues arising from the asynchronous nature of pipeline jobs by allowing only one job to run at a time, preventing conflicts and unintended rollbacks. The prevention of outdated version deployments cancels older jobs automatically when a newer one is executed, avoiding race conditions. Deployment freezes prevent any deployments during specific periods, such as holidays, ensuring stability when staffing might be limited. These features collectively enhance the security and reliability of the GitLab CI/CD pipeline.

Rust is an open-source programming language lauded for its memory safety and performance, earning it the title of "most loved language" on Stack Overflow's annual survey for several years, despite its relatively low usage compared to giants like Python and Java. Developed by Graydon Hoare at Mozilla Research in 2006, and now maintained by the Rust Foundation, Rust is designed to solve problems in other languages, offering developers cleaner, faster, and safer code. Its unique approach to memory management, lacking a garbage collector and instead using a borrow checker, enhances efficiency and safety, making it ideal for system-level programming and applications where memory usage is critical, such as embedded devices. Rust's syntax combines high-level language advantages with low-level control, and its safety features prevent common programming errors, contributing to its reputation as a secure language. Companies like Firefox, Dropbox, Discord, and Shopify have adopted Rust for its ability to handle large data and CPU-intensive tasks effectively. Despite its steep learning curve, particularly with its borrow checker, developers find Rust rewarding, providing a clear and productive coding experience once mastered. While it may not achieve the widespread popularity of languages like Go, Rust's niche for "fearless development" ensures it remains a strong contender in areas demanding high performance and security.

The fifth installment in a series on DevSecOps basics emphasizes the importance of standardizing security policies and practices to achieve consistency, traceability, and repeatability across an organization. It outlines the need for a holistic security program composed of various levels of policies and compliance measures, including regulatory compliance, access controls, and security as code. The text highlights five key strategies for standardizing security: educating employees on security best practices, coordinating security requirements across development teams, implementing robust access controls, integrating security tools into development pipelines, and automating security processes to enable scalable and secure development. The ever-evolving nature of security threats and regulations necessitates continuous evaluation and updating of security standards to ensure effectiveness and adaptability.

GitLab's acquisition of Peach Tech and Fuzzit enhances its fuzz testing capabilities by integrating advanced security testing methods into its workflow, addressing previously challenging aspects like setup complexity and result management. Fuzz testing, which identifies security vulnerabilities through random input testing, is made more accessible through GitLab's platform, allowing developers to use coverage-guided and behavioral fuzz testing without needing external tools. This integration allows for seamless inclusion of fuzz testing results alongside existing build and test outputs, facilitating easier identification and resolution of security issues. GitLab also emphasizes its commitment to open source, offering several fuzz testing engines to the community and planning future open-source developments, including potential protocol fuzz testing. The initial release of these fuzz testing capabilities is set to include behavioral-guided testing for web APIs and coverage-guided testing for various languages, starting with Go, as part of GitLab Ultimate, with aims to further integrate these results into developers' workflows and expand to other use cases.

CI/CD and DevOps have revolutionized software development by significantly enhancing the speed and efficiency of code release, as evidenced by the findings of global surveys conducted in 2020 and 2022. Automation has reduced manual tasks, leading to a shift in roles for developers and operations teams, allowing for faster deployment and more frequent code releases. Developers report a substantial decrease in time-consuming activities like manual testing and deployments, enabling a focus on code quality improvements through regular code reviews and increased testing. Operations roles are also evolving, with more emphasis on managing cloud services and infrastructure, as automation handles routine tasks. The convergence of developer and operations roles, facilitated by CI/CD, allows for greater flexibility and innovation, with developers increasingly involved in infrastructure management and operations teams having more time for strategic responsibilities. The transition has created opportunities for more comprehensive testing and improved code review processes, as teams strive to enhance overall software quality and efficiency.

DevSecOps emphasizes the integration of security into every aspect of the software development lifecycle, requiring a cultural shift where all employees, regardless of their role, are responsible for maintaining security. This approach involves embedding security practices into daily workflows, fostering cross-functional collaboration, and ensuring security is prioritized at the organizational level. Key steps to instilling a security culture include leadership involvement, comprehensive training, appointing security champions among developers, facilitating collaboration, integrating security tools seamlessly, and leveraging automation for scalable security processes. By shifting security left, companies aim to streamline workloads, enhance communication, and ultimately produce more secure software.

GitOps is an operational framework that adapts DevOps best practices, such as version control, collaboration, compliance, and CI/CD, to automate IT infrastructure management effectively. It uses infrastructure as code (IaC) with configuration files stored in version control systems, allowing teams to make changes via merge requests and deploy them through CI/CD pipelines. This approach reduces human error, fosters collaboration, and increases transparency by aligning infrastructure management with application development processes. As GitOps gains traction, a recent GitLab poll found that 23.8% of respondents are already using it, 10.6% plan to implement it, and 11.6% have explored it but not committed yet, while 54% are still unfamiliar with the concept. Although GitOps is not limited to Kubernetes, its principles can be applied to various types of infrastructure automation, and as organizations increasingly recognize its potential, GitOps could herald a significant shift in how infrastructure is managed.

Azure DevOps can integrate with GitLab, allowing for a gradual migration from Azure DevOps/VSTS source code management while adopting GitLab's CI/CD capabilities. This integration is compatible with both the self-managed and SaaS versions of GitLab but only works with Azure DevOps git version control, excluding TFVC. The process involves mirroring repositories from Azure DevOps to GitLab, enabling automatic synchronization of branches, tags, and commits. Users can configure CI/CD pipelines in GitLab by either enabling Auto DevOps or defining their own pipeline configurations with a .gitlab-ci.yml file. This setup allows developers to maintain their code in Azure DevOps during the transition while leveraging GitLab's comprehensive DevSecOps tools, including security scanning, code reviews, and deployment management. The integration supports a development workflow where code is written in an IDE, committed to Azure, and then mirrored to GitLab for building and testing, with pull requests and code reviews facilitated in both platforms. GitLab's offering is particularly beneficial for those who seek a unified source code management and CI/CD solution, with the flexibility to accommodate teams unable to migrate entirely to GitLab immediately.

Participating in the GitLab CEO Shadow program in July 2020 provided an immersive experience in understanding GitLab's values and decision-making processes through direct involvement in numerous meetings and collaborative activities. The program involved attending over ten meetings per day, including interactions with GitLab's executive group, key reviews, and investor and media interviews. A significant aspect of the experience was the active contribution to GitLab's extensive handbook, where participants authored and collaborated on several merge requests, reflecting both direct tasks from the CEO and innovative ideas from the shadows. The program underscored the importance of living GitLab's culture to truly understand it, emphasizing values such as efficiency, collaboration, diversity, and effective communication. Participants observed firsthand how GitLab's high merge request rate serves as a competitive advantage and learned to prioritize impactful cross-functional topics in meetings. The experience also highlighted the value of asynchronous communication and the principles of providing context for feedback, demonstrating gratitude, and maintaining genuine responses in all interactions.

GitLab successfully completed its migration from the Unicorn to the Puma web server, marking this transition as a significant improvement for their application infrastructure. Both Unicorn and Puma are servers for Ruby on Rails, but Puma, with its multithreaded model, offers enhanced scalability and memory efficiency compared to Unicorn's single-threaded process model. The migration journey, which began with initial explorations in 2015, involved extensive tuning and testing to ensure thread safety and optimal configuration, leading to the creation of a dedicated Memory Team in early 2019. Puma's deployment on GitLab.com resulted in a notable reduction in memory usage by approximately 37% without impacting request queuing, duration, or CPU usage, as observed after the Puma roll-out in GitLab's 13.0 release. Despite encountering challenges such as thread safety issues and tuning difficulties, GitLab's efforts yielded substantial infrastructure efficiency gains, allowing them to reconsider their memory requirements and pursue new functionalities like real-time editing.

The third installment of a series on DevSecOps emphasizes the crucial role of security automation in streamlining software development and enhancing cybersecurity. Despite a significant number of developers releasing code faster due to DevOps practices, many still do not conduct essential security tests such as static and dynamic application security testing. The text underscores the importance of shifting security left in the software development lifecycle to prevent security from becoming a bottleneck. By integrating automated security testing, such as SAST and DAST scans, organizations can improve security measures, reduce human error, and expedite vulnerability management, ultimately leading to safer and more efficient deployments. The explanation clarifies the difference between security automation, which simplifies individual security tasks, and security orchestration, which connects and optimizes these processes. It highlights various tools like SIEM, SOAR, and XDR that aid in managing security incidents and stresses the importance of balancing automation with manual oversight to maintain visibility and efficiency without replacing human expertise.

GitLab is revolutionizing software development and deployment by offering a comprehensive DevOps platform as a single application, as highlighted in Forrester's Q2 2020 Continuous Delivery and Release Automation (CDRA) report, where GitLab was recognized as a Strong Performer. The report evaluated 14 vendors based on their current offerings, product strategies, and market presence to guide companies in selecting the best CDRA solutions. GitLab has made significant advancements since 2018, enhancing its continuous delivery capabilities and emphasizing automation to meet the evolving needs of the software market. Forrester identified key differentiators in the market, such as visualization of complex deployment models and the integration of advanced analytics and machine learning. GitLab stands out for its rapid innovation and strong capabilities in continuous integration, deployment, and product innovation, despite its focus on cloud-native platforms and limited support for legacy systems. The company's strategy is rooted in an active open-source community and a commitment to progressive delivery, aiming to set a standard of excellence in the CDRA domain.

Application Security (AppSec) engineers focus on improving application security by identifying, resolving, and preventing vulnerabilities, a process streamlined by the GitLab Secure features in GitLab Ultimate. These features provide accurate, automated, and continuous security assessments through a unified dashboard. GitLab Secure offers robust security scanning capabilities, including Static and Dynamic Application Security Testing, Container Scanning, Dependency Scanning, and License Scanning, which help AppSec engineers proactively identify vulnerabilities. The Security Dashboard in GitLab Ultimate organizes and summarizes all relevant security details and allows engineers to manage vulnerabilities effectively by offering a high-level overview of their status and severity. It supports third-party scanner integration, enabling comprehensive auditing and tracking of vulnerabilities and dismissed issues. Additionally, GitLab Secure's license management feature helps ensure that application dependencies comply with acceptable licensing policies, preventing the use of restrictive licenses that could invalidate applications. These capabilities enhance the efficiency of AppSec engineers, contributing to the development of more secure applications and a security-empowered development team.

GitLab's unique organizational culture, characterized by its all-remote, transparent, and ambitious nature, significantly impressed the author during their summer internship. The company emphasizes transparency, allowing both insiders and outsiders to contribute to its content, fostering a culture of asynchronous communication and collaboration through shared documents and meeting practices. Employees are encouraged to take initiative and are granted autonomy, which deviates from conventional organizational hierarchies that restrict change to senior levels. However, the author critiques GitLab's current set of values—Collaboration, Results, Efficiency, Diversity, Inclusion, Belonging, Iteration, and Transparency—as not being ideal foundational values, suggesting they are more tactical or metric-driven rather than ethical tenets. The author argues that organizational values should be fixed, ethical, and foundational, proposing that GitLab adopt Respect, Service, and Integrity as core values to guide their business strategy and navigate potential conflicts, ultimately ensuring a cohesive, resilient culture as the company grows.

Continuous integration and continuous delivery/deployment (CI/CD) are essential components of modern software development, particularly within DevOps, offering a streamlined process for developers to implement code changes that are automatically tested and deployed, thereby minimizing downtime. This approach not only accelerates code release but also enhances developer satisfaction, which is crucial given the global shortage of software developers, and provides a significant return on investment by reducing toolchain sprawl costs. To effectively implement CI/CD, selecting the appropriate tools is vital, and it is advisable to consider factors such as budget and scalability. Practical implementation can be achieved swiftly, as exemplified by using GitLab's Auto DevOps to create a CI/CD pipeline in just 20 minutes, which can be further integrated with Google Kubernetes Engine (GKE) for enhanced deployment capabilities.

Code reviews are a critical yet challenging component of software development, as they provide opportunities for identifying bugs, sharing knowledge, and increasing collaboration among developers, according to insights from GitLab's DevSecOps Surveys. The 2020 survey highlights the stress and complexity involved in code reviews, including the issues of lengthy wait times and lack of buy-in, while the 2022 survey offers perspectives from over 5,000 DevOps professionals. Code reviews occur frequently within teams, primarily through messaging chats or offline, and are usually conducted in integrated development environments or browsers due to their accessibility. Developers express concerns over the time-consuming nature of reviews, particularly in distributed teams with differing time zones, and stress the importance of organized processes to prevent confusion and pressure. Despite these challenges, code reviews are deemed valuable by a majority of respondents, as they contribute to maintaining a clean codebase and fostering skill development, highlighting the need for IT leaders to establish effective processes and tools for successful code reviews.

Implementing an effective compliance program is a complex challenge that involves navigating costly and cumbersome processes, often managed through spreadsheets and file storage systems, to meet regulatory and legal requirements. Many organizations still struggle with administrative overhead and the difficulty of demonstrating compliance, especially when third-party applications are involved. GitLab offers a streamlined solution by automating compliance processes within a single platform, integrating development and operational tools to reduce data silos, and providing granular user roles and permissions for enforcing policies. It supports compliance with various frameworks like HIPAA and SOX by consolidating documentation, tracking user actions, and offering a compliance dashboard that aims to provide insights into compliance activities. Through these capabilities, GitLab seeks to alleviate the friction between compliance and operational teams, enabling more efficient and effective compliance management.

The text delves into the importance of cross-functional collaboration within DevSecOps, highlighting how integrating security into DevOps necessitates seamless communication and teamwork among development, security, and operations teams. It emphasizes that successful DevSecOps implementation requires a shift in roles and responsibilities, fostering an environment where team members can exchange information freely and learn from one another. Leaders play a crucial role by setting an example and ensuring that all team members understand the collective goals, which can range from broad objectives like delivering secure products to specific tasks such as complying with GDPR. The text also underscores the need for centralized information sharing to minimize context-switching and enhance efficiency, suggesting that a unified platform can help consolidate tools and reduce project delays. Additionally, it discusses the benefits of strong collaboration, such as quicker vulnerability management and reduced costs, while encouraging organizations to assess their DevSecOps maturity for continuous improvement.