March 2021 Summaries

Bugs in software development can be both frustrating and rewarding to solve, as demonstrated by various anecdotes shared by developers. Brendan O'Leary, a senior developer evangelist, humorously highlighted that not all "bugs" are actual code issues, citing a case where a perceived bug was caused by an employee accidentally pressing keys with a purse. This emphasizes the first debugging tip: not all errors stem from code, sometimes human factors are involved. Other shared stories include a day wasted due to a misinterpreted backtick, empty printer trays mistaken for software issues, and server outages caused by unplugged power for cleaning purposes. A key lesson is to ensure observability at every level to catch such errors. The text provides additional debugging tips, such as obtaining detailed bug reports, remembering that computers execute exactly as instructed, and checking obvious causes first, like DNS issues. The narrative encourages sharing more debugging stories on social media to foster a community of learning and humor among developers.

Balancing the scale of design initiatives is crucial for designers, who often face the challenge of either overwhelming their teams with overly ambitious projects or underwhelming them with overly modest ones. The ThinkBIG approach encourages designers to harness their natural skills to align the iterative nature of engineering with the visionary aspects of design, effectively bridging the gap between the two. Several signals suggest a need for this shift, such as the frequent reworking of designs, engineers asking numerous questions, and disagreements on what constitutes a minimum viable change (MVC). ThinkBIG advocates for a collaborative process where engineers are informed about long-term goals, allowing them to craft solutions that account for future iterations and reduce unnecessary rework. This approach fosters a stronger collaboration between designers, engineers, and product managers, ensuring that the team remains focused on delivering valuable customer experiences while iteratively developing products. The ThinkBIG model emphasizes moving iteration breakdowns to after the design phase, thereby enhancing communication and providing a comprehensive understanding of the product's vision to all team members.

The GitLab for Education Program has celebrated significant milestones, including issuing 2 million seats and receiving valuable feedback from program participants that has guided enhancements to their offerings. GitLab's use in educational institutions has expanded beyond traditional computer science departments to various academic and administrative fields. In response to increased demand and feedback, GitLab has revamped its licensing structure, introducing the GitLab for Campuses offering to facilitate comprehensive campus-wide adoption, combining benefits of both free and paid plans. The program offers free subscriptions for teaching, learning, or research and allows flexibility in deployment methods, catering to the diverse needs of educational institutions. The new GitLab for Campuses model focuses on administrative use, with pricing based on institutional size, while maintaining free access for students. For institutions not yet ready for campus-wide adoption, a 20% academic discount is available. The program is also focusing on automating application processes and developing educational content to further integrate DevOps into academia.

Moving to cloud-based platforms like GitLab.com enhances innovation and scalability for engineering teams by integrating data securely and providing cost-effective infrastructure. GitLab, a cloud platform for software delivery, partners with leading cloud companies to enhance DevOps ecosystems and streamline collaboration across the software development lifecycle (SDLC). A notable integration is with Atlassian's Jira, a popular project management tool, allowing seamless connectivity between GitLab and Jira. This integration enables users to link Jira issues with GitLab projects, navigate between platforms, and automate tasks such as closing Jira issues through GitLab commits and merge requests. Configuring this integration involves using the GitLab for Jira app, which syncs data in real time and offers enhanced project visibility and collaboration. Despite some limitations, such as the inability to sync past data, this integration supports various workflows and enhances the efficiency of software projects by unifying tools and workflows.

GitLab has made significant strides in fuzz testing by releasing the core protocol fuzz testing engine of Peach as the open-source GitLab Protocol Fuzzer Community Edition, which was previously available only through a commercial Peach license. This release enhances access for security researchers, students, and developers, allowing them to experiment with protocol fuzz testing to uncover vulnerabilities and bugs. Fuzz testing, an automated software testing technique, involves injecting unexpected data into a program to identify potential defects and vulnerabilities, a method that has origins dating back to 1988. The Peach Fuzzer, a sophisticated tool for both generation and mutation-based fuzzing, simplifies the process by providing a way to define data formats and configure testing runs. While fuzz testing is effective in identifying elusive bugs and assessing system robustness, it presents challenges such as setup complexities and data management. GitLab encourages integrating fuzz testing into existing CI/CD workflows to enhance security scanning, and plans to further develop the Community Edition by adding industry-specific features and tighter integration with its CI process, inviting community contributions to advance the tool's capabilities.

GitLab thrives on community collaboration, with contributors enhancing its open DevOps platform by suggesting improvements, submitting bug fixes, and adding features, totaling around 300 merge requests monthly. Highlighting its open core model, even organizations like NASA contribute to GitLab's codebase. The platform's open approach presents unique security challenges, but GitLab addresses these with public transparency, including making security bug reports visible post-resolution to foster a feedback loop with external researchers. Customers and community members actively participate in debugging and performance troubleshooting, as seen in the resolution of significant issues like the Redis memory limit problem. GitLab partners with organizations like The Last Mile and GNOME to promote values of openness and education, celebrating community-driven innovation and inviting stories for their virtual user conference, GitLab Commit.

The blog post reflects on the author's initial claims of building a "better Heroku," acknowledging that the current iteration of their "5 minute production app" falls short of Heroku's capabilities for production applications. It explores the ease of deploying a web application using Heroku, detailing the steps involved, such as creating an account, setting up a development environment, and deploying using the Heroku CLI. The post also contrasts Heroku with the author's approach, which utilizes GitLab and Terraform for provisioning stateful services like PostgreSQL and Redis on AWS, highlighting the automation and integration benefits of their method. The author concedes that while the deployment of a production app in a hypercloud is promising, it requires more development to surpass Heroku's established platform, emphasizing the need for further exploration of stateful backends, CI/CD, and security practices.

David O'Regan's blog post emphasizes the importance of iteration in software development, particularly in the context of code reviews at GitLab. Iteration, defined as breaking down large problems into smaller, manageable tasks, is crucial for creating efficient and effective merge requests (MRs). Smaller MRs are easier for reviewers to evaluate, reducing the complexity and potential for errors that can arise with larger submissions. GitLab promotes this practice by encouraging developers to submit minimal viable changes and utilize tools like DangerBot to flag oversized MRs. The post also highlights best practices for both code authors and reviewers, such as following through on review promises and handling large MRs by creating smaller ones. The concept of iteration is so integral to GitLab that it is ingrained in their values, with CEO Sid Sijbrandij dedicating weekly sessions to reinforce its application. By consistently focusing on smaller MRs, developers can enhance the quality of code reviews, foster better collaboration, and ultimately deliver more value to customers.

GitLab has expanded its presence in China by licensing its technology to JiHu, an independently operated company, to provide a tailored version of its DevOps platform specifically for the Chinese market. JiHu, which operates with complete autonomy over its governance and business functions, offers both self-managed and SaaS solutions through GitLab.cn, distinct from GitLab Inc.'s global services. This strategic move addresses China's growing developer community and the demand for locally supported, updated software solutions, contrasting with outdated GitLab forks that are unsupported. Investors in JiHu include Sequoia CBC and Gaocheng Capital, and the company aims to foster contributions to the open-source community by adhering to rigorous standards for security and code quality. By focusing on the unique needs and compliance requirements of Chinese enterprises, JiHu seeks to enhance software development efficiency in China while maintaining GitLab's global principles of collaboration and contribution.

Navigating the complexities of software development at GitLab has proven transformative for the author, who initially struggled with collaborative development but gained competence through hands-on experience and effective documentation. By setting up the GitLab Development Kit and utilizing GitPod, the author overcame early technical hurdles, allowing for active participation in tasks such as migrating button components to a new front-end framework. Emphasizing the value of dogfooding, the author highlights the importance of internal users contributing to the product, fostering empathy and continuous improvement. As a designer, acquiring a functional understanding of the frontend framework has facilitated better communication with engineers and more actionable design proposals, such as submitting Merge Requests for specific codebase changes. This hands-on learning process has broadened the author's skill set, enabling more effective collaboration and innovation within GitLab, evidenced by contributions to ongoing projects like UX enhancements and markdown improvements.

In a large-scale project, Team Geo at GitLab introduced a Maintenance Mode feature, allowing system administrators to set the platform in a read-only state, which posed significant testing challenges due to its wide-ranging impact. The complexity of testing was compounded by the impossibility of any single team having comprehensive knowledge of the entire system, necessitating a collaborative, crowd-sourced approach to identify critical features for testing across 13 stages. This led to overwhelming participation from product managers and engineers, enriching the test list and documentation, and fostering a collaborative environment that improved future iterations' planning. GitLab's Quad planning process facilitated early collaboration between QA and development teams, highlighting the importance of iterative testing, cross-team contributions, clear communication, and comprehensive documentation as tools for effective development and testing, while keeping discussions focused and engaging more stakeholders proved beneficial in addressing unknowns.

GitLab's Application Security (AppSec) team collaborates with both internal teams and external security researchers to enhance the security of its products through a bug bounty program. This program encourages security experts worldwide to identify vulnerabilities, which are then verified and triaged by the AppSec team, with successful findings rewarded monetarily. To foster engagement and transparency, GitLab has initiated a blog series called "Ask a Hacker" and hosts public Ask Me Anything (AMA) sessions with contributors. The Bug Bounty Council process, which relies on asynchronous communication due to the global distribution of the AppSec team, ensures consistency in severity and bounty assessments through a structured issue tracker system. Iterative improvements, such as automating report submissions and requiring CVSSv3 score approvals, have streamlined operations and enhanced accuracy. Transparency is a key value, and ongoing efforts aim to incorporate more comprehensive CVSS calculations to further clarify severity and bounty determinations. The program, public since December 2018, invites participation from anyone interested, with new features released monthly, providing continuous opportunities for bug hunters to contribute to GitLab's security.

David O'Regan discusses the significance of fairness and empathy in code reviews and highlights the utility of patch files in enhancing this process. Patch files, which contain metadata about commits, are instrumental in code reviews as they allow reviewers to propose actionable code changes that can be directly applied to a merge request by the author. This fosters a collaborative environment akin to a paired programming session, enabling thorough checks and corrections before finalizing changes. The post provides guidance on creating patch files using both web editors and command lines, emphasizing their advantages over other methods, such as GitLab's suggestion feature or raw Markdown code, which do not allow for code testing. By involving reviewers more deeply in the process, patch files enhance visibility and collaboration, encouraging reviewers to fully engage with the codebase changes they advocate. Despite some perceptions of patch files as a means for reviewers to push preferred changes, O'Regan argues that they represent a commitment to teamwork and responsibility in the review process. GitLab is considering integrating patch files more formally into its code review workflow, reflecting their potential to enrich collaborative coding practices.

The shift to remote work during the COVID-19 pandemic has led many engineering teams to adopt practices that GitLab has long embraced, and this roundup explores insights and best practices for remote work shared by leaders in the field. GitLab emphasizes a remote-first culture, recommending a transition from remote-friendly to remote-first to foster inclusivity, and offers guidance for thoughtful remote management, including reducing Zoom fatigue by cutting down on meetings and encouraging asynchronous communication. The importance of collaboration in remote settings is highlighted, with suggestions such as remote pair programming and the integration of Agile practices to enhance productivity. GitLab's results-driven approach is underscored by the use of metrics like merge request rates to measure engineering productivity. The company also provides various resources, including a Coursera class and a comprehensive guide, to support teams in optimizing their remote work strategies.

GitLab is implementing changes to adopt a more inclusive default branch name, transitioning from "master" to "main" in alignment with community feedback and wider industry practice. This change will be phased in across GitLab.com and self-managed users, with the first phase updating GitLab.com on May 24, 2021, and the second phase affecting self-managed GitLab as part of the major 14.0 release on June 22, 2021. This move follows Git's introduction of the init.defaultBranch configuration option in 2020, allowing users to set a default branch name other than "master," and reflects a broader effort to move away from terms with potentially offensive historical connotations. GitLab's existing projects will remain unaffected; however, new projects will adopt "main" as the default branch name, requiring users to update any hard-coded references in CI/CD configurations. The change aligns with similar shifts by GitHub and Atlassian, marking a significant step toward inclusivity in software development practices.

Adapted from a GitLab blog post by David O'Regan, this text explores the importance of effective communication and fairness in code reviews at GitLab, emphasizing the role of feedback in personal and professional development. It introduces strategies for authors, like employing detailed checklists to ensure code quality before submission, and for reviewers, such as using the "conventional comments system" developed by Paul Slaughter, which employs context-defining keywords to clarify the intent and priority of feedback. The text underscores the critical nature of assuming positive intent and maintaining fairness to mitigate cognitive biases during code reviews, suggesting that reviewers should avoid absolute language unless necessary and support their feedback with documentation. It highlights empathy as a key element in the process, encouraging both authors and reviewers to collaborate openly and focus on enhancing the merge request's quality. This post is part one of a three-part series, with future installments set to discuss the utility of patch files for reviewers.

The Digital Experience team at GitLab has rapidly developed a new design system called Slippers, aimed at solving the problem of fragmented design and code by creating a centralized repository for design assets and code. This system is intended to be scalable, easy to iterate on, and provide a single source of truth for the team, addressing past issues of design discrepancies due to the lack of a style guide. The team approached the development of Slippers by aligning around a common vision, establishing guiding principles, and using tools like Figma for design and local CSS for rapid delivery. The project serves as a research and development initiative, incorporating open-source values and involving substantial effort and investment. Despite being in its early stages, the Slippers project has already demonstrated its value by delivering over 2000 new CMS pages and continues to gather data to refine and enhance the system further. The team actively shares its progress through video updates and encourages engagement with their Slippers project on GitLab Unfiltered.

GitLab is implementing changes to enhance privacy by restricting access to identifiable information to fewer individuals and introducing a system to de-identify users and personal data from multi-user instances before it enters the analytics environment. This initiative aims to aggregate de-identified user activity at the account level, although a gap remains for single-user namespaces where users might still be linked to their accounts. The organization will seek community feedback on the design and implementation of these changes over the next 30 days, with plans to roll out the updates starting in June 2021. Meanwhile, GitLab maintains its current policy on service usage data collection from SaaS or Self-Managed customers, and additional information about their privacy policy and data usage for product improvement is available on their Product Direction page.

Alex Chapman, a seasoned bug bounty hunter, shares insights into his hacking journey and philosophy, emphasizing the importance of clear vulnerability reporting policies for companies. Inspired by the film "Hackers," Chapman began programming at a young age and eventually transitioned to a professional hacking career, focusing on penetration testing and bug bounties, particularly with GitLab's Bug Bounty Program due to its open-source nature and rewarding structure. He highlights the value of openness in security, which allows for the identification of critical issues before exploitation, and expresses concern over the growing threat of supply chain attacks. Chapman enjoys uncovering vulnerabilities that arise from complex system interconnections and advocates for more open research in this area. He also notes GitLab's strengths and areas for improvement, such as enhancing markdown editing capabilities for better user experience. Throughout his career, Chapman has balanced professional growth with personal recovery, dedicating himself to bug hunting full-time since 2019, which has allowed him flexibility and time for family.

GitLab's blog series highlights the journeys and insights of women in its security organization, focusing on their experiences and strategies for thriving in the tech industry. Despite challenges such as the low representation of women in tech and the tendency for young women to leave the field by age 35, GitLab aims to foster inclusivity through initiatives like outbound hiring, mentorship programs, and resource groups. The featured women, including security engineers and compliance specialists, share their paths into security, emphasizing the importance of embracing risk, continual learning, and collaboration. They offer advice on starting a career in security, noting that a willingness to explore new opportunities and learn on the job is crucial, even when not fully qualified. The series underscores security as a collective effort that requires ongoing adaptation and a proactive mindset, with each professional offering unique perspectives on how security can enable business growth and innovation.

Agile principles, characterized by collaboration and iteration, are central to improving communication and efficiency in both remote and office environments, as demonstrated by GitLab's adoption of these methods. Over-communication, thorough documentation, and a handbook-first mentality are emphasized to enhance team collaboration and transparency, especially in asynchronous work settings. GitLab's integration of Agile into its culture is further supported by its DevOps platform, which includes features like issues, labels, and milestones to streamline workflows and maintain lean processes. The transition from analog to digital tools, such as GitLab, has enabled teams like IssueTrak to manage sprints more effectively, reducing costs and enhancing flexibility by simplifying toolchains. Despite challenges in fostering team camaraderie remotely, the combination of Agile practices and GitLab's tools allows for continuous software delivery and adaptability, fostering a human-focused approach to work.