September 2018 Summaries

GitLab's experience with Prometheus highlights its crucial role in efficiently managing and troubleshooting complex systems by providing real-time metrics and alerts, particularly in large-scale deployments. Initially, the GitLab Support Engineering team underestimated the necessity of Prometheus, relying instead on traditional tools and experience to solve customer issues. However, a critical failure at a large client's GitLab cluster revealed the limitations of their approach and demonstrated the effectiveness of Prometheus in quickly identifying and resolving infrastructure problems. Prometheus, an open-source monitoring service originally developed by Soundcloud and maintained by the Cloud Native Computing Foundation, uses a pull model to collect data, which can be visualized through dashboards like Grafana. GitLab advocates for its adoption by all customers, noting its ability to streamline data analysis and enhance collaboration across teams. This shift towards Prometheus has not only improved GitLab's internal processes but also aims to standardize performance monitoring for clients, encouraging the adoption of shared dashboards to facilitate a common understanding of system health and performance.

During #GitLabLive, Michael Sobota, Director of Product Integration at Charter Communications, discussed how the adoption of GitLab as a unified application for their software development lifecycle significantly reduced their feedback loop from two weeks to minutes. Charter Communications, the second-largest cable operator in the US, serves over 26 million customers and employs 94,000 people globally. Sobota emphasized the importance of a cohesive platform for development, operations, and feedback to enhance the developer experience and address security and testing efficiently. He highlighted that using GitLab for source control, continuous integration, deployment, and feedback empowers developers by providing a centralized, self-service environment that streamlines processes and improves competitiveness in offering digital services. This consolidation allows Charter to quickly adapt to consumer demands for digital interactions, such as bill payments and streaming services, by enabling rapid feedback and course corrections throughout their development process.

After a recent summit, the team has been engaging in various readings, sparking diverse discussions on Slack. Erica Lindberg expressed admiration for a Medium article about deploying a serverless app without coding, calling the author her "new hero." CEO Sid Sijbrandij is immersed in the "High Growth Handbook," while Product Manager Jeremy Watson is revisiting "The Design of Everyday Things" to emphasize mindful design in GitLab products. VP of Engineering Eric Johnson is excited about the introduction of generics in Golang 2.0, and Data Analyst Emilie Schario is finding "Storytelling with Data" valuable for creating clear and impactful data narratives to aid stakeholders in making informed decisions. The cover image for these discussions is credited to Ugur Akdemir under a Creative Commons license.

GitLab is enhancing its platform to streamline the DevOps lifecycle, emphasizing collaboration, code review, and user-friendliness. The platform's Create stage focuses on managing source code through features like Git repositories, merge requests, and a Web IDE, which facilitates easier contributions and faster merge request handling. GitLab has introduced improvements such as squash and merge options, file locking integrated with Git LFS, and future enhancements like suggested approvers based on code ownership. The platform aims to simplify collaboration between private organizations and open source projects by improving forking workflows and enabling distributed merge requests, which allow patches to be published from private instances to public repositories. Enhancements to code review processes include line-by-line code quality feedback and the ability to propose changes directly in merge request comments, streamlining the review and approval process. The Web IDE is being developed to offer a robust cloud development environment, featuring both client and server-side evaluation, allowing for real-time testing and customization to align with personal IDE preferences. GitLab encourages community involvement and feedback to continue evolving as a leading tool for building and deploying applications.

GitLab recently hosted a #GitLabLive event to discuss the company's growth, recent achievements, and future vision following the announcement of a $100 million Series D funding round. Hosted by Priyanka Sharma, the event featured CEO Sid Sijbrandij discussing milestones such as GitLab's recognition by Forrester, acquisition of Gemnasium, and reaching 2,000 contributors. Matthew Jacobson from ICONiQ Capital explained their investment in GitLab, praising its product velocity and team quality. Chief Culture Officer Barbie Brewer highlighted GitLab's commitment to a strong remote work culture emphasizing diversity and inclusion. William Chia and Michael Sobota discussed GitLab's product evolution, focusing on Concurrent DevOps and its impact on user experience. Head of Product Mark Pundsack shared ambitious plans for 2019, aiming to make GitLab a leader in project management, continuous delivery, application security testing, and value stream management. The event concluded with a Q&A session encouraging community contributions to GitLab's development.

Exploring GitLab through a trial is an effective way to experience its comprehensive features, particularly the distinction between its open-source version and the enhanced capabilities of GitLab.com Gold, also known as GitLab Ultimate. The trial allows users to delve into advanced DevOps tools such as security scans (SAST, DAST, and dependency scans), portfolio management, Kubernetes integration, and more, which are essential for managing projects efficiently. Previously, accessing these enterprise features required downloading and installing GitLab Ultimate, but now users can easily access them through the GitLab Gold Trial on GitLab.com. This hosted version simplifies the process, allowing users to quickly test the platform's full range of features without the hassle of setup, making it a convenient solution for those interested in evaluating GitLab's end-to-end DevOps lifecycle capabilities.

GitLab has announced a $100 million Series D funding round led by ICONIQ Capital, elevating its valuation to over $1 billion and reinforcing its status as the first single application covering the entire DevOps lifecycle. The funding will be used to enhance its offerings across all DevOps software categories, from planning to monitoring, with a focus on becoming best-in-class in each segment. GitLab, utilized by over 100,000 organizations, aims to streamline the DevOps process through Concurrent DevOps, enabling teams to manage all stages from planning to security within a single platform. This integration addresses issues faced by enterprises that traditionally rely on multiple disparate tools, resulting in inefficiencies and poor visibility. GitLab's open core model, supported by over 2,000 contributors, emphasizes co-creation, transparency, and open source, which enhances customer relationships and directly influences product development. The company's growth is marked by significant milestones, such as a 200 percent faster DevOps lifecycle and a seamless Kubernetes integration, aligning with its mission to support digital transformation and multi-cloud deployments.

GitLab fosters a vibrant community by welcoming contributions from individuals regardless of their employment status with the company, as evidenced by over 2,500 merged community contribution merge requests (MRs). To further engage this community, GitLab is hosting a virtual Hackathon on September 27-28, encouraging participants to work on issues marked as "Accepting merge requests," with incentives such as GitLab swag vouchers for those whose MRs are merged within a week of the event. Additionally, the event will feature expert presentations and Q&A sessions, which will be recorded and shared on GitLab's YouTube channel. Following the Hackathon, an Issue Bash will take place on September 29-30, and a new GitLab Community room in Gitter has been created to facilitate communication and support among contributors. The "Contributing to GitLab" page offers guidance on how to contribute to various aspects of GitLab, including code, documentation, translation, and UX design, with additional support available via email.

GitLab's Geo feature was developed to provide fast, read-only access to GitLab instances across multiple geographical locations, improving data accessibility for distributed teams. Initially inspired by the need to synchronize entire GitLab instances beyond just repositories, Geo was designed to overcome existing competitive limitations by using native database replication and Git protocols for efficient data synchronization without the overhead of webhooks. The first-generation synchronization mechanism faced challenges with webhook-based notifications, which led to the development of a second-generation system using PostgreSQL for event queuing, improving consistency and reducing dependency on external systems. The third-generation synchronization focused on creating an idempotent replication process through the implementation of Hashed Storage, which addressed issues of data consistency and reliability by using immutable disk paths derived from internal IDs. Geo has been critical for GitLab's internal cloud migration and aims to enhance disaster recovery solutions, improve user experience, and support further functionalities like GitLab Pages and Docker Registry replication.

Business survival in today's fast-paced environment hinges on a quicker DevOps lifecycle, yet securing applications at this speed presents challenges due to the resource-intensive nature of existing security tools and the need for continuous code testing. The dilemma lies in balancing the frequency and depth of testing with the speed and cost, especially with the risk of rollbacks due to unforeseen vulnerabilities. A shift towards proactive security is necessary, with solutions like GitLab offering integrated security tools that reduce context-switching and enhance collaboration between developers and operations. GitLab includes built-in security features such as SAST, DAST, container scanning, dependency scanning, and license management, allowing for automated testing of code commits early in the development process. The introduction of GitLab's Security Dashboard in version 11.1 aims to empower security professionals by providing visibility into vulnerabilities across projects, complementing the developer-focused features and reducing the adversarial nature of security and development interactions.

In late August, GitLab hosted its sixth summit in Cape Town, South Africa, bringing together 350 participants, including team members, significant others, community members, and customers, for a blend of work, networking, and exploration. The event featured a keynote speech by company leaders and traditional challenges set by CEO Sid Sijbrandij, which were met with enthusiasm and success. Participants had the opportunity to engage in various excursions, such as visiting Boulders Beach to observe African penguins, touring Robben Island to learn about South Africa's apartheid history, exploring the Cape Winelands, and experiencing the local culture in the town of Langa. The summit also included user-generated content sessions on diverse topics, allowing attendees to share knowledge and skills in an informal setting. As the gathering concluded, plans for the next summit in May 2019 in New Orleans were announced, highlighting the event's growth and continued emphasis on cultural exchange and professional development.

Initially, GitLab.com operated on a single server, which made deployment and maintenance straightforward, but as the platform grew, vertical scaling by adding more resources like RAM and CPU became insufficient. To address this, GitLab shifted to horizontal scaling, adding new servers and using NFS (Network File System) to mount repositories across application servers. However, NFS presented challenges such as reduced visibility and system-wide outages when a storage node failed. Seeking alternatives, GitLab considered Ceph but found it lacking in cloud performance, prompting a redesign of the application architecture with a new service called Gitaly to handle Git requests. Gitaly introduced a well-defined protocol using Protocol Buffers and gRPC, facilitating a transition away from NFS towards an HTTP boundary, with changes tested and monitored using Prometheus and the ELK stack to ensure stability. The migration allowed GitLab to maintain performance and stability across installations, gradually unmounting NFS with no impact on customers. While v1.0 of Git infrastructure was not officially tagged, the successful transition set the stage for future goals, including eliminating NFS dependency entirely and further enhancing Git data availability.

The GitLab team summit in Cape Town sparked reflections on the parallels between photography and application security testing, highlighting the importance of integration and accessibility. Just as a smartphone camera, despite not being as sophisticated as a dedicated camera, is preferred for its immediate sharing capabilities and ease of use, GitLab's application security testing is favored because it seamlessly integrates into the developers' workflow. This approach contrasts with traditional methods that require separate applications and systems, leading to inefficiencies. By embedding security testing directly into the development process, GitLab ensures vulnerabilities are identified and addressed without disrupting the workflow, aligning security with development priorities. This shift is akin to the "shift left" strategy in software development, aiming to address issues earlier in the lifecycle, thus enhancing efficiency and reducing costs. GitLab's integrated model is compared to Microsoft's strategy of integrating tools across its suite, suggesting a potential new era for application security where seamless integration takes precedence over isolated, superior tools.

Auto DevOps, introduced in GitLab's 11.0 release, is designed to provide users with a suite of automated features such as Auto Build and Auto Monitoring out of the box, adhering to GitLab's principle of defaulting valuable features to ON. While it supports projects in popular programming languages, some specialized projects may require additional configuration, prompting GitLab to disable Auto DevOps automatically if a pipeline fails, notifying the project owner for potential adjustments. GitLab plans to gradually enable Auto DevOps on gitlab.com and, after monitoring its performance, aims to include it in the 11.3 release for self-managed customers by September 22, 2018, ensuring widespread access to its benefits.

GitLab emphasizes the importance of iteration by implementing Minimally Viable Changes (MVCs) to quickly deliver value to users and gather feedback. These MVCs, which are small yet impactful updates, include features such as assignee and milestone lists for improved issue tracking, a redesigned merge request widget for better information digestion, a groups dropdown for smoother navigation, and a quick action to mark issues confidential. These changes demonstrate GitLab's commitment to enhancing user experience by making incremental improvements, encouraging community contributions, and maintaining agile development practices.

Jacopo Beschi, a member of GitLab's Core Team from Italy, has been contributing to GitLab since late 2016, initially drawn to the project for its open-source nature and Ruby-based development. He has contributed to various aspects of GitLab, particularly in backend, frontend, API, utility, and quality areas, although he hasn't yet worked on its Golang components like GitLab Runner, Gitaly, or GitLab Workhorse. Professionally, Jacopo is a technical lead at Iubenda, a SaaS company specializing in privacy and cookie policies, and he enjoys spending his leisure time at the gym and with family and friends. He advises new contributors to start by exploring the "Contributing to GitLab" page and emphasizes that contributing to GitLab not only enhances one's resume but also provides an opportunity to connect with knowledgeable individuals and make a global impact on developers.

Telesphora, a human-centered data science platform, emerged from a national opioid crisis code-a-thon organized by the United States Department of Health and Human Services (HHS) to address the opioid epidemic through data-driven solutions. The code-a-thon brought together specialists from various fields, including machine learning expert Jack Cackler and pain management specialist Frank Lee, who collaborated with Origami Innovations to win the Treatment Track. They developed a platform using real-time data and predictive analytics to forecast opioid overdose outbreaks, enabling proactive allocation of life-saving resources like naloxone. By employing a spatiotemporal Poisson process, Telesphora's model can predict the geographic and temporal movement of opioid spikes, helping first responders and health agencies prepare in advance. This approach not only aims to reduce mortality rates but also strives to decrease the stigma associated with opioid use by involving stakeholders and emphasizing empathy in design, ultimately allowing communities to better manage and prevent overdose crises.

The UX team at GitLab, comprising two UX researchers and nine designers from eight countries, collaborates remotely to enhance product design and usability. In a recent webcast, they discussed their workflow, emphasizing the importance of iteration to quickly gather user feedback and improve products. The team employs various UX research methods, such as surveys and user interviews, to understand user needs and works with product managers to prioritize improvements. A major initiative has been the development of the GitLab Design System, which aims to create consistency and predictability by providing content guidelines and reusable components. As an open-source company, GitLab encourages community contributions to its UX design through labeled issues and proposals, fostering a collaborative environment where community members can engage and influence the product's future. The team is also expanding, seeking new UX designers to join specific teams like Release and Verify, Monitor, and Secure, inviting interested individuals to apply and contribute to their vibrant design community.

The announcement of two new versions of Jenkins, Cloud Native Jenkins, and Jenkins Evergreen, aims to address limitations in the current legacy Jenkins, which is considered "trapped in a local optimum." The comparison of GitLab CI with these Jenkins variants reveals that both Jenkins and GitLab are open core projects with comparable governance and distribution mechanisms, although they differ in several operational aspects. GitLab CI is integrated within GitLab and offers seamless automatic restarts in case of memory issues, while Legacy Jenkins often requires daily manual restarts. GitLab CI's architecture is based on convention over configuration, ensuring functionality is integrated into the main code base, unlike Legacy Jenkins, which faces configuration fragility due to plugin dependencies. Cloud Native Jenkins is being redesigned with simpler primitives and utilizes Kubernetes as the runtime, whereas Jenkins Evergreen focuses on a curated set of essential plugins for better integration. GitLab CI also emphasizes a secure-by-default design and configuration as code, while Jenkins’ configuration requires master-level plugin setup, highlighting GitLab’s streamlined approach in comparison to the complex plugin ecosystem of Jenkins.