May 2019 Summaries

Working in the security team at GitLab, known for its transparency and remote work environment, presents both rewarding opportunities and unique challenges. Paul Harrison, a Senior Security Engineer at GitLab since 2019, details his role in operational security, emphasizing the importance of transparency in handling vulnerabilities and issues, which fosters community trust and collaboration. He highlights the challenges of maintaining transparency while ensuring sensitive data remains secure until addressed. Harrison argues for the importance of continued learning in cybersecurity, acknowledging that while foundational knowledge is critical, the rapidly evolving landscape requires ongoing education. He also shares personal preferences, such as favoring Debian as a Linux distribution and advocating for the use of password managers and two-factor authentication to enhance security. Additionally, he humorously clarifies his stance on the hotdog and taco debate, asserting they are not sandwiches due to their distinct structural differences.

In a rapidly evolving development environment, IT automation emerges as a crucial strategy for enhancing productivity and fostering a healthier DevOps culture by allowing developers to focus on projects that add significant business value. By automating repetitive and manual tasks, organizations can not only reduce costs but also improve employee satisfaction and retention, as developers are more engaged when working on strategic, challenging initiatives. Automation is essential for scaling workloads, as it enables the seamless deployment and management of thousands of servers, which would be impractical with manual processes. Furthermore, it minimizes the risk of human error, thereby allowing IT teams to concentrate on mission-critical tasks. Automation tools, such as Kubernetes, aid in efficient security management as organizations scale their use of containers and microservices. The integration of CI/CD processes connects developers and operations, accelerating software delivery and enhancing collaboration. Companies like Monkton exemplify the benefits of automation by streamlining testing, deployment, and security processes, leading to increased efficiency and repeatability. Ultimately, IT automation not only supports innovation and growth but also creates an environment where developers can leverage their skills effectively, contributing to long-term organizational success.

GitLab has announced a partnership with Salesforce to enhance agile delivery practices and automation throughout the DevOps lifecycle for developers using Salesforce DX. This collaboration allows developers to utilize GitLab's Source Code Management, Continuous Integration, Continuous Delivery, and Project Management tools to improve the speed and efficiency of Salesforce app development. By integrating GitLab's CI/CD pipelines with Salesforce's Scratch Orgs and packaging, developers can enjoy a streamlined and collaborative workflow similar to GitLab Flow, enabling incremental changes and automated testing in a Git-based environment. To support this initiative, a new Salesforce project template and a Trailhead module titled "Build an automated CI/CD pipeline with GitLab" have been introduced, providing developers with hands-on learning opportunities to automate their Salesforce development and earn a Salesforce/GitLab Tanuki badge.

Upgrading GitLab to Rails 5 was a complex process due to the project's large codebase, necessitating a strategic approach that involved running the application on both Rails 4 and 5 using environment variables. This allowed the team to address specific issues incrementally, reducing CI failures with each fix and facilitating collaboration. By enabling compatibility with both versions, the team could maintain backward compatibility while testing new features, ultimately ensuring the transition was smooth and without production issues. The process also involved addressing Active Record changes and refactoring various monkey patches, with CI pipelines running on both Rails 4 and 5 to catch new incompatibilities. Although the upgrade took longer than expected, it was deemed successful, and the experience gained is expected to inform future upgrades, including the planned transition to Rails 6.

Gartner recognized GitLab as a 'Visionary' in its Magic Quadrant for Enterprise Agile Planning Tools, highlighting the company's innovative approach to integrating planning and delivery processes. GitLab aims to streamline collaboration among product managers, developers, operations, and security teams through its Concurrent DevOps vision, which facilitates building and managing epics, roadmaps, and organizing sprints within a single application. By enabling a seamless connection between planning and delivery, GitLab reduces cycle times and increases productivity, as evidenced by Hemmersbach's experience of cutting down planning to production time by 6.5 days and achieving 60 builds per day. The platform's features, such as Kanban boards and Value Stream Management, help manage the flow of issues and identify bottlenecks, ensuring that teams work efficiently on the right tasks at the right time. Gartner's report, however, clarifies that its evaluations are based on the organization's opinions and not endorsements of any vendor, emphasizing the subjective nature of its research.

Monkton Incorporated, a company focused on creating secure mobile solutions, transitioned to GitLab to streamline their development process, consolidating various tools and reducing manual tasks. CEO Harold Smith explained how GitLab's comprehensive offerings, including continuous integration and secure DevOps, allowed Monkton to save time and improve project management by automating processes like code scanning and deployment. The migration to GitLab also provided the organization with better documentation and support, facilitating rapid environment setup and execution. Smith emphasized the importance of automation in reducing human error and freeing up employees for more strategic tasks, while maintaining a repeatable process to ensure consistent outcomes, especially in high-security contexts such as those required by Department of Defense customers. The transition has been a collaborative learning experience, offering insights into best practices for improving processes and compliance, with potential benefits for the broader community.

Fluentd, an open-source data collector and the latest project to graduate from the Cloud Native Computing Foundation (CNCF), announced its use of GitLab CI/CD for continuous integration at KubeCon Barcelona. Fluentd, known for unifying data collection and consumption, is highly popular with over 7,868 stars on GitHub and more than 100K daily downloads, supported by a community that has contributed over 900 plugins. Its lighter counterpart, Fluent Bit, caters to environments with strict memory constraints and focuses on stream processing on the edge, which is essential for real-time data analysis in large-scale, distributed systems and IoT applications. The traditional data analysis model, involving storage and indexing in a cloud database, often falls short for time-critical decision-making due to latency issues. Fluentd's efforts in edge processing have gained significant community traction and are already integrated into many observability workflows. The project's graduation from CNCF, which requires a diverse contributor community and security audits, is expected to encourage further enterprise adoption. GitLab has been collaborating with Fluentd on its continuous integration needs, gaining insights into the challenges of fast-paced CNCF projects and offering support to optimize their CI/CD pipelines.

GitLab, in collaboration with Crossplane, is showcasing a groundbreaking deployment of a real-world application across multiple clouds using Crossplane's open-source multicloud control plane. This initiative highlights GitLab's commitment to multicloud DevOps, enabling users to deploy applications entirely through the Kubernetes API into various clouds, integrating fully-managed services from cloud providers. Crossplane enhances the Kubernetes API by introducing resource claims and classes, facilitating the composability of managed service dependencies, thus simplifying multicloud management and increasing application portability. By abstracting managed services and integrating them with Kubernetes, Crossplane allows developers to deploy complex applications across different cloud providers, while operational teams manage infrastructure based on policies and business needs. This approach not only demonstrates a significant advancement in multicloud maturity but also aligns with the growing multicloud strategies adopted by enterprises, aiming to reduce complexity and development costs. GitLab is actively engaging with the community at KubeCon Barcelona, promoting this multicloud strategy and exploring its implications for future enterprise solutions.

Joe Beda, the Principal Engineer at VMware and co-creator of Kubernetes, discusses the evolution and importance of open source communities in a conversation with GitLab CEO Sid Sijbrandij as part of the TechExplorers series. Beda emphasizes that open source projects like Kubernetes thrive because of their vibrant, diverse communities, and he highlights the significance of maintaining a social contract with these communities to ensure ongoing success. He notes the blurring lines between product and project in open source, suggesting that transparency and community engagement are crucial for sustaining trust. Beda reflects on Kubernetes' journey from a Google project to a major open source tool, underscoring the community's role in its success and the innovation it continues to inspire. As enterprises transition to cloud solutions, he advocates for independent support and advice to navigate the changing landscape, hinting at the exciting future potential of open source projects driven by community buy-in and experimentation.

KubeCon in Barcelona brings together over 12,000 attendees to explore the latest developments in Kubernetes and related cloud-native technologies. Once a small community with just five projects, the CNCF now hosts 36, including Fluentd, which recently graduated alongside other notable projects like Kubernetes and Prometheus. The event highlights GitLab's integration with Kubernetes, emphasizing how both began as open-source initiatives and have grown significantly in scope and influence. With a focus on the developer workflow, GitLab enhances efficiency and collaboration in cloud-native environments, enabling seamless CI/CD processes. Attendees are invited to explore various sessions, tutorials, and networking opportunities, including discussions on serverless computing, zero trust security, and the future of CI/CD. The event also features interactive activities like the #tanukiadventure, encouraging engagement with GitLab's partners for a chance to win exclusive merchandise.

GitLab has been recognized as one of Inc. Magazine's Best Workplaces in 2019, highlighting its exceptional workplace culture, employee engagement, and comprehensive benefits. This recognition was based on an application process and an anonymous employee survey evaluating confidence in the future, management effectiveness, and perks. Employees from various roles and locations shared their experiences, emphasizing GitLab's industry-changing product velocity, commitment to community engagement, remote work flexibility, and transparency. Team members appreciate the reduced stress, enhanced work-life balance, and the opportunity to work with a diverse global team. GitLab's culture supports personal development, encourages taking time for family, and promotes a "family first" mantra, contributing to high job satisfaction and pride in being part of the team.

Atlassian Bitbucket, GitHub, and GitLab have issued a joint blog post to educate users on secure practices following a Git ransomware incident involving account compromises across all three platforms. These compromises, discovered on May 2, were due to unintended user credential leaks, likely from external systems, leading to public and private repositories being held for ransom. The companies' security teams acted swiftly to investigate, notify affected users, and mitigate further damage, including invalidating compromised credentials and encouraging the use of multi-factor authentication. The incident highlighted the risks of exposed credentials and emphasized the importance of secure credential management, with all three platforms providing guidance on preventing unauthorized access and offering robust security features to protect user accounts.

Kelsey Hightower, a prominent figure in the tech industry and Staff Developer Advocate at Google, emphasizes the enduring importance of computing fundamentals amid the proliferation of new technologies such as Kubernetes and serverless architecture. In a conversation with GitLab CEO Sid Sijbrandij for the TechExplorers blog series, Hightower discusses the pitfalls of an "all-or-nothing" approach to technology adoption, advocating instead for selecting the most suitable platforms for specific tasks. He stresses the need for high-level interfaces that minimize server interaction, allowing developers to focus on solving real problems instead of managing infrastructure. Hightower also highlights the necessity of writing reliable code and understanding the trade-offs involved when transitioning to cloud-native platforms. He maintains that discipline remains crucial in software development, regardless of whether one is dealing with monolithic or function-based architectures, as neglecting this can lead to disorganized and inefficient systems.

GitHub has announced the public beta of its package registry, marking a step toward integrating more DevOps tools into a single application experience, following its previous Actions announcement. This move appears to parallel GitLab's approach, which has been offering a unified application for the DevOps lifecycle since 2012, incorporating integrated packaging features like Docker registry in 2016 and Maven and NPM in 2018. GitLab continues to enhance its offerings with secure and auditable package management features, such as a Dependency Proxy, which can block or delay suspect packages and trace the use of vulnerable ones, thereby improving performance, cost efficiency, and deployment stability. GitLab emphasizes the value of a comprehensive DevOps application, noting that it provides package registries and features across all ten stages of the DevOps lifecycle, including deployment, security, and monitoring, with GitHub's new features seen as a positive development post-acquisition by Microsoft.

Contribution graphs in GitLab provide a visual representation of a user's year, capturing not only professional milestones but also personal events like vacations and family leave through unique patterns of activity. These graphs include not just code commits but also actions such as commenting, and opening and closing issues, making them as distinctive as fingerprints. GitLab encourages users to share their contribution stories, emphasizing that everyone can contribute, whether through software development or other forms of work and management. Users are invited to tweet screenshots of their activity using the hashtag #contribute for a chance to win GitLab merchandise, highlighting the diverse ways people engage with the platform.

Smaller companies often struggle with information security compliance due to the complexity of integrating multiple frameworks into established processes, but GitLab's early adoption of a formalized compliance program during their Series C funding allowed them to proactively address this challenge. By choosing an "umbrella framework" approach, leveraging Adobe's open-source Common Controls Framework (CCF), GitLab efficiently aggregates security controls from various industry standards such as ISO, SOC, and PCI, reducing redundancy and increasing internal efficiency. This approach enables GitLab to create comprehensive security controls, streamline requests to internal teams, and maintain credibility within the organization. By aligning the compliance process with their product workflow, GitLab has rapidly developed a robust compliance program and plans to share their insights and resources with the wider community to support other organizations in their compliance efforts.

As businesses adopt new technologies to enhance efficiency, the integration of security into development processes has become crucial, resulting in a "shift left" approach to application security testing. This strategy aligns with methodologies like DevOps and Agile, requiring continuous testing due to constant updates, and is essential in environments involving serverless, cloud-native, container, and Kubernetes deployments, which present increased attack surfaces and complexity. The emphasis on security transcends traditional network-perimeter approaches, advocating for a Zero Trust model that encompasses data, networks, endpoints, and application infrastructure. The implementation of DevSecOps is recommended to embed security within each software iteration, automating security scans at every code commit to resolve vulnerabilities early in the software development lifecycle. Simplifying and integrating security tools, such as using platforms like GitLab that allow development and security processes to occur in the same interface, enhances the likelihood of consistent security practices, ensuring applications remain protected as they evolve.

Suspicious Git activity involving the use of valid credentials on GitLab has been identified, following a support ticket by Stefan Gabos. The issue, which is not specific to GitLab, affects other git repositories and involves attackers having knowledge of users' passwords to wipe repositories and demand ransoms. GitLab has notified affected users and recommends using password managers, enabling two-factor authentication, and employing SSH keys for enhanced security. Mitigation efforts include restoring repositories using git commands if local copies are available, while a detailed investigation continues. The attacks, traced to the IP range 185.234.216.0/24, were linked to plaintext passwords stored locally and affected 131 users and 163 repositories, primarily private, with none having two-factor authentication enabled. The attacker's script, suspected to be generic and possibly malfunctioning, targeted platforms like GitLab, GitHub, and Bitbucket. GitLab is committed to transparency in its continued investigation and encourages community feedback to bolster security measures.

Cloud Run, introduced at Google Cloud Next, is a service that leverages Knative to simplify the deployment of serverless workloads on Kubernetes, either as a fully managed service or integrated with Kubernetes clusters via "Cloud Run on GKE." GitLab, a Google Cloud partner, demonstrated its integration, highlighting how it facilitates the use of Knative for deploying Functions-as-a-Service or serverless applications. GitLab's solution automates the configuration of Istio-Ingress endpoints and DNS settings, allowing developers to focus on application development rather than infrastructure management. It offers single-click provisioning, automatic building, and deployment of applications or functions as Knative services, along with endpoint provision and load metrics. This collaboration between GitLab, Google Cloud, and the Knative community aims to enhance functionalities available to GitLab users, emphasizing their commitment to open-source technology.

Facing challenges with high error rates and low availability, GitLab decided to migrate its SaaS infrastructure from Azure to Google Cloud Platform (GCP) to better accommodate mission-critical workloads. The migration process was documented publicly, highlighting GitLab's values of efficiency, iteration, and transparency. Initially, a straightforward data transfer was considered but deemed impractical due to the vast amount of data involved. Instead, GitLab leveraged its Geo feature for replication, ensuring minimal downtime during the transition. Though plans to integrate GitLab with Kubernetes were postponed, the migration involved using Omnibus for environment provisioning and moving significant data to Google Cloud Storage. A meticulously iterative approach was employed to perfect the failover process, resulting in enhanced site availability and reduced error rates post-migration. The project not only improved GitLab's performance but also provided valuable insights into large-scale migrations, reinforcing the company's commitment to collaboration and continuous improvement.

Automation in testing is becoming increasingly vital across industries, especially as development speeds up. Test automation is crucial for deploying applications quickly and effectively, with many organizations moving towards automating more than 50% of their testing processes. Continuous testing is gaining importance within DevOps and continuous delivery models, addressing the risks associated with traditional end-of-cycle testing by integrating testing from the start of development. Concurrent DevOps further aids in maintaining code quality and speed by utilizing parallelization and cloud-based solutions, enabling efficient resource use. Additionally, AI and machine learning are transforming test automation by making it more predictive and reliable, although legacy applications still pose challenges due to their complexity and reliance on manual testing. Test automation allows testers to focus on enhancing user experience, which is crucial in a competitive app market, but legacy systems that cannot leverage these technologies risk falling behind.