October 2017 Summaries

GitHub Marketplace, launched in May, offers a platform for discovering and purchasing tools that enhance the development process, including continuous integration, project management, and code review. The introduction of free 14-day trials for featured apps allows users to experiment and determine the best tools for their workflow without immediate financial commitment. This initiative aims to simplify the integration of new tools, enabling teams to refine their processes and collaborate more effectively. The Marketplace hosts numerous integrations compatible with GitHub, with the selection expanding monthly, encouraging teams to explore and optimize their development strategies.

GitHub has introduced a new set of project automation events to streamline workflows by automatically updating project boards. These features include moving cards between columns when specific actions occur, such as adding a new card, closing an issue, merging or closing a pull request, and reopening an issue or pull request. The aim is to enhance efficiency by reducing manual updates and enabling teams to focus on more critical tasks. Users are encouraged to explore the documentation for implementing these automations and to provide feedback on additional features that could further improve their development processes. Future plans include expanding automation capabilities to cover other developer workflows, like changes in review or build status on pull requests.

GitHub has advocated for the inclusion of open source provisions in the 2018 National Defense Authorization Act (NDAA) by writing a letter to the United States Senate and House of Representatives. These provisions, supported by bipartisan senators Mike Rounds and Elizabeth Warren, would mandate that unclassified, non-defense software developed for the Department of Defense (DoD) be open source unless otherwise specified. This aligns with the DoD's historical use and creation of open source software and recent initiatives like Code.mil, which aims to modernize software practices through open source engagement. The broader adoption of open source by U.S. federal agencies, including the NSA and NASA, exemplifies a trend toward using such software to enhance efficiency, innovation, and security. GitHub's 2017 Octoverse report underscores the role of open source as essential infrastructure, embraced by developers and companies worldwide to boost innovation and security, and highlights how government involvement with open source can bolster economic competitiveness and national security while saving taxpayers money.

At the Hasso Plattner Institute in Potsdam, Germany, a unique computer science education approach emphasizes practical and engineering-oriented learning through real-world software development challenges. Researchers Arian Treffer and Christoph Matthies encourage students in the "Software Engineering II" course to embrace mistakes, assess their software development process, and collaborate within teams to deliver optimal results. Students participate in a project where they develop a single system in small teams, utilizing open-source practices and engaging with continuous integration tools like Professor CI on GitHub. The course also challenges students to gather and manage requirements from a simulated customer, honing their skills in communication, negotiation, and problem-solving. Reflection exercises such as the Sailboat activity help students analyze their teamwork dynamics and improve their development processes, ensuring that they leave the program well-equipped to handle real-world software engineering challenges.

Game Off, GitHub's fifth annual game jam, is set to begin in two weeks, and unlike typical 24-72 hour game jams, it will run for the entire month of November. Participants will have 30 days to create a game inspired by a theme announced on November 1st, with the freedom to use any technology or open-source game engines, libraries, and tools. This year's event will be hosted on itch.io, a platform for indie game developers, where participants can also judge the entries. Updates will be shared on GitHub's blog and Twitter, and participants are encouraged to follow along with the #GitHubGameOff hashtag. The event is organized by Lee Reilly from GitHub Developer Relations, who is known for his involvement in hackathons and support for developer communities.

Nearly four years after its inception, the Bug Bounty program is significantly increasing its payout amounts, with minimum and maximum rewards now set at $555 and $20,000, respectively. This adjustment aims to align with top security bug bounty programs and coincides with the annual Hack the World competition by HackerOne, which encourages participants to identify security vulnerabilities. As a sponsor of the event, the program offers twice the reputation points on HackerOne for bugs found on GitHub until November 18th. Additionally, valid submissions during this period will receive free unlimited private repositories for life, though this offer is limited to reports filed by the competition's end date. Participants are encouraged to engage in the competition by submitting their reports to the Bug Bounty program and can find more information on the Hack the World website.

Open source maintainers are encouraged to prepare their repositories for Hacktoberfest, a time when communities grow significantly, as evidenced by experiences like that of Peter Tseng, who became a core contributor to Exercism through the event. To facilitate contributions, maintainers should perform maintenance on documentation, including adding a clear README, a CONTRIBUTING.md file, a Code of Conduct, and a suitable license using resources like choosealicense.com. Enhancing searchability by tagging repositories with relevant topics, particularly #Hacktoberfest, and labeling issues and pull requests accordingly is also recommended to attract suitable contributors. Such preparations can lead to meaningful engagement and contributions, as demonstrated by the large number of pull requests received by projects like Home Assistant during past Hacktoberfests.

Four new applications have been introduced to the GitHub Marketplace to enhance code review, security, and monitoring processes. AccessLint integrates automated web accessibility testing into development workflows by reviewing pull requests for accessibility issues, providing timely feedback. Dependabot helps manage Ruby, JavaScript, Python, and PHP dependencies by identifying outdated requirements and opening pull requests to update them. Greenkeeper offers real-time monitoring for npm dependencies, ensuring safety and consistency by sending actionable pull requests and issues. LogRocket provides a unique monitoring solution by allowing developers to replay problems with a video recording that captures user interactions, console logs, network requests, and application states. These tools aim to empower developers with enhanced capabilities and flexibility in evolving their workflows.

More than 1,000 developers gathered at Pier 70 in San Francisco for GitHub's flagship conference, where they participated in workshops, listened to industry experts discuss the future of software development, and explored new GitHub products. Key highlights included product updates from GitHub executives such as the introduction of a dependency graph for tracking code dependencies, a "Discover repositories" feed for smart recommendations, and a review of notable projects and trends from the past decade. The conference featured 40 sessions with insights from various tech fields, along with hands-on workshops on technologies like Electron and command line skills. The event concluded with a benefit concert by Neon Trees supporting Maven, a nonprofit aiding LGBTQ youth, and was supported by numerous business and community sponsors who contributed to its success. The conference celebrated the contributions of the GitHub community, emphasizing the significant achievements in software development over the past ten years.

GitHub employs MySQL as its primary database system, utilizing master-replica setups to handle traffic, with a preference for reading data from replicas to scale efficiently. Challenges arise from replication lag, causing stale data on replicas, which GitHub mitigates through batching large operations into smaller tasks and throttling. This involves pausing between batches to ensure replication lag is within acceptable margins. Traditionally, GitHub used various throttling mechanisms, including a Ruby-based throttler and tools like pt-archiver and gh-ost, each with limitations in handling the scaling infrastructure. To address these, GitHub developed freno, a centralized throttling service that monitors replication lag continuously and recommends appropriate actions to applications, allowing for more efficient traffic management and reduced master reads. Freno operates independently of apps, providing recommendations rather than direct control, and is highly available through a raft cluster. It has significantly reduced read load on masters by enabling safer reads from replicas and improved overall performance by integrating with GitHub's infrastructure and being open-source for community use.

GitHub's Spokes system enhances the distribution of Git repositories across widely separated datacenters, replacing the older DRBD filesystem block-level replication with a more flexible Git application-level replication approach. Spokes ensures data integrity by maintaining multiple replica copies, utilizing strategies such as the three-phase commit protocol to handle high-latency challenges and optimize both read and write operations across distant replicas. These advancements enable GitHub to sustain high update rates for its nearly 70 million repositories, even when faced with complex workflows and internal bookkeeping tasks. By incrementally computing checksums to verify synchronization and prioritizing user-initiated updates, Spokes enhances the efficiency and reliability of reference updates. This system not only improves disaster resilience but also accelerates Git read operations for users by directing them to the nearest synchronized replica. The implementation of Spokes in both GitHub.com and GitHub Enterprise provides significant benefits, including improved speed, robustness, and the ability to handle geo-replication, thereby enhancing user experience and operational flexibility.

Over the past 18 months, GitHub has significantly invested in enhancing its physical infrastructure to improve redundancy and global availability, which is crucial for supporting its expanding user base. This effort includes the development of four facilities, comprising two points of presence (POPs) and two data centers, organized in a hub-and-spoke design with a dedicated backbone between Seattle and northern Virginia for consistent latencies and throughput. The POPs focus on internet and backbone connectivity without storing customer data, while the data centers, located in standalone cages in less connected facilities, are dedicated to customer data storage and request handling. GitHub emphasizes uniformity in cabinet types to ensure repeatability and ease of expansion, with structured cabling and pre-labeled patch panels simplifying new capacity addition. Through collaboration with a cabinet integrator, the process of expanding compute and storage capacity is streamlined, allowing for rapid deployment and provisioning of new cabinets. These infrastructure enhancements have positioned GitHub to continue its growth confidently and are supported by the adoption of a repeatable expansion process that decouples regional network edges from compute and storage systems.

GitHub is enhancing its infrastructure to efficiently manage network traffic and improve performance by focusing on DNS routing, transit, and peering connections. Transit refers to connections with Internet providers, whereas peering involves direct connections between networks, notably through private network interconnects (PNI) and Internet exchanges (IX), which facilitate cost-effective and high-performance traffic routing. By expanding transit capacity and integrating a second region, GitHub has mitigated the impact of volumetric DDoS attacks and improved latency through fewer hops between networks. The use of GeoDNS enables directing traffic to the closest region, optimizing network routing based on geographic location. Peering now accounts for a significant portion of GitHub's traffic, emphasizing its importance in performance and cost savings. The company is actively seeking to expand its team with Site Reliability Engineers and an Engineering Manager to continue advancing its network and traffic engineering capabilities.

GitHub has enhanced its integration capabilities by partnering with AWS CodeStar, allowing users to manage their software release workflow, including code commits, builds, and deployments for AWS applications, directly within AWS CodeStar's continuous integration and continuous deployment (CI/CD) toolchains. This integration simplifies project management by enabling the use of GitHub as a version control system and providing a centralized dashboard to track commits, issues, and pull requests across the CI/CD toolchain. Kyle Daigle, GitHub's Chief Operating Officer, plays a key role in leading GitHub's efforts to foster a developer-first culture and drive growth, AI adoption, and strategic collaborations since joining the company in 2013. His background in engineering and product leadership at various startups, along with his personal interests in home automation and nonprofit technology initiatives, underscores his commitment to advancing software development practices at GitHub.

GitHub, initially established as a collaborative platform for developers, has evolved into a global space fostering innovation, education, and business creation with millions of users and 1.5 billion commits. At the GitHub Universe event, the platform announced plans to leverage its vast open-source data to enhance collaboration, featuring tools like a dependency graph and security alerts to manage project dependencies and vulnerabilities. The dependency graph currently supports Ruby and JavaScript, with Python on the way, and security alerts will notify users of vulnerabilities and suggest fixes. The platform also introduced an enhanced news feed and redesigned Explore experience, enabling users to discover new projects and communities tailored to their interests. Upcoming features include Premium Support for GitHub Enterprise, a Community Forum, and a Marketplace trial program, all aimed at personalizing the user experience and fostering a more connected developer community. These developments signify the beginning of utilizing GitHub insights to improve workflows, with the potential for users to write better code and fix bugs more efficiently, thanks to the collective contributions of the GitHub community over the past decade.

In anticipation of the GitHub Universe conference, several community partners were recognized for their role in distributing scholarship tickets and promoting diversity and inclusion within the tech industry. Organizations like Code Tenderloin, /dev/color, Older Women Coders, and others shared their perspectives on why diversity is crucial, emphasizing that varied backgrounds and experiences are essential for solving societal problems through technology. These partners highlighted the importance of inclusivity across race, gender, age, and cultural backgrounds, noting that a diverse workforce leads to better problem-solving and innovation. Speakers from these organizations expressed excitement about participating in the conference, which focuses on ensuring that individuals from diverse backgrounds have the opportunity to learn and collaborate with top engineers. For those unable to attend in person, the event offers remote participation options through viewing parties in select cities and a livestream.

Security is a critical component for engineering organizations, particularly in regulated sectors like the automotive industry, and GitHub Solutions Engineer Phil Holleran provides insights on enhancing security and compliance workflows in a webcast titled “Driving secure, collaborative development.” Key strategies include enforcing multi-factor authentication (MFA) to prevent unauthorized access, regularly auditing authentication methods, and using GitHub Apps to manage permissions effectively. Additionally, protecting code integrity is emphasized through features like branch protection to prevent unauthorized modifications, and the new code owners feature, which automates the assignment of code reviewers. These measures aim to streamline security processes while maintaining a collaborative development environment.

October's Cyber Security month introduces a variety of projects and releases aimed at enhancing security, efficiency, and creativity within the GitHub community. Notable highlights include Brakeman 4.0.0, a static analysis tool for detecting security vulnerabilities in Ruby on Rails applications, and OWASP Juice Shop 5.0.0, a deliberately insecure web application for learning about common vulnerabilities. Vagrant 2.0 offers a streamlined approach for creating and configuring development environments, while Stories Untold Update #3 presents an experimental text adventure game with new macOS support. Voyager 1.0 assists with administrative tasks for Laravel applications, and AirSim 1.0, developed by Microsoft Research AI, provides a platform for experimenting with AI in autonomous vehicles. React v16.0 marks a significant update with its new core architecture, Fiber, and Yarn 1.0 brings enhancements to JavaScript dependency management. JGProgressHUD 2.0 offers easy-to-use progress HUDs for iOS and tvOS apps, and Swift 4.0 introduces new features for developers, celebrated by the GitHub community and its contributors.

GitHub's project boards now offer enhanced functionality with the introduction of summary cards, which provide a comprehensive view of work without leaving the board. Users can reference issues or pull requests in notes, and these will appear as previewed cross-referenced links, displaying valuable context such as assignee, state, and labels. This feature helps track external dependencies more effectively by allowing users to add links in notes, which will either convert URLs into first-class cards if relevant to the project or display them as summary cards if not. This new feature is automatically applied to existing boards, requiring no additional action from users.

The flagship community conference is approaching, with tickets nearly sold out, offering attendees the opportunity to learn from industry experts through over 40 breakout sessions and gain insights into new GitHub products from executive keynotes by Co-Founder Chris Wanstrath and SVP Technology Jason Warner. The conference will also feature the Universe After Party at Mezzanine, supporting the nonprofit organization Maven, with a performance by Neon Trees. For those unable to attend in person, viewing parties will be available in three European cities, allowing developers to connect with their communities and experience the event remotely.