June 2021 Summaries

GitHub Copilot, an AI tool trained on extensive public code, is evaluated for its tendency to suggest code snippets that are directly derived from its training data. An internal trial involving nearly 300 GitHub employees using the tool in their daily work highlighted that while most of Copilot's suggestions are unique, a small percentage are direct recitations from its training set, commonly appearing in generic contexts or at the initial stages of file creation. The study found that such recitations occur roughly once every ten user weeks, predominantly involving code snippets frequently encountered in public repositories, like the GNU General Public License. The research suggests implementing a UI feature to indicate when a suggestion is a direct quote, allowing users to decide on appropriate attribution. Future efforts aim to reduce recitation rates and enhance detection precision, with the intention of integrating these improvements into GitHub Copilot's technical preview.

GitHub has launched a technical preview of GitHub Copilot, an AI pair programmer designed to enhance coding efficiency by suggesting lines or entire functions based on the code context. Developed with OpenAI, it uses the OpenAI Codex, a powerful AI system trained on an extensive dataset of public source code, making it more adept at code generation compared to GPT-3. GitHub Copilot assists developers in discovering alternative solutions, writing tests, and exploring new APIs without extensive internet searches. It supports various programming languages and frameworks, with a particular emphasis on Python, JavaScript, TypeScript, Ruby, and Go during this preview phase, and interested users are encouraged to sign up due to limited availability.

Since its launch in 2014, GitHub's Security Bug Bounty Program has become a crucial element in enhancing the security of GitHub's software, with significant contributions from security researchers and the bug bounty community. The program, which saw its busiest year from February 2020 to February 2021, awarded $524,250 for 203 vulnerabilities, increasing the total rewards to over $1.5 million since joining HackerOne in 2016. Notably, the program mitigated a universal open redirect vulnerability that had the potential to compromise the OAuth flow of Gist users, showcasing the creativity and expertise within the community. Additionally, GitHub became a CVE Number Authority (CNA), allowing it to issue CVEs for vulnerabilities in GitHub Enterprise Server, which aids in communicating updates and prioritizing upgrades for customers. The company also expanded its private bug bounty programs to address security in new and beta products, such as GitHub Pages and GitHub Enterprise Server 2.22, ensuring any vulnerabilities are addressed before public release. Looking forward, GitHub continues to invest in its security initiatives, including the formation of a new team dedicated to the bug bounty program and the development of additional private programs, emphasizing the importance of collaboration for improving platform security.

The GitHub Virtual Event Kit, a new addition to the Student Developer Pack, offers students around the world free access to a suite of tools for organizing and running virtual events, aiming to enhance educational and community experiences. Responding to the challenges faced in hosting online events, the kit includes resources and support from partners like Discord, Canva, StreamYard, Namecheap, Name.com, and Microsoft Azure. These tools help streamline the organization of hackathons, meetups, and live-coding streams, allowing students to focus on creativity and engagement without technical hurdles. The initiative emphasizes GitHub Education's commitment to supporting student-led events by providing essential resources that ensure smooth and successful virtual gatherings. Access to the Virtual Event Kit requires membership in the Student Developer Pack, which is open to verified students aged 13 and above, providing them with opportunities to learn and use real-world development tools.

As teams and projects expand, the need for flexible and adaptable tools becomes paramount, prompting the introduction of new beta features in GitHub Issues to integrate planning with active work seamlessly. These features include project tables resembling spreadsheets, custom fields, a command palette navigable via keyboard, improved task lists, and issue forms, aiming to prevent plans from becoming disconnected or outdated. Mario Rodriguez, the Chief Product Officer of GitHub, leads these initiatives, drawing on his extensive experience in developer tools and AI strategy, including his pivotal role in the growth of GitHub Copilot. Outside his professional endeavors, Rodriguez is dedicated to educational advancement, evidenced by his co-founding of a charter school to enhance education in rural U.S. regions.

GitHub has introduced a new service to automatically generate Open Graph social sharing images for repositories, issues, pull requests, and commits, improving the visual and informational content shared on platforms like Twitter. The service uses a Node.js application and the GitHub GraphQL API to collect relevant data, which is then transformed into HTML and converted into an image using Puppeteer. To enhance performance, the development team replaced Puppeteer's default `networkIdle0` setting with a custom DOM-based approach, reducing idle time and boosting rendering speed by adjusting memory settings. This optimization allows the service to efficiently generate around two million unique images daily, with a significant portion being served from cache, providing a more engaging sharing experience for GitHub users.

GitHub has officially launched its Container registry as part of GitHub Packages, providing a comprehensive platform for developers to publish, manage, and consume containers with improved features since its beta phase. The Container registry supports various capabilities such as anonymous access for public containers, organizational ownership, fine-grained permissions, and integration with GitHub Actions workflows, offering a seamless experience for working with containers. Existing Docker containers are being migrated to this new registry without impacting current workflows, and although usage remains free for now, billing will soon follow the standard GitHub Packages pricing structure. The registry has already been adopted by notable projects like Homebrew and Home Assistant, which benefit from its OCI image support and integrated management features, simplifying their operations and enhancing discoverability and distribution. The unified platform allows developers to manage container images alongside their source code, reducing complexity and infrastructure sprawl, as evidenced by the experiences of The Helm Project with their Tiller application.

GitHub Desktop's recent updates enhance user experience by expanding drag-and-drop functionality, allowing users to squash and reorder commits, amend previous commits, and start new branches from earlier commits, thereby improving commit history management. These features enable developers to maintain a coherent project narrative by consolidating related commits and easily reorganizing them. Additionally, users can now create branches from any point in their history to investigate issues or build hotfixes without affecting current work. The update also includes native support for Apple Silicon, offering improved performance on devices with the M1 chip, thanks to contributions from Dennis Ameling.

In May, GitHub introduced 20 new features, highlighting significant updates such as video uploads in various sections like issues and pull requests, and single-click branch syncing for forks. Key updates include the native build of GitHub Desktop for Apple silicon, enhancements in GitHub Discussions, and the addition of a dark mode to GitHub Docs. GitHub Enterprise Server 3.1 was released as a candidate, with new compliance reports for the Enterprise Cloud. Security updates included new notification controls for security alerts and improvements to Dependabot. GitHub also enhanced its mobile application, repository management, and introduced new features in GitHub Sponsors and the GitHub Advisory Database. Users can explore these updates further on the GitHub Blog and Changelog.

Blacktocats, GitHub's first official Employee Resource Group, began in 2015 as a small initiative to amplify Black voices and talent in the tech community and has since grown to over 60 members from diverse geographical locations. This group has been instrumental in shaping GitHub's culture and products, with notable achievements such as the creation of the Black Octocat, Benjamin Bannekat, in honor of Benjamin Banneker's legacy. Blacktocats also organize events like the FLOSS and Code™ Hack Day to empower Black engineers in open source contributions, and they played a crucial role in inspiring the ReadME Project, which highlights Black excellence in open source. Their efforts are part of GitHub's broader commitment to fostering a community that reflects global diversity and innovation, supported by six other communities of belonging at GitHub.

Polkit, a system service used by many Linux distributions that employ systemd, was found to have a privilege escalation vulnerability, CVE-2021-3560, which allows an unprivileged local user to gain root access. The vulnerability, discovered by a member of the GitHub Security Lab, was introduced in polkit seven years ago but only recently affected several major distributions such as RHEL 8 and Ubuntu 20.04. The exploit is relatively simple and involves interrupting a specific sequence of authorization checks managed by polkit and dbus-daemon, leading polkit to mistakenly authorize a request as if it came from a root process. This flaw is particularly dangerous as it can be exploited using standard command-line tools, and the proof of concept provided illustrates how this can be achieved by leveraging certain polkit clients like accountsservice and gnome-control-center. The vulnerability was fixed by the polkit developers following its disclosure, and users are advised to update their systems promptly to mitigate the risk.

GitHub secret scanning, a service operational since 2015, is crucial for securing code by detecting and revoking leaked credentials, especially those related to package registries, to prevent widespread software compromise. This initiative has recently expanded to include collaboration with PyPI and RubyGems, alongside existing support for npm, NuGet, and Clojars, to protect the vast open-source ecosystem from vulnerabilities that could affect millions of dependent applications. By automatically scanning public repositories for exposed secrets and working with over 40 cloud providers, GitHub notifies relevant registries to revoke compromised credentials, thereby safeguarding the open-source supply chain. This proactive approach not only secures individual accounts but also prevents potential catastrophic impacts on downstream applications. GitHub continues to enhance this service by incorporating more secret types and collaborating with additional package registries and cloud providers to bolster security measures.

The latest Release Radar highlights numerous innovative community projects, showcasing a diverse range of tools and technologies, each marking significant advancements in their respective fields. Notable releases include QuestDB 6.0, a high-performance SQL database with new features for Apple chipsets; AdonisJS 5.0, a full-stack web framework with enhanced authentication and asset management capabilities; Observable Plot 1.0, a JavaScript library for data visualization; and Wavesurfer.js 5.0, which offers custom audio visualization with a vertical waveform display. Additional releases such as Scribbletune 4.0 for generating musical patterns, listmonk 1.0 for efficient newsletter management, Clinic.js 9.0 for diagnosing Node.js performance issues, and Laravel Activity Log 4.0 for app activity logging illustrate the breadth of open-source development. The quirky DOOM Captcha project adds a playful element to the mix, requiring users to engage in a game to verify humanity. The community celebration extends to a new GitHub feature allowing emoji reactions to releases, emphasizing the collaborative and dynamic spirit of the tech community.

GitHub recently concluded a community discussion on revising its policies concerning security research, malware, and exploits, aiming to encourage dual-use security research and collaboration on the platform. Feedback from the security research community, project maintainers, and developers was instrumental in refining the policies, leading to explicit permission for dual-use security technologies and a clarification of terms that previously seemed hostile. The updated policy outlines GitHub's stance on not supporting unlawful attacks and includes an appeals process for users to contest decisions affecting their content or account access. Additionally, GitHub recommends using a SECURITY.md file to facilitate direct conflict resolution between community members and project maintainers, thereby reducing the need for formal abuse reports. The company's Chief Security Officer, Mike Hanley, highlighted the iterative process's value in enhancing policy clarity and expressed gratitude for the community's input, underscoring a commitment to continuous improvement in collaboration with users.

GitHub Enterprise Server 3.1 is now available, providing enhanced support for managing large repositories and streamlining development and deployment processes for developers. This release introduces features such as GitHub Actions workflow visualizations for easier tracking of complex workflows, automerge functionality for automatically merging pull requests when ready, and optimizations for repository performance. Additionally, users of GitHub Advanced Security can now take advantage of secret scanning and expanded code scanning capabilities for various libraries and frameworks. For more detailed information on these updates, users are encouraged to consult the release notes or start a free trial to explore the platform's capabilities.

In May, GitHub experienced two significant incidents impacting the availability of its services, including API requests, GitHub Pages, GitHub Actions, and the GitHub Packages Container registry service. The first incident on May 8 was caused by failures in a MySQL database, leading to timeouts in the Container registry service, which was resolved by failing over to a database replica and implementing measures to reduce future impacts. The second incident on May 16 was due to a foreign key overflow exceeding max INT32, affecting GitHub Actions and Pages, which was mitigated through a schema migration to INT64 and removal of invalid token records. To prevent such issues in the future, GitHub is enhancing its internal documentation, alerting systems, and tooling, while also auditing INT32 columns and improving automation to avoid database inconsistencies and unauthorized access.

GitHub has launched an updated Social Impact website to highlight its efforts in leveraging technology and open-source communities to address global challenges such as climate change, racial injustice, and income inequality. The revamped site introduces new features, including a homepage detailing GitHub's mission and program milestones, along with two key initiatives: Tech for Social Good, which uses GitHub's platform to support the social sector in achieving Sustainable Development Goals through research, community building, and sponsorship, and Hubber Giving, aimed at employee engagement. Additionally, a new Insights page provides articles, talks, and research to support the social sector, featuring contributions from partners like UNICEF. The updates reflect GitHub's ongoing commitment to making a positive impact through collaboration between its technology, employees, and leadership.