Visualizing GitHub’s global community

The development of GitHub's new homepage involved creating an engaging, real-time 3D globe representation that relies heavily on data management and processing. The project aimed to showcase interesting, live data without compromising GitHub's performance or user privacy. To achieve this, a dedicated data service was designed to handle vast amounts of data efficiently, using tools like Presto for querying large datasets and Apache Kafka for maintaining data freshness. A model was employed to assess repository "health" to ensure the featured data was relevant and engaging. For geocoding user locations, Mapbox's API was used to map user-submitted text locations to geographical coordinates while respecting user privacy. Apache Airflow was utilized to schedule and manage data tasks, ensuring the homepage data remained up-to-date. The successful implementation of the homepage was made possible through collaboration across various teams and the contributions of the GitHub community.