Company
Date Published
Author
Risk Olson
Word count
775
Language
English
Hacker News points
None

Summary

Nodeload, a GitHub node.js project responsible for preparing git repository contents into zip and tarballs, recently celebrated its first anniversary but faced various technical challenges due to increased demand. To address the issues of high request volume and server load, the project was restructured; the Nodeload server now functions as a simple proxy app, directing requests to file servers that run an archiver app with an HTTP frontend over git archive. This new setup utilizes in-memory tmpfs partitions for cached repositories to reduce disk IO and shifts load to underutilized backup file servers. Despite improvements, initial launch issues such as high IO and memory leaks were encountered, traced back to improper closure of reader streams upon client download abortion, which prevented server space reclamation. The solution involved monitoring the 'close' event on HTTP server requests. The revised Nodeload system is now more stable, simpler, and better tested, with ongoing efforts focused on establishing improved metrics to enhance service reliability.