Company
Date Published
Author
Michelle Gienow
Word count
1883
Language
English
Hacker News points
2

Summary

In "The Engineer's Survival Guide," Derrick Miller recounts his experience during Superstorm Sandy while working at Fog Creek Software in Lower Manhattan. The company had just decommissioned its disaster recovery site to transition entirely to the cloud, leaving them vulnerable as the hurricane approached. Despite having backup generators, the flooding rendered them useless, but the data center survived due to its independent generator. A dramatic effort ensued, with staff manually transporting diesel fuel up 18 flights of stairs to keep the generator running. This incident highlighted the critical need for robust disaster recovery plans, as the real risk was not immediate revenue loss but long-term damage to reputation and customer trust. The experience underscored the importance of integrating availability and resilience into systems from the start, as unforeseen disasters can strike at any time.