Company
Date Published
Author
Emily Mosher
Word count
754
Language
-
Hacker News points
None

Summary

AmericanAncestors.org, operated by the New England Historical Genealogical Society, enhanced its user search experience by transitioning from SQL to Elasticsearch, significantly improving search flexibility and speed for its 250,000 members who access over 1.4 billion names and historical documents. Initially, the system ran on outdated Elasticsearch 1.3 across five physical servers set up by contractors, which became inefficient and unstable. Under the leadership of Don LeClair, the organization purchased a Platinum subscription to the Elasticsearch Service, optimizing their clusters, reducing system administration time, and implementing security measures. This new approach allowed for a more dynamic search system that accommodated suggestions, although it initially increased server workload, especially after vulnerability patches. Collaboration with the Elastic support team led to further optimizations, including identifying problematic queries and improving data management, resulting in a 60% reduction in disk usage and a 50% decrease in CPU usage, making searches 2-3 times faster. These improvements enabled AmericanAncestors.org to handle a 400% traffic surge during a promotional event with ease, while also reducing costs by 25% compared to their previous setup.