100x Faster: GraphQL Hive migration from Elasticsearch to ClickHouse

The GraphQL Hive team, behind The Guild, was using Elasticsearch as their database solution but faced scalability issues and a steep learning curve due to its query language. They were looking for an alternative that could handle high cardinality, support analytics and aggregation, and be easy to learn and maintain. After researching and testing several databases, they decided to switch to ClickHouse, which offered superior performance, a built-in TTL, and a type system. The migration was done gradually with zero downtime, resulting in a 100x improvement on average compared to their previous data pipeline, allowing them to scale from millions to billions of rows monthly without issues.