Elasticsearch at Intercom

Intercom uses Elasticsearch as its primary data storage and retrieval system, initially using MongoDB before switching due to scalability issues. The transition brought several challenges, including handling free-form custom data, ingesting data into Elasticsearch, and addressing mapping growth and consistency concerns. To mitigate these issues, Intercom implemented a solution involving a worker that checks data counts, reingests data from MongoDB to Elasticsearch, and uses a table in MySQL to map custom data to existing fields in Elasticsearch. The company has since addressed additional challenges related to cluster stability and scalability, including hotspotting, shard size, and dual writing for customer apps. Looking ahead, Intercom plans to further improve and scale Elasticsearch by sharding customers across different clusters, removing routing, and adopting a service-oriented architecture.

No tracked trend matches for this post yet.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.