Free-Text Search Isn't One Problem: How We Made Logs and AI Observability Searchable at Scale

Groundcover faced the challenge of implementing fast and scalable free-text search across vast datasets of logs and AI observability data, driven by the need for interactive searches that span hundreds of billions of log lines. The complexity arose from allowing customers to retain all relevant data, necessitating efficient query handling over large datasets. Groundcover experimented with various indexing strategies using ClickHouse, including native skip indexes, in-house inverted indexes for time-bound attributes, and eventually adopted ClickHouse's full-text inverted index for broader free-text search. They also employed a strategy of creating "favorite attributes" to manage JSON column path explosions, ensuring that only frequently queried fields were indexed. The solution involved optimizing query paths and indexing strategies to make searches efficient and cost-effective, ultimately enabling engineers to start investigations from familiar phrases or prompts.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.