CobbleDB: Rebuilding AI search storage for lower latency and cost

An AI-native search company rebuilt its storage stack to better support the distinct demands of processing web pages into passages and embeddings, continuously updating document state, and serving low-latency batch reads for search queries. Replacing a DynamoDB-based architecture that directly coupled processing writes with serving, the new design uses Pillar for durable, versioned document state and export decisions, Lorry to create partition-aligned update batches, and CobbleDB, a custom distributed key-value store built on RocksDB, for tunable hot-storage reads. CobbleDB uses replicated partitions, local NVMe storage, memory caching, parallel routing, batched MultiGet operations, and optional replica hedging to optimize retrieval of prepared page records, while asynchronous ingestion prevents large rebuilds or updates from competing with live traffic. Production measurements reported median batch-read latency falling from 31.4 milliseconds to 5.6 milliseconds, with similar roughly fivefold improvements at higher percentiles, while internal cost estimates indicated savings of at least 20 percent compared with DynamoDB. The project, including a roughly 40,000-line Rust database, was developed by two engineers with assistance from hundreds of internal coding agents that helped review changes, identify risks, prepare fixes, track rollouts, and maintain project status, while humans retained responsibility for architecture and production decisions.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.