How We Cut Vector Search Latency By 70x

Mem0 found that its pgvector HNSW indexes on a 1.15 TB multi-tenant PostgreSQL memory table received zero production scans because PostgreSQL’s planner judged sequential scans cheaper when queries combined selective organization, project, and entity filters with vector similarity ranking. As a result, vector searches took 8 to 14 seconds under load while unused indexes consumed roughly 325 GB of storage and imposed substantial write overhead. Removing the indexes improved average inserts from about 800 ms to 8.12 ms and updates from 500 ms to 13.7 ms, but did not solve the need for scalable vector search. The company moved embeddings and similarity search to Turbopuffer while retaining relational records, transactions, and consistency management in Postgres, using a two-stage process in which Turbopuffer returns matching IDs and Postgres retrieves full records. End-to-end retrieval then averaged 110 to 120 ms, although the design introduced an additional system, synchronization risks, and another network hop. The experience suggests that pgvector remains a practical option for smaller deployments with simple queries, while larger systems with complex multi-tenant filtering may benefit from separating vector and relational workloads after confirming actual index utilization.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.