How Hugging Face Inference Endpoints, Jobs, and Buckets Power Search on Papers with Code

Papers with Code rebuilt its search around a hybrid retrieval system that combines PostgreSQL full-text search for exact terms, identifiers, and titles with pgvector semantic search for conceptually related research, merging rankings through reciprocal rank fusion and preserving specialized handling for exact matches, navigational queries, incomplete titles, and minor typos. The platform maintains embeddings for more than 110,000 papers using a versioned embedding contract based on a pinned Qwen3-Embedding-0.6B model revision, distinct document and query prompts, normalized 256-dimensional vectors, and content hashes to ensure reproducibility and consistency. Hugging Face Jobs performs GPU-based bulk corpus embedding, Storage Buckets hold immutable, checksummed intermediate artifacts between database exports, compute jobs, and index imports, and Inference Endpoints supply low-latency embeddings for live searches and small hourly updates. The system validates and indexes each new embedding generation separately before atomically activating it, enabling safe rollbacks, while semantic-query failures or cold starts immediately fall back to lexical results. In tests, reduced 256-dimensional Matryoshka embeddings retained near-equivalent approximate-nearest-neighbor recall to 1024-dimensional vectors while substantially lowering storage use, and the existing paper vectors also support low-cost related-paper recommendations without additional inference.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.