Building a RAG Pipeline for Semantic Code Search: A Developer Diary and Field Notes - The JetBrains Blog

JetBrains describes building Context, a production RAG system for semantic code search that helps coding agents find relevant repository code by meaning rather than relying on keywords or grep. The pipeline parses source files into structure-aware chunks using language-specific syntax rules, preserving relationships such as documentation, annotations, declarations, and closing syntax while avoiding both overly broad file-level results and contextless line-level fragments. It supports detailed parsing for nine major languages and uses simpler line-based splitting for others, with chunk quality evaluated by an LLM judge and end-to-end retrieval tests. Chunks, augmented with abbreviated file-path metadata, are converted into embeddings for similarity search; to reduce the large storage and comparison costs of millions of vectors, the system retains all embedding dimensions but binarizes them to one bit, using Hamming distance for efficient ranking. Although binary vectors reduce recall slightly and cannot reliably support absolute relevance thresholds, they are considered suitable for agent workflows that need a relevant group of results rather than perfectly ordered rankings. The platform uses separate deployment optimizations for high-throughput indexing and low-latency querying, runs open-weight embedding models on JetBrains-operated infrastructure, stores only file coordinates and metadata rather than source-code copies, and does not use customer code for training.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.