Rethinking RAG without Embeddings

In exploring alternatives to vector embeddings for Retrieval-Augmented Generation (RAG) pipelines, this text discusses a simpler approach using BM25, a keyword-based search algorithm integrated with Elasticsearch, to retrieve documents based on term matching rather than semantic understanding. The approach is particularly effective when dealing with structured content and precise queries, as it avoids the latency and complexity associated with embedding models. The pipeline involves processing documents with Unstructured, indexing them into Elasticsearch Serverless, and using BM25 for retrieval, which provides clear and debuggable results without the need for vectors. However, BM25's limitation lies in its inability to handle abstract or differently phrased queries, which can lead to incomplete or inferred answers from the language model that generates responses based on retrieved content. The text suggests that while BM25 is suitable for sharp queries and consistent language, hybrid strategies combining keyword and semantic search could enhance retrieval when dealing with more complex queries.