RAG is a method that allows Large Language Models (LLMs) to access external knowledge sources as context, improving accuracy and reliability. By introducing grounded data, RAG addresses the limitations of LLMs, such as hallucinations, which occur when models generate inaccuracies or factually incorrect statements. RAG is increasingly popular in various industries, including customer support, content generation, knowledge base queries, and real-time information retrieval. CockroachDB is a highly scalable and resilient distributed SQL database that provides an ideal backend for building efficient and robust RAG applications. The unified storage of source data, metadata, and vector data in CockroachDB simplifies data management, lowers costs, and improves performance for RAG applications. Additionally, CockroachDB's scalability, reliability, and simplicity of operations make it well-suited for powering RAG systems that handle intensive workloads and large data volumes. The database also provides advanced access control and security features, such as role-based access controls and row-level security, to ensure that users access only the data they're authorized to view. By leveraging CockroachDB's vector capabilities, developers can scale their RAG applications while maintaining consistent performance under increasing load. A demo of a knowledge base chatbot using RAG on CockroachDB is available for interactive exploration.