In the latest episode of the podcast High Agency, Raza Habib, CEO of Humanloop, converses with Jeff Huber, founder of Chroma, about the significant role vector databases play in AI engineering. They discuss how vector databases enhance AI applications by providing a memory layer that augments large language models (LLMs) with specific, private data, thereby mitigating issues like hallucinations. Huber shares insights from Chroma's development, particularly their focus on improving developer experience by simplifying configurations and addressing common challenges faced when scaling AI applications. The conversation also covers the limitations and evolution of retrieval augmented generation (RAG) terminology, suggesting that retrieval and generation should be considered separate processes. They delve into the practical applications of vector databases in real-world AI systems, emphasizing the importance of retrieval for dynamically interacting with models and providing valuable use cases like automated email processing. The episode underscores the ongoing evolution of AI engineering best practices, encouraging engineers to iteratively build, test, and refine AI systems, maintaining a focus on real-world applicability and user experience.