How to Structure a Voice AI Knowledge Base

To structure a Voice AI knowledge base effectively and prevent hallucinations from voice agents, the guide recommends a four-layer approach involving curation, chunking, metadata scoping, and refusal-by-default retrieval. This involves organizing content into 512-token recursive Markdown chunks tagged with essential metadata such as product, region, and audience, and retrieved at a similarity threshold of 0.65 or higher. The architecture aims to ensure that every agent response is grounded in verified content, blocks the AI from inventing answers, and maintains low latency for natural phone conversations. Key practices include auditing source content to eliminate outdated or contradictory information, using Markdown for semantic structure, tagging chunks with specific metadata, and implementing a refusal instruction to prevent the AI from guessing answers. The knowledge base should be kept current through automated updates and tested thoroughly to ensure retrieval accuracy before going live. The structure also supports specialized use cases, such as healthcare and regulated industries, by using conversation flows with node-level knowledge bases to handle distinct workflows efficiently.