Federation Over Embeddings: Let AI Agents Query Data Where It Lives

In the pursuit of AI integration, enterprises often default to building extensive AI-specific infrastructures, such as vector databases and embedding pipelines, which can lead to unnecessary complexity and strategic lock-in. Instead, leveraging existing systems through federation—where AI agents access data directly from current sources like CRMs or data warehouses using tools like the Model Context Protocol (MCP)—can offer a more efficient solution. This approach allows for real-time data retrieval and synthesis without duplicating data, reducing the need for parallel data infrastructures. By adopting agentic AI systems with tool-calling capabilities, businesses can quickly deliver value and adapt to evolving needs without being tied down by rigid architectures. Specialized infrastructures like vector stores or custom models should be introduced only when specific use cases demand them, rather than as a foundational strategy. This agile method allows organizations to iterate rapidly and maintain competitiveness while minimizing investment in unproven infrastructure.