Building production AI agents on Elasticsearch: 5 key lessons

Over the past year, Elastic's Field Technology team has developed and deployed AI agents on the Elasticsearch platform to enhance customer support and sales workflows, resulting in over one million messages processed across five AI tools. Their analysis highlights the importance of feedback loops and retrieval relevance over selecting specific large language models, emphasizing that interaction logs serve as a strategic asset for measuring AI performance through a process called Context Observability. This approach revealed that AI tool adoption is concentrated among power users who account for the majority of interactions, and that retrieval augmented generation (RAG) systems must prioritize relevance over volume to prevent quality degradation. Additionally, while high token counts were initially viewed as a cost issue, they were found to correlate with improved session quality, suggesting that they reflect high-value tasks rather than inefficiencies. The insights from this year-long examination underscore the significance of strategic log analysis and user-centric design in building AI systems that deliver meaningful business impact.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.