Microagents: building better AI agents with microservices

Sherpa, an AI agent developed as a proof of concept by an online retailer's innovation group, was intended to streamline customer service by autonomously handling inquiries about orders, product details, and more. However, as its capabilities expanded, Sherpa became complex and unwieldy, leading to issues with maintainability, quality, and error handling. Initially a monolithic application, Sherpa faced scaling challenges as it attempted to manage numerous API endpoints and maintain conversation context across multi-step workflows. To address these issues, the team transitioned to a microservices architecture, breaking Sherpa into specialized "microagents" with clear responsibilities, such as order management, returns and refunds, and policy support. This approach allowed for independent scaling and deployment of services while maintaining coherence through service discovery and communication management. Despite solving many problems, the microservices model introduced new challenges, including coordination overhead and testing complexity. The team is now developing Microagent, a framework designed to support distributed deployments and facilitate AI-driven business capabilities, offering a promising path forward for building sophisticated AI agents with microservices benefits.