Building smarter AI agents: architecture, evals, and lessons from the field

At the AI Builders events held in San Francisco and Seattle, developers emphasized the importance of robust infrastructure and engineering practices over mere model capability for successful AI agent deployment in production. Key insights included the necessity of establishing an evaluation harness early on, separating model roles for efficiency, and implementing observability to capture real-world behaviors. The discussions highlighted that multi-model orchestration is becoming a standard architecture for managing agent systems, balancing latency, cost, and capability. Speakers also stressed the significance of continuous real-world evaluation and governance, with the Foundry Control Plane cited as an example for managing AI agents at scale. Additionally, prompt learning was noted as a cost-effective method to enhance agent performance without modifying model architectures. A major takeaway was that developer productivity has increased with AI tools, yet the code churn rate has also risen, indicating a gap between speed and stability in AI-assisted workflows. Overall, the events underscored the need for a comprehensive operational stack to support the development and scaling of reliable AI systems.