How Clay uses LangSmith to debug, evaluate, and monitor 300 million agents runs per month

Clay is a platform designed to enhance the efficiency of go-to-market teams by enabling them to build, enrich, and activate lists of companies and people through AI-powered tools. It serves a diverse clientele, from startups to large enterprises, facilitating tasks such as sourcing target accounts, qualifying leads, and drafting personalized outreach. With approximately 300 million AI agent runs monthly, Clay's operations involve complex multi-step processes like web scraping and data synthesis. To manage scalability challenges like quality assurance, cost control, and the rapid pace of model releases, Clay relies on LangSmith for observability and evaluation. This integration allows for real-time monitoring, debugging, and structured evaluations, helping Clay maintain cost reconciliation and improve its pricing strategy. LangSmith's capabilities provide critical insights into usage patterns, error rates, and model performance, enhancing Clay's ability to adapt quickly to new AI model introductions and maintain operational efficiency.