How ServiceNow uses LangSmith to get visibility into its customer success agents

ServiceNow is enhancing its sales and customer success operations by developing an intelligent multi-agent system using LangChain's LangSmith and LangGraph tools, which address the challenges of agent orchestration and observability. This system orchestrates the entire customer journey, from lead identification to post-sales adoption and expansion, by coordinating complex workflows that were previously fragmented. ServiceNow's approach involves using a supervisor agent for orchestration and specialized subagents for specific tasks, activated by customer signals and lifecycle stages. LangGraph provides the necessary tools for sophisticated multi-agent coordination, enabling a modular and efficient development process, while LangSmith offers detailed tracing and evaluation capabilities to improve agent performance. The evaluation framework includes custom metrics tailored to each agent's tasks, leveraging LLM-as-a-judge evaluators and ensuring regression prevention. Currently in testing, ServiceNow plans to integrate real user data and utilize new multi-turn evaluation features for more comprehensive assessments of agent performance across the entire customer interaction.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.