How Candidly Built State-Aware Agent Harnesses with LangSmith

Candidly's innovative approach to conversational AI involves utilizing a turn-level analysis to enhance the effectiveness of their financial planning assistant, Cait. This method departs from traditional post-conversation evaluations by focusing on real-time state inference and controllable response features to predict and influence conversation outcomes. By employing an Input-Output Hidden Markov Model, Candidly identifies four distinct engagement states—Engaged, Detailed, Guided, and Disengaging—and adjusts Cait's responses to maximize conversation resolution and minimize abandonment. These strategies are validated through a robust pipeline involving data collection, model training, and randomized experiments, all recorded in LangSmith for comprehensive monitoring and evaluation. This framework not only improves Cait's ability to assist users in making significant financial decisions but also demonstrates a scalable model for other multi-turn AI systems seeking to optimize user interactions in real-time.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.