How Candidly Built State-Aware Agent Harnesses with LangSmith

Candidly's innovative approach to conversational AI involves utilizing a turn-level analysis to enhance the effectiveness of their financial planning assistant, Cait. This method departs from traditional post-conversation evaluations by focusing on real-time state inference and controllable response features to predict and influence conversation outcomes. By employing an Input-Output Hidden Markov Model, Candidly identifies four distinct engagement states—Engaged, Detailed, Guided, and Disengaging—and adjusts Cait's responses to maximize conversation resolution and minimize abandonment. These strategies are validated through a robust pipeline involving data collection, model training, and randomized experiments, all recorded in LangSmith for comprehensive monitoring and evaluation. This framework not only improves Cait's ability to assist users in making significant financial decisions but also demonstrates a scalable model for other multi-turn AI systems seeking to optimize user interactions in real-time.

No tracked trend matches for this post yet.