CommandBar is a user assistance platform that uses artificial intelligence to capture and predict user intent, providing personalized in-product help through its Copilot widget. The company's team had to find common threads for user assistance across its diverse customer base, leveraging Large Language Models (LLMs) and integrating with various content providers to achieve this goal. To gain visibility into the performance of its Copilot, CommandBar adopted LangSmith, a tool that provided trace visibility, debugging capabilities, increased testing coverage, and monitoring features. By using LangSmith, CommandBar was able to streamline its development process, identify issues proactively, and deliver better experiences for its customers. The company's Copilot has been live since November 2023, resulting in a significant decrease in support tickets and becoming a key competitive advantage for the platform.