Wordsmith is an AI assistant designed for in-house legal teams, utilizing large language models (LLMs) powered by customer knowledge bases to review documents, draft communications, and generate contracts with ease of integration into email and messaging systems. A unique aspect of Wordsmith is its deep domain knowledge from leading law firms, which distinguishes it from other AI legal tools. The platform has experienced rapid growth in LLM-powered features, necessitating improved visibility into performance and interactions. LangSmith plays a vital role in this by offering tracing services that help the engineering team monitor and enhance LLM performance, allowing them to iterate quickly during development. The use of LangSmith's evaluation sets enables Wordsmith to establish baselines for feature requirements, optimize costs, and ensure production readiness. LangSmith also supports rapid debugging and operational monitoring by linking errors to specific traces, thus reducing the time needed to diagnose issues. Additionally, LangSmith's tagging system simplifies experimental analyses, facilitating efficient comparison and improvement of features. Looking ahead, Wordsmith plans to integrate LangSmith more deeply into its product cycle to optimize hyperparameters specific to customer use cases, aiming for an automatically optimized RAG experience tailored to each client's datasets and query patterns.