LLM monitoring vs LLM observability: What's the difference?

Large Language Model (LLM) systems require a distinct approach to monitoring and observability due to their unique ability to produce incorrect or irrelevant outputs while appearing healthy. Traditional monitoring tracks system metrics like latency and error rates to confirm operational health, but it doesn't explain why issues occur, whereas observability provides insight into the system's behavior by tracing specific requests through the pipeline. Evaluations play a key role in both processes by scoring output quality and offering data to investigate failures. Braintrust unifies these elements, offering a platform that integrates monitoring, observability, and evaluations into a cohesive workflow, allowing teams to detect, investigate, and address issues efficiently. This integration is crucial, as separating these functions across different tools often leads to inefficiencies and complicates issue resolution.