How we made continuous trace intelligence possible at scale

Braintrust's Topics is an innovative solution designed to intelligently analyze production logs at scale, addressing the challenge of extracting valuable insights from vast amounts of complex agent traces. It introduces an architecture that uses a summarization-first approach, where a language model generates concise summaries of traces that are then embedded, clustered, and classified, allowing for cost-effective and continuous processing. The architecture leverages concepts from Anthropic's Clio paper and avoids embedding raw data, focusing instead on embedding these summaries to maintain tractability and reduce costs. Topics supports various dimensions like task, sentiment, and custom categories without requiring multiple pipelines, thereby streamlining the analysis process. Automation ensures that the system runs continuously, updating topic maps regularly, while maintaining stable cluster identities, allowing for consistent trend analysis and queryable outputs. The system's design emphasizes developer experience by providing a seamless integration that requires minimal intervention while enabling custom adjustments, making it a robust and scalable solution for understanding logs in production environments.