Company
Date Published
Author
Tara Nagar, Ornella Altunyan
Word count
1259
Language
English
Hacker News points
None

Summary

Experiments in Braintrust are now significantly faster due to the introduction of Brainstore, a high-performance analytics engine designed for querying massive volumes of semi-structured LLM data. To address performance challenges, experiments were initially powered by DuckDB for client-side computation in the browser, but this approach hit limits as experiment volume and complexity grew exponentially. The solution lies in leveraging Brainstore's row-per-input aggregation over trace and span data, computing scores, applying filters, and projecting custom fields in a single request. This enables fast, aggregated views without sacrificing performance or flexibility. Additionally, Brainstore optimizes preview aggregation by truncating user-defined fields based on the `preview_length` parameter, reducing load times for large objects with many small strings. The engine also supports rich custom logic by incorporating aggregate scores and custom columns directly into the query plan, allowing customers to filter across everything from basic fields to computed metrics and custom-defined fields. Overall, these performance upgrades extend beyond experiments to provide a consistently fast experience across Braintrust's log and dataset tables.