How to Run the Backbone Breaker Benchmark (B3)

The Backbone Breaker Benchmark (b3) is developed by Lakera in partnership with the UK AI Security Institute to assess the security of backbone large language models (LLMs) against adversarial attacks. Unlike evaluations focused on capability or safety, b3 isolates the core model powering AI agents to test its resilience against manipulation. Using data from nearly 200,000 human red-team attacks collected via the Gandalf: Agent Breaker challenge, the benchmark creates structured "threat snapshots" that simulate real-world attack scenarios. These snapshots evaluate model responses at various defense levels to determine vulnerability. The benchmark employs different scoring methods according to attack objectives, offering insights into how effectively models resist manipulation. Researchers can run the benchmark using tools from the Inspect Evals GitHub repository, and the evolving nature of b3 aims to keep pace with advancements in AI and emerging attack techniques, contributing to a shared empirical approach for measuring AI security.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.