The UK AI Security Institute, in collaboration with Check Point and Lakera, has unveiled an open source benchmark ‘b3’ to strengthen LLM security for AI agents.
The UK AI Security Institute (AISI) has partnered with Check Point and its subsidiary Lakera to launch the Backbone Breaker Benchmark (b3), an open source framework designed to enhance the security and resilience of large language models (LLMs) that power AI agents.
Built to make LLM security measurable and transparent, b3 focuses on identifying the specific “pressure points” where LLMs fail—such as when prompts, files, or web inputs trigger malicious outputs. Rather than evaluating full agent workflows, the benchmark isolates the individual steps most vulnerable to attack.
At the core of b3 is a new testing method called “Threat Snapshots”, which surfaces hidden weaknesses using crowdsourced adversarial data from Lakera’s “Gandalf: Agent Breaker” initiative. The benchmark combines ten representative agent snapshots with a dataset of 19,433 adversarial attacks, covering threats such as system prompt exfiltration, phishing link insertion, malicious code injection, denial-of-service, and unauthorised tool calls.
According to Lakera, the benchmark “makes LLM security measurable, reproducible, and comparable across models and application categories.” Results have revealed that models using step-by-step reasoning tend to be more secure, and that open-weight models are narrowing the gap with closed systems faster than expected.
Lakera’s co-founder and chief scientist Mateo Rojas-Carulla stated, “By making this benchmark open to the world, we hope to equip developers and model providers with a realistic way to measure, and improve, their security posture.”
By open-sourcing b3, AISI and its partners aim to democratise LLM security, fostering community collaboration and establishing a shared standard for safeguarding AI agent frameworks.
