Company
Date Published
Author
Lakera Team
Word count
1225
Language
-
Hacker News points
None

Summary

Lakera has launched the Prompt Injection Test (PINT) Benchmark, designed to evaluate prompt injection protection solutions by providing an objective measure against a representative sample of prompt injection and jailbreak attacks. PINT assesses solutions on their ability to detect true positives and minimize false negatives across a dataset of 3,007 English inputs, which include public and proprietary attack techniques, false positives, and challenges with large documents. The benchmark aims to address the complexities in evaluating Generative AI (GenAI) performance, especially in the domain of prompt injection, and invites researchers and solution providers to contribute to and improve its dataset and evaluation methods. The benchmark's results, code, and examples are publicly available under the MIT license, though the dataset is restricted to prevent overfitting. Lakera emphasizes collaboration to enhance the robustness and comprehensiveness of this evaluation tool.