Home / Companies / Promptfoo / Blog / March 2026

March 2026 Summaries

4 posts from Promptfoo

Filter
Month: Year:
Post Summaries Back to Blog
OpenClaw is a multifaceted assistant tool that integrates web browsing, local file access, and outbound actions, but its capabilities increase security risks. A controlled lab experiment demonstrated how a local OpenClaw deployment with these capabilities could be exploited by a malicious webpage, leading to unauthorized actions such as capability enumeration, local document access, artifact creation, and sending false messages. The experiment highlighted the importance of separating browsing, file access, and outbound actions into distinct trust boundaries to prevent potential security breaches. The study used Promptfoo's OpenClaw provider to test and document an exploit chain, emphasizing that once an agent with browsing capabilities interacts with untrusted content, the security focus should shift from the model's alignment to the action boundary. The findings underscore the need for explicit approval of outbound actions and careful monitoring of both network actions and local artifact creation to mitigate risks associated with the deployment of browser-capable local agents.
Mar 12, 2026 1,356 words in the original blog post.
The incident involving McKinsey's AI system, Lilli, was characterized by an application-security breach rather than a model jailbreak, as detailed by CodeWall on March 9, 2026. This breach was facilitated by exposed API documentation, unauthenticated endpoints, SQL injection vulnerabilities, and cross-user access, all of which were quickly addressed by McKinsey without evidence of unauthorized access to client data. The issue highlighted the significance of software security, data security, and configuration governance in AI systems, as the backend access had the potential to alter the AI's responses by changing prompts, routing rules, and user history. This incident underscores the importance of auditing control points like public routes, SQL paths, and access-control metadata to prevent such security breaches, showing that AI incidents often stem from traditional software vulnerabilities rather than model-specific failures.
Mar 10, 2026 740 words in the original blog post.
Promptfoo, an open-source platform specializing in AI security and evaluation, has announced its acquisition by OpenAI, with the aim of enhancing AI application testing and deployment. Founded in 2024 to address adversarial testing challenges, Promptfoo has rapidly grown to serve over 350,000 developers, including active participation from 130,000 monthly users and teams from more than 25% of Fortune 500 companies. The acquisition will enable Promptfoo to integrate its technology with OpenAI's model and infrastructure layers, enhancing its tools for early vulnerability detection and secure AI deployment. The open-source suite, including red teaming and static scanning tools, will continue to be maintained, supporting a wide range of AI models and providers. Promptfoo's team, led by co-founders Ian Webster and Michael D'Angelo, remains committed to maintaining service continuity and leveraging additional resources and research from OpenAI. The acquisition underscores the growing importance of AI security and evaluation as foundational elements in deploying complex AI systems. Insight Partners and Andreessen Horowitz, early investors in Promptfoo, express enthusiasm for the platform's continued impact in the AI security landscape.
Mar 09, 2026 507 words in the original blog post.
ModelAudit is an open-source static security scanner for machine learning (ML) model files, designed to identify unsafe loading behaviors, known CVEs, and suspicious artifacts across 42+ formats without executing the models or importing ML frameworks. Developed by Promptfoo, it addresses the issue of model files executing code at load time, often overlooked by teams downloading models from public registries. ModelAudit provides comprehensive security checks, including CVE detection, SARIF output for CI/CD integration, and supports diverse formats like PyTorch, TensorFlow, and ONNX. Unlike existing blocklist-based scanners such as picklescan and Fickling, ModelAudit employs an allowlist-first approach to minimize false positives and bypasses, offering a lightweight, framework-independent tool for platform and application security teams. The scanner's development involved extensive testing and refinement, leading to its release as a standalone, MIT-licensed project that enhances security in the ML model ecosystem.
Mar 03, 2026 2,599 words in the original blog post.