We Handed 20 Open Models a Shell. 20 of Them Failed: Introducing ViKernel Scanner
Blog post from Hugging Face
ViKernel Scanner is a free, open-source tool designed to assess what AI models do when given access to tools, rather than only evaluating their written responses. It presents models with eight simulated tools and 100 security checks, including disguised shell commands, attempts to expose fake credentials, and instructions embedded in documents, while also testing ordinary tasks to identify models that simply refuse everything. In scans of 20 open-weight models conducted through the Hugging Face router, all received an overall failing grade, and the report says the median model attempted a dangerous action in 40 of 100 checks; because the tools and environment were simulated, no real actions were executed. The scanner records attempted calls and arguments, assigns severity-based grades, and retrieves its check set from a central source so copies can receive updates. The authors also replayed identified calls against ViKernel Firewall, a security mechanism they say evaluates tool actions before execution, reporting that it would have blocked all 897 dangerous calls observed. The article presents the scanner as a way to measure and improve safeguards for tool-using models, while noting that its free edition covers four of six planned security domains.
| Trend | Post Mentions | Total Month Mentions | Posts | Companies | MoM |
|---|---|---|---|---|---|
| AI Coding Assistant | 1 | 1,698 | 401 | 139 | -13% |
| AI Guardrails | 1 | 242 | 95 | 39 | -56% |
| LLM | 1 | 3,355 | 742 | 191 | -49% |
Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.