Content Deep Dive

We Handed 20 Open Models a Shell. 20 of Them Failed: Introducing ViKernel Scanner

Blog post from Hugging Face

Post Details
Company
Date Published
Author
VIDRAFT_LAB
Word Count
1,725
Company Posts That Month
60
Language
-
Hacker News Points
-
Post removed?
No
Summary

ViKernel Scanner is a free, open-source tool designed to assess what AI models do when given access to tools, rather than only evaluating their written responses. It presents models with eight simulated tools and 100 security checks, including disguised shell commands, attempts to expose fake credentials, and instructions embedded in documents, while also testing ordinary tasks to identify models that simply refuse everything. In scans of 20 open-weight models conducted through the Hugging Face router, all received an overall failing grade, and the report says the median model attempted a dangerous action in 40 of 100 checks; because the tools and environment were simulated, no real actions were executed. The scanner records attempted calls and arguments, assigns severity-based grades, and retrieves its check set from a central source so copies can receive updates. The authors also replayed identified calls against ViKernel Firewall, a security mechanism they say evaluates tool actions before execution, reporting that it would have blocked all 897 dangerous calls observed. The article presents the scanner as a way to measure and improve safeguards for tool-using models, while noting that its free edition covers four of six planned security domains.

Trends Found in this Post
Trend Post Mentions Total Month Mentions Posts Companies MoM
AI Coding Assistant 1 1,698 401 139 -13%
AI Guardrails 1 242 95 39 -56%
LLM 1 3,355 742 191 -49%
Use This Data

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.