Researchers audited 6,309 agent skills: half have shell access

To an agent, a skill looks like a harmless Markdown file, but the agent reads it as an instruction and executes it with the privileges of the session that loaded it. The authors of a new paper on Hugging Face Daily Papers measured how much power a user hands over on installation. They analyzed 6,309 listings from two marketplaces, ClawHub and skillsmp. Half show signs of being able to run commands in a shell, and seven in ten carry at least one elevated capability. Almost none of this is visible at install time.

The method has two phases. First comes static analysis of the entire bundle, not just a single SKILL.md, along with a check of where dependencies come from. Then each skill is run in an instrumented sandbox, and a model separately evaluates the resulting traces. One important detail of the sampling: the listings chosen for execution were selected without relying on any prior signal of threat or risk. The picture therefore describes the market as a whole, not just the suspicious entries that usually end up in reports.

What the measurement showed

The authors stress that installing a skill grants no new privileges, but rather the right to direct the agent. They measure the size of that right. The shares of capabilities across the whole corpus are as follows:

• shell execution: 50% of listings;
• filesystem reads: 35%;
• filesystem writes: 31%;
• network egress: 31%;
• credential access: 13%;
• at least one elevated capability: about 70%.

The work is not just a hunt for malicious skills. Most of these capabilities may be entirely legitimate: a deployment skill cannot do without a shell. But the user accepts this entire attack surface with one click, without knowing its boundaries.

Declarations versus reality

Formally, a permission mechanism exists. A skill's author can list, in its own frontmatter, the tools that are pre-approved. However, this option removes confirmation prompts and restricts nothing. The skill keeps access to everything it can reach.

By the numbers, the mechanism is barely used. The frontmatter could be parsed for 6,061 listings, and only 770 of them declare anything. The vocabulary cannot express access to credentials or environment variables. Nobody checks whether the declarations match the contents of the bundle.

Even when an author does declare, they often get it wrong or stay silent: 59% of listings that declare anything understate their actual capabilities. The marketplaces differ noticeably. ClawHub declares roughly twice as often as skillsmp, but understates in 88% of its declarations versus 31% for its competitor. In other words, filling in the fields more willingly does not mean being more honest.

How to build vetting in practice

The authors' main takeaway for marketplaces is that triage should not rest on what is declared, only on what the bundle is able to do. A maliciousness detector answers the question "is the author malicious?" A capability profile answers a different one: "what happens if the skill is compromised, makes a mistake, or is fooled by prompt injection?" The two checks complement each other.

Teams that connect third-party skills to agents can do several things right now:

• scan the whole bundle, including scripts and dependencies, rather than reading only the description;
• put skills with shell, filesystem writes, network egress, and credential access on a separate list and approve them manually;
• do not treat pre-approved tools in frontmatter as a security measure: it is a way to remove confirmations;
• run unfamiliar skills in a sandbox without secrets and with restricted outbound traffic;
• do not give an agent session more privileges than the most sensitive of the loaded skills needs.

For marketplaces, the conclusion is even more direct: they need an automatically computed capability profile shown to the user before installation, and a check that declarations match the bundle. Until that exists, the 70% of elevated capabilities remain an invisible term of the deal.