Picture this scenario: An Anthropic Skill scanner runs a full analysis of a Skill pulled from ClawHub or skills.sh. Its markdown instructions are clean, and no prompt injection is detected. No shell commands are hiding in the SKILL.md. Green across the board.
The scanner never looked at the .test.ts file sitting one directory over. It didn’t need to. Test files aren’t part of the agent execution surface, so no publicly documented scanner inspects them (as of publication of this post). The file runs anyway. Not through the agent but through the test runner, with full access to the filesystem, environment variables, and SSH keys.
Gecko Security researcher Jeevan Jutla detailed this attack flow, demonstrating that when a developer runs npx Skills add, the installer copies the entire skill directory into the repo. If a malicious Skill bundles a *.test.ts file, the Jest and Vitest testing frameworks discover it through recursive glob patterns, treat it as a first-class test, and execute it during npm test or when the IDE auto-runs tests on save. The default configuration in open-source JavaScript test framework Mocha follows a similar recursive discovery pattern. The payload fires in beforeAll, before any assertions run. Nothing in the test output flags anything unusual. In CI, process.env holds deployment tokens, cloud credentials, and every secret the pipeline can reach.
The attack class is not new; malicious npm postinstall scripts and pytest plugins have exploited trust-on-install for years. What makes the Skill vector worse is that installed Skills land in a directory designed to be committed and shared across the team, propagate to every teammate who clones, and sit outside every scanner's detection surface.
The agent is never invoked, and the Anthropic Skill scanner reads the right files for the wrong threat model.
Three audits, one blind spot
Gecko's disclosure didn’t arrive in isolation. It landed on top of two large-scale security audits that had already documented the scope of the problem from the other direction, illustrating what scanners detect rather than what they miss. Both audits did exactly what they're designed to do: They measured the threat on the execution surface scanners already inspect. Gecko measured what sits outside it.
A SkillScan academic study, published on January 15, analyzed 31,132 unique Anthropic Skills collected from two major marketplaces. Their findings: 26.1% of Skills contained at least one vulnerability spanning 14 distinct patterns across four categories. Data exfiltration showed up in 13.3% of Skills. Privilege escalation appeared in 11.8%. Skills bundling executable scripts were 2.12x more likely to contain vulnerabilities than instruction-only Skills.
Three weeks later, Snyk published ToxicSkills, the first comprehensive security audit of the ClawHub and skills.sh marketplaces. Snyk's team scanned 3,984 Skills (as of February 5). The results: 13.4% of all Skills contained at least one critical-level security issue. Seventy-six confirmed malicious payloads were identified through a combination of automated scanning and human-in-the-loop review. Eight of those malicious Skills were still publicly available on ClawHub when the research was published.
Then Cisco shipped its AI Agent Security Scanner for IDEs on April 21, integrating its open-source Skill Scanner directly into VS Code, Cursor, and Windsurf. The scanner brings genuine capability to developers’ workflows. It does not inspect bundled test files, because the detection categories Cisco built target the agent interaction layer, not the developer toolchain layer.
The three major Anthropic Skill scanners share a structural blind spot: None inspects bundled test files as an execution surface, even though Gecko Security proved that those files execute with full local permissions through standard test runners.
Snyk Agent Scan, Cisco's AI Agent Security Scanner, and VirusTotal Code Insight all work. They catch prompt injection, shell commands, and data exfiltration in Skill definitions and agent-referenced scripts. What they do not do is look beyond the agent execution surface to the developer execution surface sitting in the same directory.
How the attack chain works
The mechanics of the attack chain matter because the fix is precise. When a developer runs npx skills add owner/repo-name, the installer clones the Skill repository and copies its contents into .agents/skills/<skill-name>/ inside the project. Claude Code, Cursor, and other agent IDEs get symlinks into their own Skill directories. The only files excluded are .git, metadata.json, and files prefixed with _. Everything else lands on disk.
Jest and Vitest both pass dot: true to their glob engines. That means they discover test files inside dot-prefixed directories like .agents/. Mocha's behavior depends on configuration but follows similar recursive patterns by default. None of them exclude .agents/, .claude/, or .cursor/ from their default discovery paths.
An attacker publishes a Skill with a clean SKILL.md and a tests/reviewer.test.ts file containing a beforeAll block. The block reads process.env, .env files, ~/.ssh/ private keys, and ~/.aws/credentials. It posts everything to an external endpoint. The test cases look real. The exfiltration happens during setup, silently, whether the tests pass or fail.
The vector is not limited to TypeScript. Python repos face the same exposure through conftest.py, which pytest auto-executes during test collection. Add .agents to testpaths exclusion in pyproject.toml to block it.
The .agents/skills/ directory is designed to be committed to the repo so teammates can share Skills. GitHub's default .gitignore templates do not include .agents/. Once the malicious test file enters the repo, every developer who clones and runs tests executes the payload. So does every CI pipeline on every branch and every fork that inherits the test suite.
Scanners are reading the wrong threat surface
CrowdStrike CTO Elia Zaitsev put the structural challenge in operational terms during an exclusive VentureBeat interview at RSAC 2026. "Observing actual kinetic actions is a structured, solvable problem," Zaitsev said. "Intent is not."
That distinction cuts directly at the Anthropic Skill scanner gap. No publicly documented scanner operates outside the assumption that the threat lives in the SKILL.md and in scripts the agent is instructed to run. These tools analyze intent: What does the Skill tell the agent to do? Gecko's finding sits on the kinetic side. The test file executes through the developer's own toolchain. No agent is involved. No prompt is interpreted. The payload is TypeScript, running with full local permissions through a legitimate test runner. The scanner was solving the wrong problem.
CrowdStrike's Zaitsev framed the identity dimension: "AI agents and non-human identities will explode across the enterprise, expanding exponentially and dwarfing human identities," he told VentureBeat. "Each agent will operate as a privileged super-human with OAuth tokens, API keys, and continuous access to previously siloed data sets."
CrowdStrike's Charlotte AI and similar enterprise agents operate with exactly these privileges. When those credentials live in environment variables accessible to any process in the repo, a test-file payload does not need agent privileges. It already has developer privileges, which in most CI configurations means deployment tokens and cloud access.
Mike Riemer, SVP of the network security group and field CISO at Ivanti, quantified the exploitation window in a VentureBeat interview. "Threat actors are reverse engineering patches within 72 hours," Riemer said. "If a customer doesn't patch within 72 hours of release, they're open to exploit."
Most enterprises take weeks. The Anthropic Skill scanner blind spot compounds that window. A developer installs a malicious Skill today. The test file executes immediately. No patch exists because no scanner flagged it.
The Anthropic Skill Audit Grid
VentureBeat has covered the Anthropic Skill supply chain since the ClawHavoc campaign hit ClawHub in January. Every conversation with security leaders lands on the same frustration. Their teams bought a scanner, it reports clean, and they have no framework for asking what it does not check.
VentureBeat has polled dev teams who install Anthropic Skills from ClawHub and skills.sh. The grid below connects the published-audit half (Snyk, SkillScan) with the scanner-bypass half (Gecko). Each row represents a detection surface a security team should verify before approving any Skill scanning tool for Q2 procurement.
| Audit question |
What scanners do today |
The gap |
Recommended action |
|
Inspect SKILL.md and agent-invoked scripts |
Covered by Snyk Agent Scan, Cisco AI Agent Security Scanner, VirusTotal Code Insight |
This is the covered surface. Attackers shift payloads to files outside it. |
Continue running current scanners. They catch real threats at the instruction layer. |
|
Inspect bundled test files (*.test.ts, *.spec.js, conftest.py) |
Not currently inspected as attack surface by any scanner |
Gecko proved test files execute via Jest/Vitest (documented) and Mocha (config-dependent) with full local permissions. No agent invoked. |
Add .agents/ to testPathIgnorePatterns (Jest) or exclude (Vitest). One config line. |
|
Flag Skills that bundle test files or build configs |
Not flagged as higher-risk metadata by any scanner |
Trivial static check. Skills with extra executables are 2.12x more likely to be vulnerable (SkillScan). |
Add CI gate: find .agents/ -name "*.test.*" | grep -q . && exit 1. Block merge on match. |
|
Restrict test-runner globs to project-owned paths |
Rare. Most CI configs use recursive glob. Jest/Vitest pass dot: true by default. |
Default globs traverse .agents/, .claude/, .cursor/ directories. Malicious test files auto-discovered. |
Scope test roots to first-party directories (src/, app/). Deny .agents/, .claude/, .cursor/. |
|
Distinguish script-bundling Skills vs. instruction-only |
Partial coverage via static and semantic analysis |
SkillScan: script-bundling Skills 2.12x more likely to contain vulnerabilities than instruction-only. |
Require structured audit entry: Skill type, execution surfaces, scanner coverage, residual risk. |
|
Publish audit methodology with sample size |
Snyk yes (3,984 Skills). SkillScan yes (31,132 Skills). |
Cisco and emerging scanners have not published equivalent ecosystem-scale audits. |
Ask vendors: methodology, sample size, detection rate. No published audit = no independent baseline. |
|
Pin Skill sources to immutable commits |
Not enforced by any scanner or marketplace |
Skill authors can push clean version for review, add malicious test file after approval. |
Pin to specific commit hash. Review diffs on every update. OWASP Agentic Skills Top 10 recommends this. |
Three CI hardening steps to add now
Riemer made the broader point in VentureBeat interviews that placing security controls at the perimeter invites every threat to that exact boundary. Anthropic Skill scanners placed the boundary at SKILL.md. Attackers put the payload one directory over. The three changes below move the boundary to where the code actually executes.
These changes take minutes. None requires replacing current tools or waiting for scanner vendors to close the gap.
Add .agents/ to the test runner's ignore list. In Jest, add /\.agents/ to testPathIgnorePatterns in jest.config.js. In Vitest, add **/.agents/** to the exclude array in vitest.config.ts. One line in one config file prevents the test runner from discovering files inside installed Skill directories. Do it whether or not the team currently uses Anthropic Skills. The directory may appear in a cloned repo without anyone installing the Skill directly.
Audit every Skill install for non-instruction files before merge. Add a CI check that flags any file in .agents/skills/ matching *.test.*, *.spec.*, __tests__/, *.config.*, or conftest.py. These files have no legitimate reason to exist inside a Skill directory. The check is a shell one-liner: [ -d .agents ] && find .agents/ -name "*.test.*" -o -name "*.spec.*" -o -name "conftest.py" -o -name "*.config.*" -o -type d -name "__tests__" | grep -q . && exit 1. If it matches, block the merge. For any test files that do land in a PR, require a reviewer to skim for shell invocations (exec, spawn, child_process), external network calls, and file operations touching secrets or SSH keys.
Pin Skill sources to specific commits, not latest. The npx skills add command copies whatever the repo contains at the moment of install. A Skill author can push a clean version for scanner review, then add a malicious test file after approval. Pinning to a specific commit hash converts a trust-on-first-use model into a verify-on-every-change model. The OWASP Agentic Skills Top 10 recommends exactly this.
If Skills are already in your repo: Run the find command above against your existing .agents/ directory now. If test files are present, treat them as a potential compromise: Rotate any credentials accessible to CI (deployment tokens, cloud keys, SSH keys), audit CI logs for unexpected outbound network calls during test execution, and review git history to determine when the test files entered the repo and which pipelines have executed them.
Five questions to ask your Anthropic Skill scanner vendor
Security teams are signing contracts for their first dedicated Skill scanning tools. The Gecko bypass means the questions on those sales calls need to change. Do not stop at "Do you detect prompt injection?" Ask:
- Which files and directories do you actually analyze in a Skill repo?
-
Do you treat test files as potential execution surfaces?
-
Can you flag Skills that bundle tests, CI configs, or build scripts as higher-risk? SkillScan showed script-bundling Skills are 2.12x more likely to be vulnerable.
-
Do you provide integration or guidance for restricting test-runner globs in CI? Cisco deserves credit for open-sourcing its Skill Scanner on GitHub, which lets security teams inspect exactly which detection categories the tool implements. That transparency is the baseline every vendor should meet. If your vendor will not publish detection categories or open-source their scanning logic, you cannot verify what they check and what they skip.
-
Have you published an ecosystem-scale audit with methodology and sample size? Snyk published at 3,984 Skills. SkillScan published at 31,132. Riemer described the disclosure pattern: "They chose not to publish a CVE. They just quietly patched it and moved on with life," he said. The Anthropic Skills ecosystem is showing early signs of the same pattern: scanners document what they detect without mapping the surfaces they do not reach. The gap between documented coverage and actual execution surface is where the test-file vector lives.
The audit grid matters because the scanner model is incomplete
The Anthropic Skills ecosystem is repeating the early npm supply chain story, except without the decade of accumulated incidents that forced package registries to build security infrastructure. SkillScan's 31,132-Skill dataset showed a quarter of the ecosystem carrying vulnerabilities. Snyk found 76 confirmed malicious payloads in fewer than 4,000 Skills. Gecko proved the scanner model itself has a structural gap that no vendor has publicly documented closing.
Scanner evaluations consistently test the covered surface. The Anthropic Skill Audit Grid gives security teams the seven audit surfaces to verify before signing. The three CI steps are the fixes to deploy before the next Skill install. Riemer's Ivanti team watches the patch-to-exploit cycle compress in real time across enterprise environments. The test-file vector compresses it further: No scanner flagged the threat, so no patch window exists.
The scanner is not broken. It is incomplete. The threat model stopped at the agent. The test runner did not.
<a href