skill-scanner

Skill scanner

Contract

Inputs

• skill_path (required): path to the skill directory containing SKILL.md and any associated files.
• risk_threshold (optional): minimum severity to report: one of low, medium, high, critical. Defaults to low.

Procedure

1. Validate that skill_path exists and contains a SKILL.md file. If not, report blocked with the missing-path evidence and stop. Done when: the path exists and SKILL.md is present, or a blocked report is returned.
2. Read and parse the SKILL.md frontmatter. Extract name, description, and any invocation flags. If the file is malformed, note the parse failure and continue with remaining files. Done when: the frontmatter is parsed or the failure is recorded.
3. Inventory every file under the skill directory. Record path, extension, and size. Done when: the inventory is complete.
4. For each file, run the following read-only analysis passes: a. Command execution: scan for shell exec patterns, subprocess spawning, os.system, eval, exec, backtick execution, pipe-to-shell, and child_process calls. b. Network access: scan for HTTP client usage, fetch calls, socket connections, DNS resolution, and outbound URL references outside documentation. c. File system scope: scan for write operations (open(..., 'w'), fs.writeFile, writeFileSync, mkdir, rm, rename) and path traversal (.., absolute paths outside the skill directory). d. Credential and secret access: scan for environment variable reads of keys containing TOKEN, SECRET, KEY, PASSWORD, CREDENTIAL, AUTH, and for file reads of .env, credential stores, or key files. e. Prompt injection vectors: scan for instructions that override system prompts, attempt role confusion, inject <system> tags, or contradict safety policies. Distinguish patterns that perform injection (malicious) from patterns that discuss or detect injection (legitimate in security skills). f. Behavioral analysis: read the full SKILL.md instructions and evaluate description-versus-instructions alignment (a skill described as a formatter that reads ~/.ssh is misaligned), config and memory poisoning (instructions to modify CLAUDE.md, MEMORY.md, settings.json, .mcp.json, or hook configurations; instructions to add itself to allowlists or auto-approve permissions; scripts that append to global config files), scope creep (instructions exceeding the stated purpose, unnecessary data gathering), and information gathering (reading environment variables beyond what is needed, accessing git history or credentials unnecessarily). g. Structural attacks: check for symlinks resolving outside the skill directory (can disguise reads of ~/.ssh/id_rsa or ~/.aws/credentials as example files), frontmatter PostToolUse/PreToolUse hooks that execute shell commands automatically, `!`command syntax that runs at skill load time, test files (conftest.py, test_*.py, *.test.js) that test runners auto-discover and execute, npm postinstall scripts in bundled package.json, and PNG image metadata with text in tEXt/iTXt chunks that multimodal LLMs can read as hidden instructions. h. Script analysis: if the skill has a scripts/ directory, read each script fully. Check for data exfiltration (sending data to external URLs), reverse shells (socket connections with redirected I/O), credential theft (reading SSH keys, .env files, tokens from environment), dangerous execution (eval/exec with dynamic input, shell=True with interpolation), and config modification (writing to agent settings, shell configs, git hooks). Verify each script's behavior matches the SKILL.md description. i. Dependency risk: scan for undeclared external package imports, pinned versions with known vulnerability patterns, and network dependencies that may be unavailable. Review URLs: trusted domains (GitHub, PyPI, official docs) are normal; untrusted domains (unknown domains, personal sites, URL shorteners) are flagged; remote instruction loading (URLs that fetch content to be executed as instructions) is high risk; dependency downloads that execute binaries or code at runtime are flagged. j. Permission analysis: evaluate least privilege: are all declared tools actually used in the skill instructions? Rate the permission profile: read-only tools (Read, Grep, Glob) are low risk; adding Bash is medium risk requiring justification; near-full access (Read, Grep, Glob, Bash, Write, Edit, WebFetch, Task) is high risk. k. Self-containment: verify the skill does not reference other skills or modules as required dependencies, does not require AGENTS.md or CLAUDE.md, and has no hidden file dependencies outside its directory. Done when: every file has been analyzed or its failure recorded.
5. For each finding, record: category, severity (low/medium/high/critical), file path, line range, matched pattern, and the surrounding context (two lines before and after). Done when: every finding has a complete record.
6. Filter false positives: a. Patterns inside fenced code blocks or examples that are illustrative, not executable instructions. b. Read-only operations flagged as mutations (e.g., open(..., 'r') flagged as file write). c. Dangerous patterns guarded by explicit conditionals that prevent execution in the skill's declared authority. d. Credential references that are documentation about environment variables the skill does not access. e. Security skills that reference injection or attack patterns in their references are documenting threats, not attacking. Only flag patterns that would execute against the agent running the skill. Done when: the filter count and retained findings are set.
7. Classify overall risk level as the maximum severity across all retained findings. If no findings remain after filtering, classify as LOW. Done when: the risk level is assigned with evidence.
8. Compile the report. Done when: the report matches the Output contract.