Caddy Audit

Caddy Audit

Auto-enrollment makes adding skills easy. Easy adds without periodic audits accumulate drift. /caddy-audit is the quarterly hygiene pass that keeps the skill index honest.

When to use

• Quarterly — once per ~90 days
• After bulk skill changes (importing skills from another principal, refactoring skills folder structure)
• When /caddy-explain shows surprising results that suggest the index doesn't match reality
• Triggered by Caddy on prompts like: "/caddy-audit", "audit caddy", "skill index drift", "is the skill index correct", "verify skill catalog", "skill drift check"

How to execute

Step 1 — Read the skill index

Load .claude/skill-index.json into memory. Each entry has at minimum: name, triggers, why.

Step 2 — Walk the skills folder

List every SKILL.md file in skills/<name>/SKILL.md and any project-scope .claude/skills/<name>.md. For each, parse the YAML frontmatter for name and description.

Step 3 — Cross-reference

For each skill in the index, check:

• Does the corresponding SKILL.md exist? If not → INDEX-ORPHAN (in index, not on disk)
• Does the SKILL.md frontmatter match what was indexed? Specifically:
  • The skill name matches
  • The skill description is consistent with the indexed why (the indexed why is usually a tighter summary; flag if they're contradictory)

For each SKILL.md file on disk, check:

• Is it in the index? If not → DISK-ORPHAN (on disk, not indexed)

Step 4 — Optional: trigger freshness check

If the skill files store original triggers in their frontmatter (not all do), compare to the indexed triggers. If the SKILL.md has been edited since enrollment AND its trigger frontmatter has diverged from the index, flag as TRIGGER-STALE.

This step is best-effort; not all skills carry trigger frontmatter, and that's fine.

Step 5 — Output the audit report

🔍 Caddy Audit ({date})

Index: {N} entries. Disk: {M} SKILL.md files.

✅ HEALTHY — {K} skills are correctly indexed and on disk.

⚠️ INDEX-ORPHANS ({count}) — in index, no matching SKILL.md:
   - /skill-a — last seen at skills/skill-a/SKILL.md (deleted? renamed?)
   - /skill-b — never had a file?

⚠️ DISK-ORPHANS ({count}) — SKILL.md exists, not in index:
   - skills/new-thing/SKILL.md — run /caddy-enroll skills/new-thing/SKILL.md
   - skills/experimental/SKILL.md — never enrolled

⚠️ TRIGGER-STALE ({count}, best-effort) — SKILL.md frontmatter diverges from index:
   - /skill-c — SKILL.md has 12 triggers, index has 8

Recommended actions:
1. {1-3 specific commands the principal should run to remediate}