ai-audit-docs

Audit docs against repo reality

Lifecycle

Lane: light Writes: nothing — report-only, never modifies files Read by: humans (consumed as a findings table) Dies: on completion Next: ai-verify (to validate fixes if user acts on findings)

What it produces

A ranked markdown table of findings. Every finding names the doc file, the line, the severity, and the broken claim.

Steps

1. Determine scope. If the user gave a path, audit that. Otherwise default to README.md, AGENTS.md, and any docs/ directory at the repo root. Glob for **/*.md under docs if the directory exists. Never audit vendored or generated files (node_modules/, dist/, *.generated.*).

2. Read every target doc. Extract all of these per file:

   • File links: markdown links pointing at local paths (e.g. ./path, ../path, path/to/file). Skip anchors (#section) and bare URLs — those are checked in step 3.
   • External URLs: bare http:// / https:// links. Collect for batch HEAD-check.
   • Command references: inline code or code blocks containing shell commands (npm run X, yarn X, make X, cargo X, python -m X, npx X). Strip arguments; extract just the command name.
   • Version strings: v1.2.3, 1.2.3, >=1.2.3, pinned deps in prose.
   • Feature claims: sentences that assert existence — "supports X", "includes X", "has X", "provides X", "X is built-in", "X is enabled by default".

3. Cross-check file links. For each local path extracted from a markdown link:

   • Resolve it relative to the doc file's directory.
   • find or read to confirm the target exists. If not -> critical finding.

4. Cross-check command references. For each command name:

   • grep the repo for a script/package-bin definition (package.json scripts, Makefile targets, Cargo.toml bin, setup.py/pyproject.toml entry points, shell scripts in bin/ or scripts/).
   • If no definition exists AND the command is not a well-known system binary (git, ls, cat, curl, jq, etc.) -> warning finding.
   • If a script IS defined, check that the arguments the doc uses still match the script's actual flags — flag mismatches -> info.