seo-indexability

seo-indexability (M2)

If a page can't be crawled, indexed, or canonicalized correctly, every other signal is wasted — this is the floor under the Search score. Schema-side context: references/schema-tier1.md. AI-bot access and snippet eligibility are M14 (seo-ai-crawlers); this module owns the classic-Search side, including the search-axis cloaking check.

Inputs

Work from the PageSnapshot named in your dispatch envelope: read parsed from <run_dir>/pages/<slug>.json (canonicals[], robots_meta[]) plus status_chain, headers, robots_directives.effective, and the crawl graph in <run_dir>/crawl.json for orphans, depth, duplicate clusters, and internal link status; <run_dir>/profile.json carries environment.kind (production|preview|staging|local); when the run included a user-agent diff, <run_dir>/ua-diff.json holds the per-UA response fields. Grep pages/<slug>.html for verbatim evidence; site artifacts live in <run_dir>/site/{robots.json,sitemaps.json,discovery.json}. Deterministic findings already emitted by audit.mjs are listed in <run_dir>/findings.deterministic.json — do not re-emit those ids; add model-judged findings only. If invoked directly with a URL/path and no snapshot exists, first run node "${CLAUDE_PLUGIN_ROOT}/scripts/snapshot.mjs" <target> --out "${CLAUDE_PLUGIN_DATA}/runs" and use the printed snapshot path.

Audits

Working from the PageSnapshot (parsed_rendered when render.used is not none, else parsed) plus headers, status_chain, and robots_directives:

1. Canonical: exactly one <link rel="canonical">; absolute HTTPS URL; classify self-referential vs cross-domain vs chained (canonical points to a URL that itself canonicalizes elsewhere). Flag a canonical that resolves to a redirect or a 4xx.
2. Canonical + noindex — read it correctly. noindex together with a self-referential canonical is not the lethal pair: it is a coherent, usually deliberate exclusion (thank-you pages, filtered views, staging). Report it as M2.robots.noindex_present and ask before touching it. The pair is lethal only when the canonical points elsewhere: the page tells Google "index that other URL instead" while also saying "don't index me", and the contradiction can propagate the noindex to the canonical target — that is M2.canonical.noindex_conflict.
3. Robots directives: <meta name="robots"> and the X-Robots-Tag header for noindex/nofollow/none; reconcile header vs meta (the most restrictive applies; a header directive reaches non-HTML responses too). Record the scope — a directive emitted by a template or an edge rule is template/site, one written into a single document is page.
4. Duplicate clusters: near-identical title/H1/body across URLs with no consolidating canonical.
5. Pagination: paginated series signals (self-canonical per page; do not canonicalize page 2+ to page 1 — that delists deep items).
6. Site health (covers M3): redirect chains (>1 hop) and loops, internal links returning 4xx/5xx, soft-404 (200 status on an empty/"not found" page), mixed content (HTTP subresources on an HTTPS page), HTTP-to-HTTPS enforcement, orphan pages (no internal inlinks inside the crawl frontier) and click-depth from the homepage.
7. User-agent content divergence (cloaking, search axis): from ua-diff.json, compare what the origin serves a default browser UA against what it serves googlebot — status, title, H1, word count, canonical, robots directives, JSON-LD. A material difference is the search-axis cloaking signal (M2.cloaking.ua_content_divergence). The AI-bot side of the same diff (a challenged GPTBot, a refused spoofed Googlebot) belongs to M14.
8. Environment: read profile.json.environment.kind. On preview/staging/local, a noindex or a blanket robots block is the expected configuration, not a defect — record M2.hosting.preview_environment_audited so the report says which host was audited.