citation-audit

Citation Audit Skill

Purpose

Verify every citation in a manuscript against its actual source. LLMs hallucinate citations, invent arXiv IDs, misattribute findings, and confuse authors. This skill catches all of that by fetching and reading each cited work.

Why This Exists

LLMs are unreliable with citations in three distinct ways:

1. Ghost papers — The paper does not exist. Title, authors, or venue are fabricated.
2. Wrong metadata — The paper exists but the bib entry has the wrong arXiv ID, wrong authors, wrong year, or wrong venue.
3. Inverted claims — The paper exists and the bib is correct, but the manuscript mischaracterizes what the paper says.

All three are invisible to structural audits (cross-reference checks, compilation tests). They require reading the actual cited work.

Inputs

• The manuscript .tex file(s)
• The .bib file
• Web access (to fetch papers from arXiv, conference sites, URLs)

Execution

Phase 1: Extract citation contexts

For each \citep{}, \citet{}, \cite{} in the manuscript:

1. Record the bib key
2. Record the surrounding sentence or paragraph (the claim context)
3. Classify the claim type:
   • FACTUAL: "X et al. found Y" / "X et al. measured Y"
   • METHODOLOGICAL: "We follow X" / "We use the benchmark from X"
   • POSITIONAL: "Unlike X, we..." / "X does not measure..."
   • PARENTHETICAL: "(X, 2024)" — no specific claim, just a reference
4. For FACTUAL and POSITIONAL claims, extract the specific assertion the manuscript makes about the cited work

Phase 2: Verify bib entry metadata

For each bib entry, verify against the actual source:

For arXiv papers (eprint field present):

1. Fetch https://arxiv.org/abs/{eprint_id}
2. Compare: title, authors, year
3. If the fetched paper has a DIFFERENT title/authors than the bib entry, this is a WRONG ID or GHOST PAPER

For conference/journal papers (booktitle or journal field):

1. Search for the paper by title + author on the web
2. Verify: venue, year, author list
3. If the paper cannot be found at the stated venue, flag as UNVERIFIABLE or GHOST PAPER