ai-output-verifier

AI-Output Verifier

AI is fluent, confident, and sometimes completely wrong — inventing facts, citations, and details in the same authoritative tone as the correct ones. That confidence is exactly what makes unverified trust dangerous. This checks a specific output: which claims are most likely wrong or fabricated, what genuinely needs independent verification, how to verify it, and the tells of hallucination — so you use AI's speed without inheriting its errors.

What This Skill Produces

• A risk read of the output — which specific claims are most likely to be wrong, outdated, or made up (facts, numbers, citations, names, recent events, specifics)
• Verify vs. low-risk split — what genuinely needs independent checking vs. what's low-stakes or self-evident, so you spend effort where it counts
• How to verify each — the concrete way to check the high-risk claims (a primary source, a second tool, a domain expert, testing it)
• The hallucination tells — the signs AI is likely fabricating (oddly specific citations, confident claims about recent/niche facts, plausible-but-unverifiable details)
• A verification habit — how to build appropriate checking into your AI use by default, scaled to the stakes (trust more for low-stakes, verify hard for high-stakes)

Required Inputs

Ask for these if not provided:

• The output — the AI response to check (paste it)
• What it's for — the stakes (a casual question vs. something you'll publish, decide on, or act on)
• The domain — factual/technical/legal/medical/current-events (some are far higher-risk for AI)
• What you'd do with it — trust it, act on it, share it, build on it

Framework: Risk-Rate The Claims, Verify What Matters

1. Scan for the high-risk claim types. Specific facts, numbers, dates, names, citations, recent events, and niche/technical specifics are where AI most often invents — flag these.
2. Split by risk and stakes. Separate the claims that genuinely need verification (high-risk × high-stakes) from the low-risk or low-stakes ones you can reasonably accept — don't verify everything equally.
3. Verify against real sources. For the high-risk claims, check a primary source, a second independent tool, an expert, or by testing — not by asking the same AI "are you sure?" (it'll often just re-confirm).
4. Watch the hallucination tells. Oddly precise citations, confident answers about very recent or obscure things, and unverifiable specifics are red flags — treat them as unverified until checked.
5. Scale trust to stakes. For low-stakes uses, light verification is fine; for anything you'll publish, decide on, or that could harm if wrong, verify hard. Build this reflex in.