answer-audit

answer-audit is a skill for Claude Code from olehsvyrydov/AI-development-team. It costs 78 tokens per session (1,481 once invoked), scanned A, original, MIT.

A review method for checking whether a retrieval-augmented generation answer is both supported and accurate against the original documents it cites. Retrieval-augmented generation means an AI answer built from retrieved source text.

In plain words
What is it for?
Use it to fact-check grounded answers, especially figures, units, percentages, thresholds, and other precise details.
Why use it?
A citation can point to a real passage without proving that the answer copied its details correctly. This audit checks each claim against the source text.

Skill for Claude Code

Written for Claude Code: shipped in a Claude Code plugin.

Part of the ai-dev-team plugin — 7 skills, 50 commands shipped together

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

agentmods
npx agentmods add skills/olehsvyrydov/ai-development-team/answer-audit
Any agent
npx skills add olehsvyrydov/AI-development-team --skill answer-audit
Clone the repo
git clone --depth 1 https://github.com/olehsvyrydov/AI-development-team

Made for: Claude Code.

Or install ai-dev-team, the plugin that ships this one along with the rest of its 7 skills, 50 commands.

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

agentmods badge for answer-audit

README.md
[![agentmods](https://agentmods.dev/badge/skills/olehsvyrydov/ai-development-team/answer-audit.svg)](https://agentmods.dev/skills/olehsvyrydov/ai-development-team/answer-audit)
Your own site
<a href="https://agentmods.dev/skills/olehsvyrydov/ai-development-team/answer-audit"><img src="https://agentmods.dev/badge/skills/olehsvyrydov/ai-development-team/answer-audit.svg" alt="Measured on agentmods" height="20"></a>
Per session 78 Skills are progressive disclosure: only the name and description are preloaded; the body loads when the skill is used.
When invoked 1,481 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 1 finding. Scan, not verified.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5.1 $0.00078 $0.01481
Opus 5 $0.00039 $0.00740
Sonnet 5 $0.00016 $0.00296
Haiku 4.5 $0.00008 $0.00148

Measured 6d ago against content hash 6dc0f3ebc7e6, method: parsed. Prices are Anthropic first-party input rates as of 2026-09-06, from the pricing page.

Security

Grade A, and why

answer-audit scanned grade A with 1 finding against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 6d ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Makes network callslowCapability

Not a fault in itself. Listed so you know the mod talks to something, and to what.

`chunks` table. Neither container has curl; use bash `/dev/tcp` from the api container.
claude/skills/answer-audit/SKILL.md · 94 lines

How it starts

The opening of the file, as written. The whole thing — 94 lines — stays where its author put it; the contents beside it link to each section on GitHub.

Answer Audit — skeptical review of a grounded answer

A RAG answer that is fully citation-grounded can still be materially wrong. Every claim can trace to a real passage while the values inside those claims are silently corrupted in transit. This skill exists because a confident, well-structured, 100%-grounded answer is the hardest kind to catch — the citations create trust the content has not earned.

Default posture: the answer is guilty until proven innocent. Do not grade it on how it reads. Grade it on whether each atom survives a character-level diff against its source.

The rule that matters

Grounded ≠ accurate. Citation presence is not citation support.

Most RAG guardrails ("no sources → no answer", "cite every claim") only prove a passage was retrieved. They do not prove the passage says what the answer says it says. Audit the gap.

Method

Never audit from memory or from the model's own citation list. Go to the corpus.

  1. Pull the corpus the answer was drawn from (see Canon commands below). Get the raw chunk text, not a summary of it.
  2. Atomise the answer. Extract every checkable atom: figures, currencies, units, percentages, thresholds, dates, proper nouns, and any quoted phrase.
  3. Verify each atom verbatim. Regex/grep the corpus for the exact token. An atom that does not appear verbatim is a finding, even if a similar one does. $300k and £300k are different facts.
  4. Diff the source passage against the answer's rendering of it, sentence by sentence, for the passages that carry the answer's most consequential claims. This is where the real damage hides — step 3 alone will not catch a dropped caveat.
  5. Hunt for what is missing, not just what is wrong. Omission is the most under-detected failure mode and the most dangerous, because nothing in the output signals it.
  6. Check the telemetry — which model tier answered, how many attempts, how many passages were stuffed into the prompt. Compression pressure predicts distortion: many passages + a weak tier is the setup for exactly these errors.
  7. Report defects with the source text quoted beside the answer text. Never assert a defect you have not shown.

Read the full file on GitHub · 94 lines

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 6d ago First seen · 94 lines · 78 tokens per session scan A 6dc0f3ebc7e6

Subscribe to this mod's changes

answer-audit is a skill published in the GitHub repository olehsvyrydov/AI-development-team (16 stars, last pushed 27d ago), licensed MIT. It adds 78 tokens to every session and 1,481 once invoked, about $0.0004 per session on Opus 5. A static security scan graded it A with 1 finding (makes network calls). No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.