validation

An agent for independently checking an analytical report by redoing important calculations, reviewing data sources, and finding statistical mistakes.

In plain words
What is it for?
Use it to validate selected or all quantitative claims in a report against its SQL, Python, notebook, or underlying data.
Why use it?
It reduces the risk of publishing incorrect numbers or conclusions from analysis code and reports.

Agent

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

agentmods
npx agentmods add agents/ai-analyst-lab/ai-analyst/validation
Clone the repo
git clone --depth 1 https://github.com/ai-analyst-lab/ai-analyst
Per session 0 Only the description is in the session, so the agent can decide to use it. The body loads when it is invoked.
When invoked 4,449 The whole file, excluding the scripts and references it only reads on demand.
Security scan C 1 finding. Scan, not verified.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5 $0.00000 $0.04449
Opus 5 $0.00000 $0.02225
Sonnet 5 $0.00000 $0.00890
Haiku 4.5 $0.00000 $0.00445

Measured 3d ago against content hash 48606c27143a, method: parsed. Prices are Anthropic first-party input rates as of 2026-08-30, from the pricing page.

Security

Grade C, and why

validation scanned grade C with 1 finding against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 3d ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Hidden instructionshighPrompt injection

Directives inside HTML comments, invisible characters or bidirectional overrides are read by the model and not by the person reviewing the file.

<!-- CONTRACT_START name: validation description: Independently verify analytical findings by re-deriving key numbers, checking arithmetic, cross-referencing data sources, and flagging common statistical errors. inputs:
Origin

Copies of this mod

1 near-identical copy found in the catalogue:

agents/validation.md · 323 lines

How it starts

The opening of the file, as written. The whole thing — 323 lines — stays where its author put it; the contents beside it link to each section on GitHub.

Agent: Validation

Purpose

Independently verify analytical findings by re-deriving key numbers, checking arithmetic, cross-referencing data sources, and flagging common statistical errors — producing a pass/fail validation report with confidence ratings.

Inputs

  • {{ANALYSIS_CODE}}: Path to the analysis code (SQL queries, Python scripts, or notebook) that produced the results. The agent will re-execute key queries independently.
  • {{ANALYSIS_RESULTS}}: Path to the analysis report containing findings, numbers, charts, and conclusions. This is what gets validated.
  • {{DATA_SOURCE}}: (optional) Connection string, file path, or database reference for the underlying data. If not provided, the agent will attempt to extract the data source from the analysis code.
  • {{VALIDATION_SCOPE}}: (optional) Which findings to validate — "all" (default), or a comma-separated list of finding numbers (e.g., "1,3,5") for targeted validation. Use targeted validation when the full analysis is large and only specific findings need checking.

Workflow

Step 1: Inventory the claims

Read {{ANALYSIS_RESULTS}} end to end. Extract every quantitative claim into a numbered list. A "claim" is any statement that includes a specific number, percentage, ratio, trend direction, comparison, or ranking. For each claim, record:

  • Claim ID: Sequential number (C1, C2, C3...)
  • Statement: The exact text of the claim as it appears in the report
  • Number(s): The specific values cited (e.g., "23%", "$1.2M", "3.5x")
  • Source section: Where in the report the claim appears
  • Derivable?: Whether the claim can be independently re-derived from the code and data (yes/no)

Read the full file on GitHub · 323 lines

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 3d ago First seen · 323 lines · 0 tokens per session scan C 48606c27143a

Subscribe to this mod's changes

validation is an agent published in the GitHub repository ai-analyst-lab/ai-analyst (296 stars, last pushed 5d ago), licensed MIT. It costs nothing until one of its globs matches a file; then it loads 4,449 tokens. A static security scan graded it C with 1 finding (hidden instructions). No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.

Related

Other agents, from other repositories

editor

Journal editor who desk-reviews manuscripts, selects two referees with deliberately different dispositions, calibrates to a target journal from .claude/references/journal-profiles.md, and synthesizes an editorial decision (FATAL / ADDRESSABLE / TASTE). Used by /review-paper --peer [journal].

pedrohcgs/claude-code-my-workflow · 64 tokens

algorithm-expert

RL algorithm expert. Fire when working on GRPO/PPO/DAPO/GSPO/SAPO algorithms, reward functions, advantage normalization, loss computation, or training loop implementation.

redai-infra/Relax · 37 tokens

by-epitope

Deep epitope analysis agent. Maps binding interfaces from PDB structures, classifies epitope type, assesses druggability, identifies cryptic sites, cross-references SAbDab, and generates hotspot arrays in BoltzGen entities YAML format.

001TMF/blatant-why · 58 tokens

mathodology-coder

Use for reproducible computation, simulation, optimization, figures, tables, and experiment logs.

sweetcornna/mathodology · 24 tokens

mathodology-problem-analyst

Use for contest problem decomposition, scoring criteria, constraints, variables, assumptions, and deliverable mapping.

sweetcornna/mathodology · 29 tokens

scientist

AI/ML researcher — paper analysis, hypothesis generation, experiment design. ONLY for named research paper/hypothesis/experiment. NOT for general Python (foundry:sw-engineer), SOTA surveys (/research:topic), web content (foundry:web-explorer), dataset acquisition (research:data-steward). TRIGGER: implementing from…

Borda/AI-Rig · 77 tokens