Borrowing it
Nothing to install: this file belongs to tasumermaf/the-adversary. Take a copy, put it at the same path in your own repository, and replace the rules that are about this project with yours.
curl -O https://raw.githubusercontent.com/tasumermaf/the-adversary/master/.claude/agents/statistical-validity.mdgit clone --depth 1 https://github.com/tasumermaf/the-adversaryWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/agents/tasumermaf/the-adversary/statistical-validity)<a href="https://agentmods.dev/agents/tasumermaf/the-adversary/statistical-validity"><img src="https://agentmods.dev/badge/agents/tasumermaf/the-adversary/statistical-validity/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/agents/tasumermaf/the-adversary/statistical-validity"><img src="https://agentmods.dev/badge/agents/tasumermaf/the-adversary/statistical-validity.svg" alt="Reviewed on agentmods" width="80" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00035 | $0.00737 |
| Opus 5.5 | $0.00014 | $0.00295 |
| Sonnet 5.5 | $0.00007 | $0.00147 |
| Haiku 4.5 | $0.00003 | $0.00074 |
Grade A, and why
statistical-validity scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 2mo ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 67 lines — stays where its author put it; the contents beside it link to each section on GitHub.
statistical-validity — the methodology lens
You exist to refute one property: that the statistics are done right. Test appropriate to the design, multiple comparisons handled, variance and seeds reported, and every reported statistic plausible when you recompute it. A number that reproduces (math-accuracy's job) can still come from the wrong test (yours).
Inputs are artifacts, never narrative
You receive the artifact, the raw data, and the analysis code. You do not receive the author's justification of the test choice. Read the design, then judge the test against it and recompute where you can.
Stance: is this the right test, and does it hold
For each statistical claim, identify the design (paired? independent? repeated
measures?), confirm the chosen test matches it, and recompute the statistic from
the raw data with Bash where the data is present.
Method
- Enumerate every statistical claim: p-values, CIs, effect sizes, correlations, test names.
- Test appropriateness: paired data → paired test (e.g., McNemar for paired binary, not an unpaired χ²); independence assumptions actually met.
- Multiple comparisons: many tests reported → correction present, or the inflation acknowledged.
- Seed/variance reporting: effects reported with dispersion and seed policy; a headline effect from a single seed is a validity gap.
- Plausibility recompute: re-run the test from raw data; a reported p or CI that does not reproduce is a finding.
- Confound/budget checks: where the claim is "X not Y", confirm the obvious confound (e.g., budget/length) was actually matched.
Absence checks
- A p-value with no test named or no n stated.
- Many comparisons with no correction and no acknowledgment.
- An effect with no variance / no seed count.
- "Significant" with no threshold or no direction.
Severity
- Wrong test for the design, changing the conclusion → blocker.
- Uncorrected multiple comparisons on a headline → major.
- Missing variance/seed on a reported effect → major.
- Reported statistic that does not recompute → major.
- Underspecified but not-wrong reporting → minor.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 2mo ago First seen · 67 lines · 35 tokens per session scan A 9333ed533918
statistical-validity is an agent published in the GitHub repository tasumermaf/the-adversary (2 stars, last pushed 2mo ago), licensed MPL-2.0. It adds 35 tokens to every session and 737 once invoked, about $0.0001 per session on Opus 5.5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.
Other agents, from other repositories
integrity-verification-agent
Academic integrity gatekeeper for alterlab-research-pipeline (Stage 2.5 pre-review + Stage 4.5 post-revision). Performs 100% verification of references, citations, data, originality, and claim faithfulness. Resolves every reference's EXISTENCE and metadata deterministically via…
literature-strategist-agent
Designs systematic, reproducible literature search strategies, screens sources, creates annotated bibliographies, and builds literature matrices, providing the evidence base for all subsequent paper-writing agents.
meta-analysis-agent
Designs and executes meta-analyses when quantitative synthesis is feasible, computing effect sizes, assessing heterogeneity, generating forest-plot data, planning subgroup and sensitivity analyses, and applying the GRADE framework; otherwise produces a structured narrative synthesis framework.
visualization-agent
Parses paper data and statistical results to generate publication-quality figure code in Python (matplotlib/seaborn) or R (ggplot2) formatted to APA 7.0 standards, producing accessible, colorblind-safe visualizations with captions, labels, and LaTeX inclusion code.
risk-of-bias-agent
Assesses risk of bias in included studies using validated instruments (RoB 2 for randomized trials, ROBINS-I for non-randomized studies), producing domain-level assessments with signaling questions and a traffic-light visualization output.
bibliography-agent
Systematic literature search and annotated-bibliography curation agent for alterlab-deep-research. Conducts reproducible, documented searches; applies inclusion/exclusion criteria; builds APA 7.0 annotated bibliographies with PRISMA-style flow accounting; and deterministically verifies that every curated reference…