Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx agentmods add skills/markusneusinger/cite-citadel/bench-modelnpx skills add MarkusNeusinger/cite-citadel --skill bench-modelgit clone --depth 1 https://github.com/MarkusNeusinger/cite-citadelWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/markusneusinger/cite-citadel/bench-model)<a href="https://agentmods.dev/skills/markusneusinger/cite-citadel/bench-model"><img src="https://agentmods.dev/badge/skills/markusneusinger/cite-citadel/bench-model.svg" alt="Measured on agentmods" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5 | $0.00306 | $0.02536 |
| Opus 5 | $0.00153 | $0.01268 |
| Sonnet 5 | $0.00061 | $0.00507 |
| Haiku 4.5 | $0.00031 | $0.00254 |
Grade C, and why
bench-model scanned grade C with 1 finding against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 3d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Recursive force deletehighDestructive command
rm -rf with a variable or a broad path is one typo away from removing the wrong tree.
throwaways (`rm -rf`), kept only while a `--baseline` comparison still needs them. How it starts
The opening of the file, as written. The whole thing — 149 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Bench a model on wiki creation
Usage: bench-model <corpus> <model> [--cli claude|copilot|agy] [--baseline SANDBOX] [--grade-only SANDBOX]
Everything sandbox/grading-related follows verify-corpus (read its SKILL.md first — sandbox recipe, phase 1 structural gates, phase 2 retrieval-first grading, the creation-vs-retrieval miss matrix, the gotchas). This skill changes the question: not "does the pipeline still work?" but "how well does THIS model drive it?" — so the model is the variable, the grade gains a discriminative tier, and the output is a comparison, not just pass/fail.
Reference points (2026-07 audit, beverages, 14 sources)
| claude-sonnet-5 | claude-haiku-4-5 | |
|---|---|---|
| pace | ~7.5 min/source | ~3 min/source (~2.5× faster) |
| structural gates | check+lint clean | check+lint clean |
| ground-truth grade | no wiki-defects on its subset | 7 wiki-defects |
| judgment failures | none observed | 3 uncaught contradictions; a planted-false claim ("coffee loses caffeine with age") adopted as unqualified wiki-voice truth; partial 2024→2026 supersession |
The pattern to test for: cheap models produce structurally valid, well-cited pages at speed but miss judgment-heavy work — surfacing contradictions, quarantining suspicious claims, temporal supersession, open-points discipline. Structural gates alone will NOT separate models.
Mode A — build with the model under test
Per verify-corpus's sandbox recipe, with the model pinned; run ingest in the background (a foreground shell call gets killed by per-command timeouts long before a full corpus finishes):
REPO="$(git rev-parse --show-toplevel)"
SANDBOX="$(mktemp -d)/bench-<corpus>-<model>"
uv run python -m citadel init "$SANDBOX"
cat > "$SANDBOX/.env" <<EOF
CITADEL_LLM_CLI=claude # or copilot / agy — the seam for open/local models
CITADEL_INGEST_MODEL=<model>
CITADEL_LLM_LOG_DIR=$SANDBOX/logs
CITADEL_RAW_DIR=$REPO/corpora/<corpus>/raw
EOF
# sanity-check the model id first (cheap): claude --model <model> -p "Reply with exactly: ok"
CITADEL_WORKSPACE="$SANDBOX" uv run python -m citadel ingest --quiet & # background; poll:
uv run python -c "import json;print(len(json.load(open('$SANDBOX/wiki/.citadel_ingested.json'))['sources']))"
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 3d ago First seen · 149 lines · 306 tokens per session scan C 54e4ffde665a
bench-model is a skill published in the GitHub repository MarkusNeusinger/cite-citadel (2 stars, last pushed 5d ago), licensed MIT. It adds 306 tokens to every session and 2,536 once invoked, about $0.0015 per session on Opus 5. A static security scan graded it C with 1 finding (recursive force delete). No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.
Other skills, from other repositories
design-debt-audit
Inventory and prioritise accumulated design inconsistencies across a product. Use when drift has built up over time. For token coverage specifically use design-token-audit (designer-toolkit); for WCAG gaps use accessibility-audit (design-systems).
design-impact-reporting
Communicate design's contribution to business and user outcomes in stakeholder language. Use when reporting results upward. For choosing the metrics in the first place, use metrics-definition (ux-strategy).
research-repository
Build a repository that makes findings findable, reusable, and cumulative across teams. Use when the same research keeps getting redone. For synthesising one study, use affinity-diagram.
survey-design
Design unbiased survey instruments — question wording, scales, and sampling — to measure attitudes at scale. Use when you need quantitative breadth. For behavioural experiments, use a-b-test-design (prototyping-testing).
localization-design
Design for multiple languages, writing directions, and cultural contexts — text expansion, RTL mirroring, and locale formats. Use when shipping beyond one locale. For the words themselves, use ux-writing (designer-toolkit).
motion-system
Define motion tokens — durations, easing vocabulary, and reduced-motion handling — for consistency product-wide. Use when standardising motion across a system. For crafting one specific animation, use animation-principles (interaction-design).