Borrowing it
Nothing to install: this file belongs to heidihelena/citevahti. Take a copy, put it at the same path in your own repository, and replace the rules that are about this project with yours.
curl -O https://raw.githubusercontent.com/heidihelena/citevahti/main/.claude/skills/prescreen-benchmark/SKILL.mdgit clone --depth 1 https://github.com/heidihelena/citevahtiWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/heidihelena/citevahti/prescreen-benchmark)<a href="https://agentmods.dev/skills/heidihelena/citevahti/prescreen-benchmark"><img src="https://agentmods.dev/badge/skills/heidihelena/citevahti/prescreen-benchmark/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/heidihelena/citevahti/prescreen-benchmark"><img src="https://agentmods.dev/badge/skills/heidihelena/citevahti/prescreen-benchmark.svg" alt="Reviewed on agentmods" width="80" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00240 | $0.02120 |
| Opus 5 | $0.00120 | $0.01060 |
| Sonnet 5 | $0.00048 | $0.00424 |
| Haiku 4.5 | $0.00024 | $0.00212 |
Grade A, and why
prescreen-benchmark scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 9d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 133 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Prescreen benchmark — local LLMs as CiteVahti prescreening agents
This skill runs the experiment: take a topic, author a small corpus of claim ↔ cited-source pairs each with an independently-authored anchor label, have local Ollama models (plus a Claude column) prescreen every pair blind, then measure agreement, ingest the run into a dedicated CiteVahti store, and render a self-contained prescreen report.
Naming. This report is not the Atlas. In CiteVahti, Atlas means the panel's claim↔evidence graph (nodes and links —
panel/web/evidence-map.js, Spine layout). What this skill renders is a scoreboard and a rating matrix, so it is called the prescreen report (scripts/report.py). Pages published before this rename still carry the old "Evidence Atlas" masthead; leave them as they are.
CiteVahti is a local-first citation-integrity tool used in live pilots with real researchers. A benchmark that overstates what a laptop model can do — or that quietly grades a model against its own author — is exactly the kind of self-flattering evidence this product exists to prevent. So the guardrails below are load-bearing, not decoration.
Non-negotiable guardrails (read first)
- The anchor must be authored independently of every rater. Agreement is not accuracy (house doctrine). If the same author writes the claims and the anchor and rates them, a 100% score is self-consistency, not correctness. Best: the clinician/domain expert (or a cited guideline document) supplies the anchor labels; the models — Claude included — are then scored against that anchor through the identical blind prompt. Whenever the anchor was NOT independently authored, say so in the write-up.
- Frozen vocabulary. Exactly four coarse labels:
supports,contrasts,unclear,not_relevant. Never invent a new scale. (ingest.pymaps these onto CiteVahti's canonical 7-value support vocabulary.) - Never touch the real pilot ledger. Every run ingests into a separate root
(
~/Documents/CiteVahti-<theme>). The founder's ledger at~/Documents/CiteVahtiis off-limits. - Blind + human-first. Models never see the anchor. In the store the anchor is the human rating; the model is the blind AI second opinion; divergences are adjudicated to the anchor.
- Trust language. In any report copy: check / assess, never verify / prove / guarantee. The report already carries the "agreement ≠ accuracy" caveat — keep it.
What ships with it
10 files beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.
- references/method.md 5.6 KB
- scripts/bench.py 8.8 KB runs code
- scripts/ingest.py 4.8 KB runs code
- scripts/prompt_v2.py 4.2 KB runs code
- scripts/report.py 16 KB runs code
- scripts/suite.py 15 KB runs code
- seeds/_TEMPLATE.json 1.9 KB
- seeds/lung-cancer-neoadjuvant.json 20 KB
- seeds/lung-cancer-risk-factors.json 18 KB
- seeds/lung-nodule-management.json 22 KB
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 9d ago First seen · 133 lines · 240 tokens per session scan A 431612ac2e97
prescreen-benchmark is a skill published in the GitHub repository heidihelena/citevahti (1 stars, last pushed 4d ago), licensed Apache-2.0. It adds 240 tokens to every session and 2,120 once invoked, about $0.0012 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.
Other skills, from other repositories
ml-paper-writing
Write publication-ready ML/AI papers for NeurIPS, ICML, ICLR, ACL, AAAI, COLM. Use when drafting papers from research repos, structuring arguments, verifying citations, or preparing camera-ready submissions. Includes LaTeX templates, reviewer guidelines, and citation verification workflows.
idea-bib-review
A writing workflow for drafting a literature review from a research idea and BibTeX files, which are structured records of academic sources. It preserves citation keys and checks whether claims are supported by the supplied evidence.
ml-paper-writing
Write publication-ready ML/AI papers for NeurIPS, ICML, ICLR, ACL, AAAI, COLM. Use when drafting papers from research repos, structuring arguments, verifying citations, or preparing camera-ready submissions. Includes LaTeX templates, reviewer guidelines, and citation verification workflows.
ml-paper-writing
Write publication-ready ML/AI papers for NeurIPS, ICML, ICLR, ACL, AAAI, COLM. Use when drafting papers from research repos, conducting literature reviews, finding related work, verifying citations, or preparing camera-ready submissions. Includes LaTeX templates, citation verification workflows, and paper…
econ-management-paper-polish
A writing and editing guide for papers in economics, management, finance, accounting, marketing, information systems, public administration, and related fields.
socsci-paper-orchestrator
An orchestrator for writing Korean social-science research papers with a team of six specialized agents. Social science studies people, organizations, and society; a research paper presents a question, evidence, methods, and conclusions.