Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx agentmods add rules/xclaw-bot/benchmark-task-authoring/50-verifiergit clone --depth 1 https://github.com/Xclaw-bot/benchmark-task-authoringWhat it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5 | $0.01016 | $0.01016 |
| Opus 5 | $0.00508 | $0.00508 |
| Sonnet 5 | $0.00203 | $0.00203 |
| Haiku 4.5 | $0.00102 | $0.00102 |
Grade A, and why
50-verifier scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 2d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 91 lines — stays where its author put it; the contents beside it link to each section on GitHub.
description: Verifier design: ground-truth protection, tolerance justification, anti-cheat, and the pre-submission self-check. Auto-attaches to test files. globs: /task/tests/ alwaysApply: false
Verifier craft
The verifier runs in the same image as the agent, after the agent stops.
Harbor overlays tests/ at /tests only at verify time. Everything it needs must
be baked into environment/Dockerfile (pinned) — test.sh installs and downloads
nothing.
Ground truth must be unreachable
❌ truth read from a path the submission can also open
✅ truth lives where the submission cannot reach it — tests/, overlaid at verify time
Expected values belong in tests/, never in environment/, never in the image.
Prefer values derived in the solution over hardcoded constants, so a
non-specialist reviewer can follow the derivation.
Assertions
- Every assertion traces to a requirement stated in
instruction.md; every stated requirement has coverage. Grading on unstated criteria is an unfair stump. - One requirement per test function, each with a docstring naming the behaviour it checks in the instruction's terms.
- Verify behaviour by execution, never by scanning source for keywords, imports or regexes — an agent can insert a keyword without implementing anything.
- Existence and formatting checks should rarely fail for a competent solver. If they are what fails, the defect is in the instruction.
Tolerances
Exact comparison is preferred and needs no defence when the pipeline is
integer-deterministic and the format is normatively pinned. Any inequality-based
check — ranges, thresholds, percentile bounds, fuzzy matching — must have its
calibration justified in verification_explanation: what the band brackets, which
legitimate variation it absorbs (float precision, alternative valid algorithms,
quadrature, rounding), and whether it was validated against an alternative correct
method rather than only the reference. A band only the reference can hit is too
tight; a band that admits obviously wrong answers is too loose. Bare statements
like "accepted range [29, 31]" are insufficient.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 2d ago First seen · 91 lines · 1,016 tokens per session scan A 95cac74d33cf
50-verifier is a cursor rule published in the GitHub repository Xclaw-bot/benchmark-task-authoring (2 stars, last pushed 18d ago), licensed MIT. It adds 1,016 tokens to every session, about $0.0051 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.
Other cursor rules, from other repositories
angular-20
This rule provides comprehensive best practices and coding standards for Angular development, focusing on modern TypeScript, standalone components, signals, and performance optimizations.
dev-standard
Apache Superset development standards and guidelines for Cursor IDE.
cli-error-handling
CLI command error handling patterns.
prefer-direct-imports-over-module-mocks
Prefer extracting a testable core over vi.mock / vi.resetModules when unit tests need to reach production logic entangled with config, env, or singletons.
control-plane-descriptors
Control plane descriptor and instance implementation patterns.
family-instance-domain-actions
Family instance domain action implementation patterns.