Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx agentmods add commands/micheltlutz/dev-challenge/verifygit clone --depth 1 https://github.com/micheltlutz/dev-challengeWhat it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5 | $0.00011 | $0.00483 |
| Opus 5 | $0.00005 | $0.00242 |
| Sonnet 5 | $0.00002 | $0.00097 |
| Haiku 4.5 | $0.00001 | $0.00048 |
Grade A, and why
verify scanned grade A with 1 finding against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 2d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Makes network callslowCapability
Not a fault in itself. Listed so you know the mod talks to something, and to what.
allowed-tools: Read, Grep, Glob, Bash(python harness/cli.py:*), Bash(pytest harness/:*), Bash(curl:*) What it actually says
Verify a solution against its acceptance criteria.
Work out the target
Default --base-url is http://localhost:8000. If $ARGUMENTS names a
challenge or a URL, use those. If you do not know where their app is running,
ask — do not guess and report a connection error as a failure.
python harness/cli.py --base-url <url> --challenge <id>
Omit --challenge to run everything.
For the web track the runner is Playwright:
cd harness/ui/web && BASE_URL=<their-frontend> npx playwright test
Explain the scorecard
Do not paste the raw output and stop. For each failing criterion:
- Quote the criterion — its ID and its Given/When/Then.
- State what actually happened.
- Point at the likely cause, in their code if you can see it.
- Ask what they think is going on, before giving the fix. Default posture is
mentoring; see
AGENTS.md.
For each skipped criterion, explain that a skip is not a pass and name the missing prerequisite:
| Skip | Meaning |
|---|---|
| "Could not register a user" | backend-02-create-user is missing |
| "Could not authenticate" | backend-03-login is missing |
| "Reference seed data not present" | needs two users matching the fixture |
| "Cannot create statements" | POST /statement/ is missing |
| "GET /users/me returned 404" | the client cannot learn its own id |
Ignore uncovered criteria unless they ask — those are UI or manual-review items the API harness cannot reach, not something they did wrong.
Close
One line on where they stand — "6 of 7, and the last one is the scoping rule" —
and what to do next. If everything passes, say so without padding, and point at
/next.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 2d ago First seen · 56 lines · 0 tokens per session scan A d1dd2068801f
verify is a command published in the GitHub repository micheltlutz/dev-challenge (32 stars, last pushed 15d ago), licensed MIT. It adds 11 tokens to every session and 483 once invoked, about $0.0001 per session on Opus 5. A static security scan graded it A with 1 finding (makes network calls). No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other commands, from other repositories
wikiweb
Command "wikiweb" from kimsanguine/llm-brain, covering 실행, 제공 기능 and 데이터가 없다면 (선택).
kg
把当前 session 蒸馏成知识图谱 (KG) - 4 类节点 (person/event/concept/artifact) + 6 类边. 适合关系密集的复盘 / 跨团队事件 / 季度回顾.
speckit.assess.intake
Capture and normalize a raw idea (text, URL, ticket, or codebase pointer) into an intake note.
auto-browse
Auto-browse — learn, optimize, and graduate browser operations or web data-mining workflows.
os-meeting
快速创建和管理AI团队会议.
research-perplexity
Run a deep research query using Perplexity's /research mode via Playwright browser automation. This is an alternative to /export-to-council that uses Perplexity's dedicated research mode instead of multi-model council.