What is verify-setup?

A setup kit that adds executable checks to a code repository. It includes checks for prerequisites, types, linting, tests, feature drivers, repeated observations, and whether evidence matches the current code.

Use it to add verification to a repository that lacks it, create feature-specific checks, or ensure that checks pass on the exact current code.

It replaces written instructions about how to test with commands that can actually prove whether a change works. This gives an agent unfamiliar with the repository a repeatable way to verify changes.

verify-setup — Skill for Claude Code by Railly/skills

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

Any agent

npx skills add Railly/skills --skill verify-setup

Clone the repo

git clone --depth 1 https://github.com/Railly/skills

Made for: Claude Code.

Or install experimental, the plugin that ships this one along with the rest of its 3 skills.

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

README.md

[![agentmods](https://agentmods.dev/badge/skills/railly/skills/verify-setup/github.svg)](https://agentmods.dev/skills/railly/skills/verify-setup)

Your own site

<a href="https://agentmods.dev/skills/railly/skills/verify-setup"><img src="https://agentmods.dev/badge/skills/railly/skills/verify-setup/github.svg" alt="Measured on agentmods" height="20"></a>

Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.

Your own site · 80×15

<a href="https://agentmods.dev/skills/railly/skills/verify-setup"><img src="https://agentmods.dev/badge/skills/railly/skills/verify-setup.svg" alt="Reviewed on agentmods" width="80" height="20"></a>

Per session 124 Skills are progressive disclosure: only the name and description are preloaded; the body loads when the skill is used.

When invoked 1,189 The whole file, excluding the scripts and references it only reads on demand.

Security scan A 1 finding. A grade says what 26 rules found in the file — not that it is safe.

Origin original No closer match found in the catalogue.

Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

Model	Per session	Once invoked
Fable 5.1	$0.00124	$0.01189
Opus 5.5	$0.00050	$0.00476
Sonnet 5.5	$0.00025	$0.00238
Haiku 4.5	$0.00012	$0.00119

Measured 24d ago against content hash d7dc41aea59d, method: parsed. Prices are Anthropic first-party input rates as of 2026-10-07, from the pricing page.

Security

Grade A, and why

verify-setup scanned grade A with 1 finding against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 24d ago.

The scan reads SKILL.md. This mod also ships 10 executable files (assets/verify/check.sh, assets/verify/doctor.sh, assets/verify/drive/example.sh, …), listed below but not scanned — reading those needs a real analyzer, not pattern matching.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Makes network callslowCapability

Not a fault in itself. Listed so you know the mod talks to something, and to what.

compatibility: "Requires git, bash, and Node or Bun for the scaffold. Drivers use whatever the surface needs, such as agent-browser, curl, or the repo's own CLI."

skills/.experimental/verify-setup/SKILL.md · 52 lines

How it starts

The opening of the file, as written. The whole thing — 52 lines — stays where its author put it; the contents beside it link to each section on GitHub.

Verify setup

A repo is verifiable when an agent that has never seen it can run one command and get evidence, from the surface a user touches, that a change works. Prose that says how to check is not enough. This skill ships scripts.

Kit

node scripts/scaffold.mjs <repo-root> [--local] copies the kit into <repo>/.agents/verify/ and never overwrites. --local adds .agents/verify/ to .git/info/exclude, so the kit works on this machine without ever being versioned. Use it for repos you do not own:

File	Contract
`doctor.sh`	exit 0 only when every precondition the drivers need exists
`check.sh`	types, lint, unit tests; non-zero on any failure
`observe.sh <feature> [--runs N]`	runs `drive/<feature>.sh` with `EVIDENCE_DIR` set, N times when asked; passes only if every run passes; `meta.json` records tree digest, exit, runs, passed. `observe.sh check` runs `check.sh`
`fresh.sh`	exit 0 when every contract feature passed on its latest observation of the current tree contents; `observe: f x5` also needs at least 5 runs; the Stop hook calls it
`lib.sh` `artifact_current <artifact> <src>...`	fails when a built artifact is older than its sources, so a driver never observes another build
`drive/<feature>.sh`	reaches the feature as a user does and writes proof into `EVIDENCE_DIR`
`features/<feature>.md`	where it lives, how a user reaches it, how to drive it, what usually lies
`ENFORCERS.md`	rule to enforcer table

evidence/ is gitignored. The tree digest covers tracked changes and untracked files, so evidence from before an edit never counts after it.

Procedure

Read the repo's README, CI config, and agent instructions. List the surfaces: CLI, HTTP API, web UI, desktop, library.
Scaffold the kit. Fill doctor.sh with the real binaries, env vars and services. Fill check.sh with the commands CI runs, fastest first.
Pick the three to five features changes touch most often (git log, open issues). For each, write the driver and the feature file.
A driver observes the outcome, not a proxy. When it loads a built artifact (binary, WASM, bundle), it first proves the artifact came from this tree: artifact_current, or rebuild and compare with the committed copy. A benchmark driver measures the baseline in the same run (origin/main and this tree, same browser) and is declared with x<N> in the contract, since one run of a noisy number is a coin flip. A CLI driver runs the built binary and asserts on output. A web driver uses agent-browser to screenshot and asserts on the DOM or text. An API driver curls the running service and asserts on the body, not only the status.
Fill "What usually lies" from real failures: cases in the canonical repo under cases/<repo>/, review findings, and papercuts.
Prove the kit before handing it over: doctor.sh, check.sh, then observe.sh <feature> for every driver, on a clean tree. Open each screenshot and look at it. Then break the feature on purpose (or revert a known fix) and confirm the driver fails. A driver that cannot fail is not a driver.
Report the commands run, exit codes, and evidence paths.

Read the full file on GitHub · 52 lines

Files

What ships with it

15 files beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

24d ago First seen · 52 lines · 124 tokens per session scan A d7dc41aea59d

Subscribe to this mod's changes

verify-setup is a skill published in the GitHub repository Railly/skills (10 stars, last pushed 24d ago), licensed MIT. It adds 124 tokens to every session and 1,189 once invoked, about $0.0005 per session on Opus 5.5. A static security scan graded it A with 1 finding (makes network calls). No closer match exists in the catalogue, so it is treated as the original; first seen 2026-10-17.

Testing

verify-setup next to 6 skills from other repositories
Mod	Type	Stars	Last push	Licence	Scan	Tokens	Works with
verify-setupRailly/skills	Skill	10	24d ago	MIT	A	124	Claude Code
imagegenopenai/codex	Skill	134k	today	Apache-2.0	A	113	Codex
playwright-climicrosoft/playwright	Skill	100k	today	Apache-2.0	A	19	Claude Code
local-ai-agentsmicrosoft/ai-agents-for-beginners	Skill	79k	3d ago	MIT	A	200	Claude Code, Codex
browser-automationopenclaw/openclaw	Skill	394k	today	MIT	A	36	Claude Code
systematic-debuggingobra/superpowers	Skill	315k	6d ago	MIT	A	21	Claude Code
next-cache-components-adoptionvercel/next.js	Skill	143k	today	MIT	A	95	Claude Code

verify-setup