Paperclip is a Node.js server and React application for coordinating AI agents as a business team, with goals, organizational roles, budgets, governance, and work tracking in one interface. It is used by people managing multiple agents or autonomous agent organizations across tools such as OpenClaw, Claude Code, Codex, and Cursor. The catalogue includes skills, agents, an MCP add-on, and an instruction for working with Paperclip.
Borrowing it
Nothing to install: this file belongs to paperclipai/paperclip. Take a copy, put it at the same path in your own repository, and replace the rules that are about this project with yours.
curl -O https://raw.githubusercontent.com/paperclipai/paperclip/master/.agents/skills/add-runner-eval/SKILL.mdgit clone --depth 1 https://github.com/paperclipai/paperclipWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/paperclipai/paperclip/add-runner-eval)<a href="https://agentmods.dev/skills/paperclipai/paperclip/add-runner-eval"><img src="https://agentmods.dev/badge/skills/paperclipai/paperclip/add-runner-eval/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/paperclipai/paperclip/add-runner-eval"><img src="https://agentmods.dev/badge/skills/paperclipai/paperclip/add-runner-eval.svg" alt="Reviewed on agentmods" width="80" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00030 | $0.00895 |
| Opus 5.5 | $0.00012 | $0.00358 |
| Sonnet 5.5 | $0.00006 | $0.00179 |
| Haiku 4.5 | $0.00003 | $0.00089 |
Grade A, and why
add-runner-eval scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 1mo ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
What it actually says
Add a Runner Eval
Use this skill for the Runner Evals family: a real Runner/provider session against a seeded mock control plane. Product browser/server/database/Daytona coverage belongs in add-product-e2e-eval.
Locate the Paperclip checkout using PAPERCLIP_ROOT when supplied, or
git rev-parse --show-toplevel from a checkout. From outside Git, inspect the
workspace roots (for example ~/paperclipai/paperclip) and verify that the
selected root contains packages/paperclip-runner and tests/runner-e2e.
Locate paperclip-evals using PAPERCLIP_EVALS_ROOT or a discovered sibling;
a worktree's parent directory need not contain that repository. Read
doc/evals.md and packages/paperclip-runner/docs/runner-protocol-live-evals.md,
then inspect the nearest existing case, roster, schema, and report test before
editing. Definitions and authored cases belong in the sibling
paperclip-evals/evals/paperclip-runner; Runner integration, aggregation,
viewer, and publication behavior belongs in packages/paperclip-runner.
Keep the control-plane boundary explicit in names and documentation.
The sibling eval README is the concrete map: cases live under cases/,
company fixtures under fixtures/, runtime/model settings under configs/,
selections under rosters/, and maintained campaign membership under
campaigns/live-direct-full.json. Update inventory/coverage mappings when the
program requires them; a new file alone does not join the maintained campaign.
From the Evals repository root, adapt these provider-free checks to the case
and roster you changed. Run the reliability-plan validator only when that
separate plan changes:
python3 evals/paperclip-runner/tools/eval_program.py validate \
--case evals/paperclip-runner/cases/get-task-context.json \
--config evals/paperclip-runner/configs/live-codex-pinned.json
python3 evals/paperclip-runner/tools/run_live_roster.py validate \
--roster evals/paperclip-runner/rosters/live-mini.json --run-id validate-new-case
python3 evals/paperclip-runner/tools/run_live_campaign.py validate \
--campaign evals/paperclip-runner/campaigns/live-direct-full.json
python3 evals/paperclip-runner/tools/reliability_campaign.py validate \
--plan evals/paperclip-runner/campaigns/paperclip-runner-reliability.json
Use nearby positive and negative grader cases/fixtures to calibrate the new assertion, including malformed or missing evidence where the grader must fail closed. Preserve the existing machine disposition and grade; product, model/provider, grading, and infrastructure labels are analytical annotations, not instructions to rewrite classifiers.
Author one bounded case with a deterministic semantic assertion and an inspectable result. Declare its expected operation, state effect, provider lane/profile, timeout and retry policy, and any required evidence. Do not grade hidden reasoning, infer success from a provider terminal message, or invent conversation/tool evidence. Public output follows the reviewed projection: sanitized fixture conversation and allowlisted tool outcomes may be published; raw trusted artifacts, credentials, secrets, private references, and hidden reasoning may not.
Validate without provider calls first using the commands above and the relevant
report/render validation documented in the Runner docs. When a live run is
authorized, pin the Paperclip commit and exact 40-character
paperclip-evals commit, select the smallest useful roster, and retain the
complete provenance and cost record.
Update authoritative detailed docs when the contract or command changes, then
link from doc/evals.md rather than duplicating the Runner runbook. Keep public
reports immutable and use the reviewed projection; sanitized fixture
conversation and allowlisted tool outcomes may be public, while credentials,
secrets, private references, raw trusted payloads, and hidden reasoning must not
be exposed.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 1mo ago First seen · 73 lines · 30 tokens per session scan A 49c725736955
add-runner-eval is a skill published in the GitHub repository paperclipai/paperclip (107,044 stars, last pushed today), licensed MIT. It adds 30 tokens to every session and 895 once invoked, about $0.0001 per session on Opus 5.5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-09-17.
Other skills, from other repositories
deep-reasoning
Structured multi-step reasoning tool for complex problem solving with branching, hypothesis testing, and evidence tracking.
agent-benchmark-suite
Agent skill for benchmark-suite - invoke with $agent-benchmark-suite.
Verification & Quality Assurance
Comprehensive truth scoring, code quality verification, and automatic rollback system with 0.95 accuracy threshold for ensuring high-quality agent outputs and codebase reliability.
agent-production-validator
Agent skill for production-validator - invoke with $agent-production-validator.
agent-tester
Agent skill for tester - invoke with $agent-tester.
agent-implementer-sparc-coder
Agent skill for implementer-sparc-coder - invoke with $agent-implementer-sparc-coder.