Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx agentmods add skills/danielvm-git/bigpowers/run-evalsnpx skills add danielvm-git/bigpowers --skill run-evalsgit clone --depth 1 https://github.com/danielvm-git/bigpowersWhat it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5 | $0.00053 | $0.00823 |
| Opus 5 | $0.00026 | $0.00411 |
| Sonnet 5 | $0.00011 | $0.00165 |
| Haiku 4.5 | $0.00005 | $0.00082 |
Grade A, and why
run-evals scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 2d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 81 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Run Evals
HARD GATE — Define evals before implementation. Code graders = runnable
verify:commands; model graders = explicit rubric with pass/fail criteria.
Process
-
Name the capability under test (one sentence).
-
Write
specs/EVALS-<feature>.mdwith:- Capability evals (does it do the job?)
- Regression evals (did we break anything?)
-
Assign grader type per eval:
code(shell verify) ormodel(rubric). -
Assign strictness tier per eval (graduated promotion — e45s37):
Tier Meaning Promotion rule EXPERIMENTALNew eval, may flake Not gating USUALLY_PASSESStable in dev; ≥2/3 recent runs pass Blocks BUILD only when combined with ALWAYS_PASSES suite ALWAYS_PASSESZero tolerance; required for release Any single failure blocks BUILD and merge Promote:
EXPERIMENTAL → USUALLY_PASSESafter 3 consecutive passes;USUALLY_PASSES → ALWAYS_PASSESafter 5 consecutive passes with zero flakes documented inspecs/state.yaml. -
Run evals; log results table with pass@k (e.g. 3/3 runs) and tier per eval.
-
Block BUILD phase until all
ALWAYS_PASSESevals pass at agreed k.USUALLY_PASSESfailures warn;EXPERIMENTALfailures log only.
Artefact
specs/verifications/eNNsYY-eval-report.md — see REFERENCE.md for template. Eval reports are stored alongside verification evidence in specs/verifications/, keyed by story ID for traceability.
Verify
→ verify: test -d specs/benchmarks && test -f specs/benchmarks/SCHEMA.md
Run Evals — Reference
Strictness tiers (e45s37)
Add a tier: column to each eval row:
| Tier | Gate behaviour |
|---|---|
EXPERIMENTAL |
Log only — does not block |
USUALLY_PASSES |
Warn on failure; blocks only when paired with failing ALWAYS_PASSES |
ALWAYS_PASSES |
Hard block on any failure |
EVALS template
# EVALS: <feature>
## Capability
| ID | Eval | Grader | Tier | verify / rubric |
|----|------|--------|------|-----------------|
| C1 | ... | code | ALWAYS_PASSES | `verify: npm test -- <file>` |
| C2 | ... | model | USUALLY_PASSES | Rubric: [ ] criterion A [ ] criterion B |
## Regression
| ID | Eval | Grader | verify / rubric |
|----|------|--------|-----------------|
| R1 | Full suite passes | code | `verify: npm test` |
## Results
| Run | C1 | C2 | R1 | pass@k |
|-----|----|----|-----|--------|
| 1 | PASS | PASS | PASS | 3/3 |
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 2d ago First seen · 81 lines · 53 tokens per session scan A 000e88ab93a6
run-evals is a skill published in the GitHub repository danielvm-git/bigpowers (156 stars, last pushed 25d ago), licensed MIT. It adds 53 tokens to every session and 823 once invoked, about $0.0003 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other skills, from other repositories
shared/tech-stack-detection
检测项目技术栈的通用方法，通过分析配置文件识别语言、框架、工具链.
devops/changelog-generation
自动生成 CHANGELOG，基于 git 提交历史和 pipeline 产物信息，遵循 Conventional Commits 和 Keep a Changelog 规范.
boss
可审计的 agent 团队：BMAD 全自动研发流水线编排器。编排 9 个专业 Agent（PM、架构师、UI Designer、Tech Lead、Scrum Master、Frontend、Backend、QA、DevOps）从需求到部署，每一步都有事件溯源 + 不可绕过门禁 + 确定性 eval——可验证测试真跑、门禁真过。支持单环节切片命令（/boss:plan /review /qa /ship）与无 CLI 纯 Markdown 降级。 Triggers: 'boss mode', '/boss', '全自动开发', '从需求到部署', '帮我做一个', 'build this', 'ship it', '全流程'…
debloat
Compress an artifact that has accreted into bloat — padding, over-qualification, fused sentences, walls of enumeration, adjacent restatement — down to its load-bearing density, meaning preserved. Use when prose is correct and current but has grown verbose or patched-over and you want it tight without a full rewrite.
bmad-dev-story
Execute story implementation following a context filled story spec file. Use when the user says "dev this story [story file]" or "implement the next story in the sprint plan".
bmad-validate-prd
Validate a PRD against standards. Use when the user says "validate this PRD" or "run PRD validation".