Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add richfrem/agent-plugins-skills --skill improving-skills-in-labgit clone --depth 1 https://github.com/richfrem/agent-plugins-skillsWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/richfrem/agent-plugins-skills/improving-skills-in-lab)<a href="https://agentmods.dev/skills/richfrem/agent-plugins-skills/improving-skills-in-lab"><img src="https://agentmods.dev/badge/skills/richfrem/agent-plugins-skills/improving-skills-in-lab/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/richfrem/agent-plugins-skills/improving-skills-in-lab"><img src="https://agentmods.dev/badge/skills/richfrem/agent-plugins-skills/improving-skills-in-lab.svg" alt="Reviewed on agentmods" width="80" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00235 | $0.01911 |
| Opus 5.5 | $0.00094 | $0.00764 |
| Sonnet 5.5 | $0.00047 | $0.00382 |
| Haiku 4.5 | $0.00023 | $0.00191 |
Grade A, and why
improving-skills-in-lab scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured yesterday.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 86 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Improving skills in the lab
One conversation takes a skill or agent from a plain-language goal to a scored, backported improvement. Each mode below was a separate skill; its full workflow is a reference loaded when that mode starts.
Contents
Critical constraints
- One question at a time. Read the whole message first, never re-ask, and offer a default with each choice.
- Stay read-only until the user confirms the summary (the interview, step 4).
- The scorer is locked. Every change gets exit 0 (KEEP) or 1 (DISCARD, auto-reverted); never edit
evals/evals.jsonor the scorer during a run. - Never record an approval for the user. Present the command and let them run it.
- Lab work stays in the isolated lab repo. Master sources change only through the backport mode, and only after checking the target skill is installed.
Quick start
Keep the user's own wording. Ask in this order, skipping what is known, then confirm before writing anything:
- What to improve and why, and whether it exists or is new.
- Read the target's source and look for overlap (skills, agents, earlier runs, the target skill's
program.md,evals/results.tsv); report what was found. - Success definition, depth (about 10, 50 or 100+ runs; recommend 50), available tools (
which gh agy claude), second-opinion check, known problems. - Restate and ask "Does this look right?". Then write the target skill's
program.md,improvement/run-config.jsonandimprovement/session-brief.md, asking before overwriting. Resume fromimprovement/run-config.jsonwhen it exists.
Full conversation script: interview.md.
Workflow
| Mode | Use when | Run | Workflow |
|---|---|---|---|
plan |
Planning an improvement, writing a delegation prompt, auditing what already exists, deciding update vs create, a new pattern to generalize | agent work, no script | planner |
setup |
An isolated lab repo must be built and baselined | scripts/generate_eval_instructions.py, scripts/init_autoresearch.py |
setup |
loop |
Running the improvement cycle (orient, mutate once, score, close); solo or ORCHESTRATOR with INNER_AGENT and PEER_AGENT; fan-out, event bus, claim_task, triple-loop; closing the cycle | solo, or ORCHESTRATOR with INNER_AGENT and PEER_AGENT | loop |
score |
Evaluating a skill; one change needs KEEP or DISCARD; skill routing or description is wrong; RED-GREEN-REFACTOR; measuring a gain; running tests on a skill | scripts/evaluate.py, scripts/eval_runner.py |
scorer |
log |
Persist a run, query past runs, render the trend chart | scripts/experiment_log.py, scripts/generate_report.py, scripts/plot_eval_progress.py |
log |
backport |
A completed lab run's diff must reach the master sources | agent review, ACCEPT, ADAPT, REJECT or REVIEW per file | backport |
memory |
Closing a session; "remember this", "update memory", session log, capture learnings; deciding what to record or promote | installed context/kernel.py |
memory |
What ships with it
60 files beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.
- assets/diagrams/backport-phases.mmd 5.7 KB
- assets/diagrams/backport-sequence.mmd 3.7 KB
- assets/diagrams/sibling-repo-labs.mmd 865 B
- assets/templates/autoresearch/copilot_proposer_prompt.md.template 2.9 KB
- assets/templates/autoresearch/evals.json.template 496 B
- assets/templates/autoresearch/program.md.template 1.7 KB
- assets/templates/autoresearch/results.tsv.template 73 B
- assets/templates/eval-instructions.template.md 22 KB
- evals/evals.json 20 KB
- evals/results.tsv 1.9 KB
- references/acceptance-criteria.md 3.9 KB
- references/audit-lenses.md 997 B
- references/backport-detailed-reference.md 8.1 KB
- references/chart-reading-guide.md 2.6 KB
- references/cheapest_models.md 1.1 KB
- references/check-protocol.md 2.3 KB
- references/copilot_proposer_prompt.md 2.9 KB
- references/delegation.md 914 B
- references/domain-patterns-routing-skill.md 2.8 KB
- references/domain-patterns.md 1.8 KB
- references/fallback-tree.md 2.4 KB
- references/improvement-ledger-spec.md 12 KB
- references/interview.md 6.2 KB
- references/karpathy-autoresearch-3-file-eval.md 22 KB
- references/lab-limits.md 5.5 KB
- references/lab-space-protocol.md 4.7 KB
- references/log-detailed-reference.md 2.8 KB
- references/loop-metrics.md 1.2 KB
- references/memory-detailed-reference.md 9.1 KB
- references/memory-promotion-guide.md 3.0 KB
- references/meta-harness-lee-2026.md 3.8 KB
- references/modes/backport.md 2.7 KB
- references/modes/log.md 4.6 KB
- references/modes/loop.md 5.2 KB
- references/modes/memory.md 2.3 KB
- references/modes/plan.md 5.6 KB
- references/modes/score.md 4.7 KB
- references/modes/setup.md 2.8 KB
- references/mutation-protocol.md 2.8 KB
- references/operating-protocols.md 2.3 KB
- references/optimizer-engine-patterns.md 5.4 KB
- references/orchestrator-meta-survey.md 950 B
- references/overfitting-gate.md 1.1 KB
- references/phase-guide.md 5.4 KB
- references/plan-templates.md 1.5 KB
- references/post_run_survey.md 1.5 KB
- references/program-example.md 750 B
- references/red-green-refactor.md 6.3 KB
- references/report-reference.md 4.8 KB
- references/routing-taxonomy.md 1006 B
- references/score-troubleshooting.md 3.3 KB
- references/setup-detailed-reference.md 10 KB
- references/stage-0-orientation.md 3.3 KB
- references/stage-1-execution.md 1.4 KB
- references/stage-2-verification.md 1.1 KB
- references/stage-3-decision.md 585 B
- references/stage-4-close.md 13 KB
- references/test-registry-protocol.md 6.2 KB
- references/test-scenarios-seed.md 40 KB
- references/triple-loop.md 2.4 KB
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- yesterday First seen · 86 lines · 235 tokens per session scan A c46a4e67908f
improving-skills-in-lab is a skill published in the GitHub repository richfrem/agent-plugins-skills (7 stars, last pushed yesterday), licensed MIT. It adds 235 tokens to every session and 1,911 once invoked, about $0.0009 per session on Opus 5.5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-10-25.
Other skills, from other repositories
qa
Run scalable, isolated live QA for nac development. The top-level local orchestrator must parse n (default 4), dispatch one setup worker with this skill, copy its n assignment contracts verbatim into exactly n parallel test workers with this skill, then dispatch one aggregate worker with this skill using all test…
ap-independent-checker
Independently review the exact result and run its real checks in one context, using isolated resources and without changing the deliverable.
ap-execharness-resolver
Report the compatibility redirect to ap-worker; this retired role cannot perform new work.
ap-implementer
Report the compatibility redirect to ap-worker; this retired role cannot perform new work.
ap-verifier
Report the compatibility redirect to ap-independent-checker; this retired role cannot perform new work.
feature-loop
Coding agent for the project-level harness — one feature per session, must verify, must be mergeable.