Getting it into your agent
This one installs as part of its plugin. Adding the marketplace and installing the plugin brings it with everything else the plugin ships.
/plugin marketplace add build-with-dhiraj/ai-workflow-framework-portability-kit/plugin install vercel-pluginWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/build-with-dhiraj/ai-workflow-framework-portability-kit/benchmark-agents)<a href="https://agentmods.dev/skills/build-with-dhiraj/ai-workflow-framework-portability-kit/benchmark-agents"><img src="https://agentmods.dev/badge/skills/build-with-dhiraj/ai-workflow-framework-portability-kit/benchmark-agents/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/build-with-dhiraj/ai-workflow-framework-portability-kit/benchmark-agents"><img src="https://agentmods.dev/badge/skills/build-with-dhiraj/ai-workflow-framework-portability-kit/benchmark-agents.svg" alt="Reviewed on agentmods" width="80" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00058 | $0.03904 |
| Opus 5 | $0.00029 | $0.01952 |
| Sonnet 5 | $0.00012 | $0.00781 |
| Haiku 4.5 | $0.00006 | $0.00390 |
Grade B, and why
benchmark-agents scanned grade B with 2 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 9d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Reads agent configuration directorieslowAgent snooping
.claude/, .codex/, .gemini/ hold keys, settings and other credentials a mod has no legitimate need for.
find ~/.claude/debug -name "*.txt" -mmin -2 -exec grep -l "$SLUG" {} + Downgraded: this mod is about security review, or the phrase is quoted, so it is likely naming the pattern rather than instructing it.
Recursive force deletemediumDestructive command
rm -rf with a variable or a broad path is one typo away from removing the wrong tree.
rm -rf ~/dev/vercel-plugin-testing Downgraded: this mod is about security review, or the phrase is quoted, so it is likely naming the pattern rather than instructing it.
How it starts
The opening of the file, as written. The whole thing — 311 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Benchmark Agents — Advanced AI Systems
Launch real Claude Code sessions with the plugin installed, verify skill injection, monitor PostToolUse validation catches, and produce a coverage report. This skill covers the full eval loop: setup → launch → monitor → verify → fix → release → repeat.
How Evals Work (The Only Correct Method)
Evals are run by you, in this conversation, not by scripts. The process is:
- You create directories and install the plugin via Bash tool calls
- You spawn WezTerm panes with
wezterm cli spawn— each pane runs an independent Claude Code interactive session - You wait, then check debug logs and claim dirs to see what the plugin injected
- You inspect the generated source code for correctness
- You read conversation logs to find what the user had to correct
- You update skills/hooks, run
/release, and spawn more evals
Never use claude --print, eval scripts, or Bun.spawn(["claude", ...]). These do not work because:
- Plugin hooks (PreToolUse, PostToolUse, UserPromptSubmit) only fire during interactive tool-calling sessions
--printmode generates text without executing tools — no files are created, no deps installed, no dev servers started- No
session_idmeans dedup, profiler, and claim files don't work
The WezTerm interactive approach is the only method that exercises the plugin correctly. Every eval in our history (60+ sessions) used this approach.
DO NOT (Hard Rules)
These are absolute prohibitions. Violating any of them wastes the entire eval run:
- DO NOT use
claude --printor-pflag — hooks don't fire, no files created - DO NOT use
--dangerously-skip-permissions— changes agent behavior - DO NOT create projects in
/tmp/— always use~/dev/vercel-plugin-testing/ - DO NOT manually create
settings.local.jsonor wire hooks by hand — usenpx add-plugin - DO NOT set
CLAUDE_PLUGIN_ROOTmanually — the plugin manages this - DO NOT use
bash -corbash -lcin WezTerm — always use/bin/zsh -ic - DO NOT use the full path to claude — use the
xalias (it's configured in zsh) - DO NOT create custom
debug.logfiles with stderr redirects — debug logs go to~/.claude/debug/ - DO NOT write eval runner scripts in TypeScript/JavaScript — do everything as Bash tool calls in the conversation
- DO NOT try to
git initor createpackage.jsonmanually —npx add-plugin+ the WezTerm session handle all scaffolding - DO NOT use uppercase letters in directory names — npm rejects them (e.g.
Tin timestamps breakscreate-next-app)
What ships with it
1 file beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 9d ago First seen · 311 lines · 58 tokens per session scan B 95fd9d33be1b
benchmark-agents is a skill published in the GitHub repository build-with-dhiraj/ai-workflow-framework-portability-kit (4 stars, last pushed 27d ago), licensed MIT. It adds 58 tokens to every session and 3,904 once invoked, about $0.0003 per session on Opus 5. A static security scan graded it B with 2 findings (reads agent configuration directories, recursive force delete). No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.
Other skills, from other repositories
semgrep-rule-variant-creator
Creates language variants of existing Semgrep rules. Use when porting a Semgrep rule to specified target languages. Takes an existing rule and target languages as input, produces independent rule+test directories for each language.
forward-derivation
Derive BDD scenarios and TDD test skeletons from approved SDD specifications. ATDD acceptance test tables are optional output for specialized needs. Use when: spec is approved, starting BDD/TDD implementation, generating test structures. Keywords: forward derivation, spec to test, BDD generation, TDD skeleton, test…
atdd-assistant
A guide to Acceptance Test-Driven Development, or ATDD, a process where a team agrees on user-story requirements and acceptance tests before coding. Acceptance tests check whether a feature meets the agreed business requirements.
testing-guide
A testing guide covering common testing levels, including unit, integration, system, and end-to-end testing. It also supports ISTQB and industry testing-pyramid approaches.
bdd-assistant
A guide to Behavior-Driven Development, or BDD, a way to describe software behavior with examples in plain language before building it. It uses Given-When-Then scenarios and Gherkin, a structured format for those examples.
methodology-system
Manage and guide developers through active development methodology workflows. Use when: TDD, BDD, SDD, ATDD, or custom methodology workflows are needed. Keywords: methodology, workflow, TDD, BDD, SDD, ATDD, phase, checkpoint, development process.