Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
git clone --depth 1 https://github.com/sigistry/marketplaceWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/commands/sigistry/marketplace/eval-scaffold)<a href="https://agentmods.dev/commands/sigistry/marketplace/eval-scaffold"><img src="https://agentmods.dev/badge/commands/sigistry/marketplace/eval-scaffold.svg" alt="Measured on agentmods" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00024 | $0.00965 |
| Opus 5 | $0.00012 | $0.00483 |
| Sonnet 5 | $0.00005 | $0.00193 |
| Haiku 4.5 | $0.00002 | $0.00097 |
Grade A, and why
eval-scaffold scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 4d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 45 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Scaffold a runnable eval suite for the prompt, agent, or LLM call site at $ARGUMENTS (default: the app's prompt/agent files under prompts/, agents/, chains/, or the LLM call sites in the current diff). This escapes "vibes-based" development: after this command, a prompt change is graded by tests instead of by reading three outputs. It applies the eval-authoring skill and rides an existing eval framework where one is present rather than reinventing it.
Process
Step 1: Detect the SDK, runner, and any eval framework
Read manifests and imports to fix the stack before writing anything:
| Layer | Signal |
|---|---|
| Anthropic SDK | anthropic, @anthropic-ai/sdk, client.messages.create |
| OpenAI SDK | openai, OpenAI(, chat.completions.create, responses.create |
| LangChain | langchain, ChatOpenAI, ChatAnthropic, .invoke( |
| LlamaIndex | llama-index, VectorStoreIndex, query_engine |
| Vercel AI SDK | ai, generateText, generateObject, streamText |
| Pydantic AI | pydantic-ai, Agent(, result_type= |
| Test runner | pytest/pyproject.toml, vitest/jest/package.json, go test |
| Eval framework | promptfooconfig.yaml, promptfoo, deepeval, .deepeval, ragas |
If promptfoo or DeepEval is already present, generate configs/tests for it (see the skill's references/eval-frameworks.md). If neither is present, generate framework-free tests on the detected unit runner (Vitest/pytest) plus a small LLM-judge helper.
Step 2: Identify what to evaluate
Read the target prompt/agent and name its contract: the inputs it takes, the output shape it promises, and the failure modes worth catching (wrong format, missing field, hallucinated fact, ignored instruction, unsafe content, verbosity). Pull 3–6 representative cases from fixtures, logs, or the prompt's own examples, never invent domain facts.
Step 3: Generate the suite (apply the eval-authoring skill)
Produce four artifacts:
- Assertion tests: deterministic checks per case:
equals/contains/icontains,regex,is-json+json-schema,not-contains(banned strings), latency/cost budget. Cheap, fast, run every commit. - LLM-as-judge tests: for qualities assertions can't capture (faithfulness, helpfulness, tone). Use a rubric from
references/llm-judge-rubrics.md: explicit criteria, a fixed scale, and a pass threshold. Prefer pairwise (new vs. baseline) for regression gates. - Golden-dataset stub: a versioned
casesfile (jsonl/yaml) withinput,expected/assert, and atagsfield, plus a short note on how to grow it from production traces. - CI job: a GitHub Actions workflow that runs the suite on PRs and fails the check on a quality regression (assertion failure, or judge score below threshold / below the baseline).
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 4d ago First seen · 45 lines · 24 tokens per session scan A 6f404549b39e
eval-scaffold is a command published in the GitHub repository sigistry/marketplace (3 stars, last pushed 4d ago), licensed MIT. It adds 24 tokens to every session and 965 once invoked, about $0.0001 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-09-03.
Other commands, from other repositories
atdd
Start the ATDD workflow for a new feature. Optionally provide a feature description as an argument.
loop
Autonomous execution loop. Runs the TDD cycle repeatedly until all tasks in an epic are complete, with circuit breaker protection and PR workflow.
execute
Execute the next available tracked task with TDD, pre-commit validation, PR creation, AI code review, and issue tracker bridge sync.
prompt-generator
Generate effective prompts for Claude 4.5 Sonnet to achieve user-defined outcomes.
partition-dev
Derive test cases from Z specification using TTF testing tactics.
ccc-feature-start
Kickoff a new feature with branch creation, TDD setup, and task breakdown.