Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add GiustoPiedimonte/agentic-engineering-marketplace --skill evalgit clone --depth 1 https://github.com/GiustoPiedimonte/agentic-engineering-marketplaceWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/giustopiedimonte/agentic-engineering-marketplace/eval)<a href="https://agentmods.dev/skills/giustopiedimonte/agentic-engineering-marketplace/eval"><img src="https://agentmods.dev/badge/skills/giustopiedimonte/agentic-engineering-marketplace/eval.svg" alt="Measured on agentmods" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00091 | $0.00810 |
| Opus 5 | $0.00046 | $0.00405 |
| Sonnet 5 | $0.00018 | $0.00162 |
| Haiku 4.5 | $0.00009 | $0.00081 |
Grade A, and why
eval scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 8d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 65 lines — stays where its author put it; the contents beside it link to each section on GitHub.
/eval — make eval the unit of progress
You don't improve what you don't measure, and you don't measure with invented
test cases. Evals start from real failures, get coded into failure modes,
and become the harness that gates change. $ARGUMENTS is the feature / agent /
flow to evaluate (or a pointer to a trace dump).
This skill is offline and diagnostic — it builds and runs the harness over
traces/datasets and tells you where and how things break. Its live sibling
is /measure (a one-shot read-only verdict on a gated decision); /eval is what
defines what "good" even means, and produces the criteria /measure later checks.
Process
-
Start from real traces, never synthetic. Pull actual failing runs (logs, recorded sessions, the dev DB). Delegate the wide read to the
researchersubagent and read-only data pulls to themeasurer. Synthetic cases validate the mechanism, not the magnitude — they cannot tell you what really breaks. -
Error-analysis (open-code → axial-code). Read a sample of failures and write a free-text note on what went wrong for each. Then cluster those notes into a small set of named failure modes (e.g. wrong-tool-selected, argument-extraction-error, voice/format-drift, hallucinated-fact). Count them — frequency ranks what to fix first. Stop sampling when new traces stop producing new modes (theoretical saturation), and say how many you read.
-
Localize with a transition-failure matrix (for multi-step / agentic flows). For each step boundary, record where the failure first appears, so a regression is pinned to a stage (tool-selection vs extraction vs formatting) rather than to a vague pass-rate. See
references/EVAL_PLAYBOOK.md. -
Pick the assertion level per mode — the cheapest that's faithful:
- Deterministic / component-level (code asserts): exact tool chosen, schema valid, value in range. Prefer these — fast, free, no judge drift.
- LLM-judge / task-level only where judgment is irreducible (voice, helpfulness, end-to-end success). Align the judge to human labels on a held-out set before trusting it; report its agreement rate.
What ships with it
1 file beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 8d ago First seen · 65 lines · 91 tokens per session scan A 0af868cf0d85
eval is a skill published in the GitHub repository GiustoPiedimonte/agentic-engineering-marketplace (13 stars, last pushed 1mo ago), licensed MIT. It adds 91 tokens to every session and 810 once invoked, about $0.0005 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other skills, from other repositories
test-run
Run plugin test suites in this monorepo and report a concise pass/fail summary. Optional plugin slug arg; without arg, runs all plugins under plugins/.
implement
Implement a ticket or feature with proportional planning, test first behavior changes, independent review when available, and verification through the real affected surface.
symfony:doctrine-fixtures-foundry
Create test data with Zenstruck Foundry v2 factories (PersistentObjectFactory, real objects); define states, sequences, and relationships.
symfony:e2e-panther-playwright
Write end-to-end tests with Symfony Panther 2.4 for browser automation or Playwright for complex scenarios.
symfony:functional-tests
Write functional tests for Symfony controllers and HTTP endpoints using WebTestCase, getContainer, loginUser, and DAMA rollback.
symfony:tdd-with-pest
Apply RED-GREEN-REFACTOR with Pest v4 (PHP 8.3+) for Symfony via the PHPUnit bridge; Foundry factories, WebTestCase, verify failures first.