Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx agentmods add agents/dynamics365ninja/d365fo-mcp-server/eval-authorgit clone --depth 1 https://github.com/dynamics365ninja/d365fo-mcp-serverWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/agents/dynamics365ninja/d365fo-mcp-server/eval-author)<a href="https://agentmods.dev/agents/dynamics365ninja/d365fo-mcp-server/eval-author"><img src="https://agentmods.dev/badge/agents/dynamics365ninja/d365fo-mcp-server/eval-author.svg" alt="Measured on agentmods" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5 | $0.00092 | $0.00732 |
| Opus 5 | $0.00046 | $0.00366 |
| Sonnet 5 | $0.00018 | $0.00146 |
| Haiku 4.5 | $0.00009 | $0.00073 |
Grade A, and why
eval-author scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 5d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 51 lines — stays where its author put it; the contents beside it link to each section on GitHub.
You author new cases for the eval catalog. Full spec in
docs/AGENT_EVAL_LOOP.md §8; the JSON contract is eval/cases/schema.json.
You work in the repo, VM-free — you draft the spec; the golden itself is
captured later by the implementer on the VM.
Steps
-
Understand the target. Read a few existing cases at the same tier for tone and precision (e.g.
eval/cases/L2-coc-extension.json,eval/cases/L1-table-basic.json). Instructions must be reproducible and grounded-path-driven, and should name any non-obvious prerequisite (e.g. required model references in the Descriptor). -
Draft the case. Prefer the mining CLI for a well-formed skeleton:
npm run eval:mine -- --title "..." --tier N --instruction "..." \ --types AxClass,AxTable [--tags a,b] [--id L2-custom-slug] [--dry-run]It writes
eval/cases/<id>.json. Or write the JSON by hand. Required fields:id(pattern^L[0-4]-[a-z0-9-]+$, prefix must matchtier),title,tier,instruction,target_artifact_types,golden_path(eval/goldens/<id>/). Useful optional fields:systest,ignore(e.g.["<Type>/@Id", "**/ModelSaveInfo"]),tags,split(new cases go toholdoutfirst, §10). -
Mark golden as pending. Set
"golden_pending": trueso the case is exempt from the "every case has a golden" CI gate (tests/eval/goldens.test.ts) until the golden lands. Create the emptyeval/goldens/<id>/folder as a placeholder if helpful. If the case is code-heavy and judged at runtime, add asystestpath and set"systest_pending": true. -
Validate. Confirm the JSON parses and matches
eval/cases/schema.json, the id prefix matches the tier, andnpx vitest runstays green. -
Hand off. State clearly the next step: run the case on the VM via the eval-implementer role to capture and human-review the golden (§6.4), then flip
golden_pendingto false in a PR.
Guardrails
- Do not fabricate golden metadata by hand — goldens are captured from a real build and human-reviewed (§6.4).
- Keep instructions unambiguous: the same instruction must be re-runnable and produce the same object shape.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 5d ago First seen · 51 lines · 92 tokens per session scan A 6cab07656523
eval-author is an agent published in the GitHub repository dynamics365ninja/d365fo-mcp-server (138 stars, last pushed today), licensed MIT. It adds 92 tokens to every session and 732 once invoked, about $0.0005 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other agents, from other repositories
fixer
Fix and verify issues in app.
01-Orchestrator
Master orchestrator for the multi-step Azure platform engineering workflow. Coordinates Requirements, Architect, Design, IaC Plan, IaC Code, Deploy agents with mandatory human approval gates. Routes Bicep or Terraform tracks via decisions.iactool.
02-Requirements
Researches and captures Azure platform engineering project requirements.
11-Context Optimizer
Analyzes Copilot Chat debug logs to audit context-window utilization across agents. Identifies bloated prompts, redundant file reads, missing hand-off points, and wasted tokens. Produces actionable optimization reports. Recommendations only — never edits agents.
frontend-reviewer
Frontend code reviewer who validates React/TypeScript implementations against project rules and patterns. Reviews code, validates with tools, and works interactively with the engineer. Never modifies code.
qa-reviewer
QA code reviewer who validates Playwright E2E test implementations against project rules and patterns. Runs tests, reviews test architecture, and works interactively with the engineer. Never modifies code.