agentmods.dev

d365fo-mcp-server: Agent for Claude Code

.claude/agents/eval-improver.md

eval-improver is an agent for Claude Code from dynamics365ninja/d365fo-mcp-server. It costs 111 tokens per session (886 once invoked), scanned A, original, MIT.

An agent role for improving a D365FO coding-agent evaluation system by studying recorded test runs. It works in the repository and does not use the virtual machine or run platform builds.

In plain words
What is it for?
Use it to report pass rates, group failures into clusters, create a fix brief, reproduce one actionable problem with a repository test, and prepare a pull request with evidence.
Why use it?
It helps separate repeated tool, knowledge, and validation problems from model mistakes or temporary environment failures. It uses those patterns to guide a focused fix and checks the result on held-out tests.

Agent for Claude Code

Written for Claude Code: installed under .claude/. Also seen: model in frontmatter.

This is dynamics365ninja/d365fo-mcp-server's own configuration. It tells Claude Code how to work on d365fo-mcp-server itself, so it is not a mod to install elsewhere. Copy it as a starting point and replace the rules that are about this project. Everything d365fo-mcp-server configures →

not rated 165repo +2 2d ago A scan 111 tokens original MIT
View source ↗ dynamics365ninja/d365fo-mcp-server
Reuse

Borrowing it

Nothing to install: this file belongs to dynamics365ninja/d365fo-mcp-server. Take a copy, put it at the same path in your own repository, and replace the rules that are about this project with yours.

Copy the file
curl -O https://raw.githubusercontent.com/dynamics365ninja/d365fo-mcp-server/main/.claude/agents/eval-improver.md
Clone the repo
git clone --depth 1 https://github.com/dynamics365ninja/d365fo-mcp-server

Made for: Claude Code.

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

agentmods badge for eval-improver

README.md
[![agentmods](https://agentmods.dev/badge/agents/dynamics365ninja/d365fo-mcp-server/eval-improver/github.svg)](https://agentmods.dev/agents/dynamics365ninja/d365fo-mcp-server/eval-improver)
Your own site
<a href="https://agentmods.dev/agents/dynamics365ninja/d365fo-mcp-server/eval-improver"><img src="https://agentmods.dev/badge/agents/dynamics365ninja/d365fo-mcp-server/eval-improver/github.svg" alt="Measured on agentmods" height="20"></a>

Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.

agentmods 80×15 button for eval-improver

Your own site · 80×15
<a href="https://agentmods.dev/agents/dynamics365ninja/d365fo-mcp-server/eval-improver"><img src="https://agentmods.dev/badge/agents/dynamics365ninja/d365fo-mcp-server/eval-improver.svg" alt="Reviewed on agentmods" width="80" height="20"></a>
Per session 111 Only the description is in the session, so the agent can decide to use it. The body loads when it is invoked.
When invoked 886 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 0 findings. A grade says what 26 rules found in the file — not that it is safe.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5.1 $0.00111 $0.00886
Opus 5.5 $0.00044 $0.00354
Sonnet 5.5 $0.00022 $0.00177
Haiku 4.5 $0.00011 $0.00089

Measured 2mo ago against content hash 1985891d7041, method: parsed. Prices are Anthropic first-party input rates as of 2026-10-07, from the pricing page.

Security

Grade A, and why

eval-improver scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 2mo ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

.claude/agents/eval-improver.md · 45 lines

How it starts

The opening of the file, as written. The whole thing — 45 lines — stays where its author put it; the contents beside it link to each section on GitHub.

You are the improver agent of the self-improving D365FO agent eval loop. The full design is in docs/AGENT_EVAL_LOOP.md (read §9 rubric and §10 improver workflow before acting). You run in the repo and communicate with the implementer only through the corpus — never touch the VM, never run platform builds.

Your job (one actionable cluster per invocation, unless told otherwise)

  1. Survey the corpus. Run these read-only, VM-free tools:

    • npm run eval:report — per-tier pass-rates + headline tool-defect rate.
    • npm run eval:clusters — actionable clusters ranked by frequency × tier_weight.
    • npm run eval:brief — the top-priority cluster rendered as a Markdown fix brief (--all for every cluster, --out file.md to save).
    • npm run eval:flakes — separate ENV_FLAKE noise from real defects.
    • Corpus records live in eval/corpus/runs/*.json (gitignored, VM-produced). If the directory is empty on this machine, say so — there is nothing to improve without evidence; do not invent failures.

  2. Pick the top actionable cluster (classification ∈ {TOOL_DEFECT, KNOWLEDGE_GAP, VALIDATOR_GAP}). MODEL_ERROR and ENV_FLAKE are not fixes — at most a prompt/instruction tweak; do not open code PRs for them.

  3. Confirm the classification. Re-derive it from the record's evidence_refs and tool output. You must be able to reproduce it deterministically in the repo without the VM. If you cannot reproduce it, downgrade to MODEL_ERROR and stop.

  4. Reproduce as a minimal repo test that fails on main — a new golden/unit/oracle test under tests/ (or a new/updated golden under eval/goldens/). This is the regression proof.

  5. Fix the real cause in one place:

    • TOOL_DEFECT → the TypeScript tool (src/tools/…, generators like src/…/generateSmartReport.ts).
    • KNOWLEDGE_GAP → the knowledge base (use npm run eval:knowledge for MODEL_ERROR→KB proposals as a starting point).
    • VALIDATOR_GAP → the validator rule (validate_code path).

  6. Validate — anti-overfitting is mandatory (§10).

    • npx vitest run — full suite must stay green (includes tests/eval/goldens.test.ts golden-integrity gate).
    • npm run eval:report — confirm the fix does not regress the held-out split. Never validate only on the failing case.

  7. Open a PR (branch off main; the repo has no origin — push/PR via remote d365fo-mcp-server). The PR body must link the corpus evidence_refs, the new repro test, and before/after scorecards. Do not auto-merge — humans review.

Read the full file on GitHub · 45 lines

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 2mo ago First seen · 45 lines · 111 tokens per session scan A 1985891d7041

Subscribe to this mod's changes

eval-improver is an agent published in the GitHub repository dynamics365ninja/d365fo-mcp-server (165 stars, last pushed 2d ago), licensed MIT. It adds 111 tokens to every session and 886 once invoked, about $0.0004 per session on Opus 5.5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.

Building agentsTestingazurecode-completiond365fodynamics-365erpmcp
Related

Other agents, from other repositories

implementation

Implements, tests, and documents scoped GPT-RAG changes after requirements are clear. Do not use to decide broad architecture or publish releases.

Azure/GPT-RAG · 30 tokens

kingdee-qa-engineer

QA & Test Engineer for the kingdee-mcp project. Authors evals/ and tests/ cases, reproduces bugs against the live K3Cloud environment, and runs regression scans via bin/kmcp test.

WaHaiLong/KingdeeMCP · 49 tokens

FAI Collective Tester

Multi-agent tester — generates unit/integration/E2E tests, AI evaluation pipelines, mutation testing, and quality assurance for AI outputs with deterministic seed-based testing.

frootai/frootai · 37 tokens

FAI Deterministic Expert

Deterministic AI specialist — makes AI outputs reproducible, grounded, and auditable with temperature control, seed pinning, JSON schema output, RAG grounding, citation enforcement, and multi-layer hallucination defense.

frootai/frootai · 50 tokens

ci-checker

Audits each project's test targets, local hooks, and pipeline definitions against the adopted gate standards and returns rated findings, without modifying anything.

wahidyankf/ose-public · 31 tokens

ci-fixer

Applies a CI checker's findings to test targets, hooks, and pipeline definitions after re-validating each one, and records what it fixed, disproved, and left for a person.

wahidyankf/ose-public · 42 tokens