What is agent-eval-coverage?

An evaluation and test-coverage review for AI or software-agent repositories. It checks whether changes can be tested for correct behavior, tool use, safety, and unintended regressions.

Use it to inspect regression datasets, prompt tests, behavior and tool-use tests, hallucination and safety checks, CI release gates, and evaluation metrics.

AI systems can change behavior after a prompt, model, or tool change, even when ordinary tests pass. This review shows whether the repository would detect those changes before release.

agent-eval-coverage — Skill for Claude Code by vikast908

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

Any agent

npx skills add vikast908/agent-repo-card --skill agent-eval-coverage

Clone the repo

git clone --depth 1 https://github.com/vikast908/agent-repo-card

Made for: Claude Code.

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

README.md

[![agentmods](https://agentmods.dev/badge/skills/vikast908/agent-repo-card/agent-eval-coverage/github.svg)](https://agentmods.dev/skills/vikast908/agent-repo-card/agent-eval-coverage)

Your own site

<a href="https://agentmods.dev/skills/vikast908/agent-repo-card/agent-eval-coverage"><img src="https://agentmods.dev/badge/skills/vikast908/agent-repo-card/agent-eval-coverage/github.svg" alt="Measured on agentmods" height="20"></a>

Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.

Your own site · 80×15

<a href="https://agentmods.dev/skills/vikast908/agent-repo-card/agent-eval-coverage"><img src="https://agentmods.dev/badge/skills/vikast908/agent-repo-card/agent-eval-coverage.svg" alt="Reviewed on agentmods" width="80" height="20"></a>

Per session 107 Skills are progressive disclosure: only the name and description are preloaded; the body loads when the skill is used.

When invoked 1,263 The whole file, excluding the scripts and references it only reads on demand.

Security scan A 0 findings. A grade says what 26 rules found in the file — not that it is safe.

Origin original No closer match found in the catalogue.

Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

Model	Per session	Once invoked
Fable 5.1	$0.00107	$0.01263
Opus 5.5	$0.00043	$0.00505
Sonnet 5.5	$0.00021	$0.00253
Haiku 4.5	$0.00011	$0.00126

Measured 2mo ago against content hash bcbc787345c1, method: parsed. Prices are Anthropic first-party input rates as of 2026-10-07, from the pricing page.

Security

Grade A, and why

agent-eval-coverage scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 2mo ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

skills/agent-eval-coverage/SKILL.md · 63 lines

How it starts

The opening of the file, as written. The whole thing — 63 lines — stays where its author put it; the contents beside it link to each section on GitHub.

Agent evaluation & test-coverage review

You are an ML/eval engineer who has built evaluation harnesses for LLM and agent products. You know the core risk: LLM apps change behavior silently — a prompt tweak, a model upgrade, a new tool — and without evals nobody notices until users do. You review this repo for whether the team would actually catch a regression before shipping it.

Protocol (shared across all checks)

Plan first (default). Present a short plan: what test/eval assets you'll look for, the coverage gaps you'll assess, the outputs, and assumptions/missing info. Ask "Proceed with the full eval-coverage review, or adjust scope?" and wait. Skip if invoked with auto / "just do it".
Evidence rule. Cite file:line / file paths for tests and eval assets. Don't credit evals that don't exist; if you can't find a suite, say so plainly. Label guesses unverified.
Severity: Critical / High / Medium / Low.
Score dimensions below to 0–100 → grade.
Output inline, then offer to save to agent-review/agent-eval-coverage.md.

What to inspect

Test presence at all: test/, tests/, __tests__/, *.test.*, *.spec.*, eval/evals/evaluation dirs, notebooks. Identify the test runner and how tests run.
Eval datasets: golden sets, fixtures, cases/examples/dataset/*.jsonl of input→expected. Are they versioned? How big? How representative?
Prompt regression: are prompts/templates covered by tests that catch behavior change? Snapshot tests of prompt-rendered output? Search: prompt, snapshot, __snapshots__.
LLM-as-judge / scoring: automated grading of open-ended output (rubric, judge model, similarity, assertions). Search: judge, score, rubric, assert, expect, eval.
Behavioral & tool-use tests: does the agent call the right tool with the right args? End-to-end task success? Multi-step trajectories? Mocked tools?
Safety/quality checks: hallucination, refusal, prompt-injection resistance, format/schema validity, regression on known bad cases.
Metrics & reporting: is success/accuracy/cost/latency measured and tracked over time, or is "it looked fine" the bar?
CI gating: do evals/tests run in CI and block merges? Search: .github/workflows, ci, pipeline config; look for the eval/test step and any pass thresholds.

Read the full file on GitHub · 63 lines

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

2mo ago First seen · 63 lines · 107 tokens per session scan A bcbc787345c1

Subscribe to this mod's changes

agent-eval-coverage is a skill published in the GitHub repository vikast908/agent-repo-card (1 stars, last pushed 4mo ago), licensed MIT. It adds 107 tokens to every session and 1,263 once invoked, about $0.0004 per session on Opus 5.5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.

Building agents Claude Code Testing agent-skills ai-agents ai-evaluation claude-skills code-quality

agent-eval-coverage next to 6 skills from other repositories
Mod	Type	Stars	Last push	Licence	Scan	Tokens	Works with
agent-eval-coveragevikast908/agent-repo-card	Skill	1	4mo ago	MIT	A	107	Claude Code
nasde-benchmark-runnerNoesisVision/nasde-toolkit	Skill	14	4d ago	MIT	A	139	Claude Code
tactical-dddNoesisVision/nasde-toolkit	Skill	14	4d ago	MIT	A	70	Claude Code, Codex
nasde-benchmark-calibrationNoesisVision/nasde-toolkit	Skill	14	4d ago	MIT	A	171	Claude Code
nasde-devNoesisVision/nasde-toolkit	Skill	14	4d ago	MIT	A	103	Claude Code
python-best-practicesNoesisVision/nasde-toolkit	Skill	14	4d ago	MIT	A	36	Claude Code, Codex
plumb-line-auditslopstopper/plumb-line	Skill	4	6d ago	Apache-2.0	A	0	Claude Code