What is skill-evaluator?

A repeatable checking skill for evaluating other coding-agent skills. It examines their files and tests, runs realistic prompts, and compares results with a baseline, meaning an earlier or no-skill version.

Use it to check a skill's structure, review or add evaluations, run usage tests, score results, or compare versions.

It helps reveal missing setup files, weak tests, broken instructions, and differences in real use. This makes skill quality easier to judge with evidence.

skill-evaluator — Skill for Claude Code and Codex by d-o-hub

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

Any agent

npx skills add d-o-hub/github-template-ai-agents --skill skill-evaluator

Clone the repo

git clone --depth 1 https://github.com/d-o-hub/github-template-ai-agents

Made for: Claude Code, Codex.

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

README.md

[![agentmods](https://agentmods.dev/badge/skills/d-o-hub/github-template-ai-agents/skill-evaluator/github.svg)](https://agentmods.dev/skills/d-o-hub/github-template-ai-agents/skill-evaluator)

Your own site

<a href="https://agentmods.dev/skills/d-o-hub/github-template-ai-agents/skill-evaluator"><img src="https://agentmods.dev/badge/skills/d-o-hub/github-template-ai-agents/skill-evaluator/github.svg" alt="Measured on agentmods" height="20"></a>

Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.

Your own site · 80×15

<a href="https://agentmods.dev/skills/d-o-hub/github-template-ai-agents/skill-evaluator"><img src="https://agentmods.dev/badge/skills/d-o-hub/github-template-ai-agents/skill-evaluator.svg" alt="Reviewed on agentmods" width="80" height="20"></a>

Per session 87 Skills are progressive disclosure: only the name and description are preloaded; the body loads when the skill is used.

When invoked 1,973 The whole file, excluding the scripts and references it only reads on demand.

Security scan A 0 findings. A grade says what 26 rules found in the file — not that it is safe.

Origin original No closer match found in the catalogue.

Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

Model	Per session	Once invoked
Fable 5.1	$0.00087	$0.01973
Opus 5.5	$0.00035	$0.00789
Sonnet 5.5	$0.00017	$0.00395
Haiku 4.5	$0.00009	$0.00197

Measured 13d ago against content hash 2f60af85ae79, method: parsed. Prices are Anthropic first-party input rates as of 2026-10-07, from the pricing page.

Security

Grade A, and why

skill-evaluator scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 13d ago.

The scan reads SKILL.md. This mod also ships 1 executable file (scripts/check_structure.py), listed below but not scanned — reading those needs a real analyzer, not pattern matching.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

.agents/skills/skill-evaluator/SKILL.md · 205 lines

How it starts

The opening of the file, as written. The whole thing — 205 lines — stays where its author put it; the contents beside it link to each section on GitHub.

Skill Evaluator

Evaluate local skills with evidence and an explicit scope. Choose static audit or manual paired behavioral evaluation; never confuse their verdicts.

When to Use

Check whether a skill is wired correctly and has usable eval definitions
Review assertion quality and coverage without executing user prompts
Grade actual outputs or compare a skill against a no-skill/older baseline
Identify missing fixtures, weak evals, and unclear routing boundaries

Required Inputs

SKILL_PATH: absolute or workspace-relative path to the skill directory
MODE: static (default) / manual-paired
GOAL: structure / coverage review / behavioral comparison

If the user requests behavioral evaluation but execution is unavailable, report behavioral: NOT_RUN or BLOCKED, not a static result relabeled as behavior.

Mode 1: Static Audit

This mode reads definitions only. It invokes no model, executes no eval prompt, and scores no assertion against an answer. No secrets or authentication are needed for static evaluation, including evals describing authenticated tools.

1. Structure and Schema

python3 .agents/skills/skill-evaluator/scripts/check_structure.py --path "$SKILL_PATH"

The standalone checker accepts one skill directory or an inventory directory. It ignores support folders without SKILL.md, _-prefixed directories, and *-workspace directories. An empty inventory fails rather than passing vacuously.

Template-local requirements (not Agent Skills specification requirements):

SKILL.md exists; no nested skill-name/skill-name/ duplicate
evals/evals.json exists, even if evals/ itself was never created
Root JSON object with matching non-empty skill_name and evals array
At least 3 cases; each is an object with unique integer id, non-empty prompt and expected_output strings, and a non-empty assertions string array
Optional files is an array of non-empty paths to existing input files under the skill root; absolute paths, escapes, and missing fixtures fail
Optional bucket uses explicit, implicit, contextual, or negative; a tagged set needs a real out-of-scope negative case

Read the full file on GitHub · 205 lines

Files

What ships with it

6 files beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

13d ago Changed · -36 lines · +7 tokens per session 2f60af85ae79
24d ago Changed · +5 lines · +80 tokens per session 1ec41f737010
25d ago Changed · +3 lines 3424b439c3e8
1mo ago First seen · 233 lines · 0 tokens per session scan A ee2f8bdaabdd

Subscribe to this mod's changes

skill-evaluator is a skill published in the GitHub repository d-o-hub/github-template-ai-agents (2 stars, last pushed yesterday), licensed MIT. It adds 87 tokens to every session and 1,973 once invoked, about $0.0003 per session on Opus 5.5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.

Building agents ai-agents ai-tools quality

verification-quality

Truth scoring, code-quality verification, and automatic rollback with a high accuracy threshold. Use when validating agent output before it ships, gating changes on quality, or adding a verify-and-rollback safety loop.

frankxai/claude-skills-library · 43 tokens

code-review

Orchestrates code review by detecting the project language and architecture, then routing to the appropriate specialized review skill (code-review-go, code-review-typescript, code-review-python, code-review-php, code-review-architecture). Falls back to the generic checklist when no specific skill applies. Invoked when…

soulcodex/agentic · 80 tokens

static-code-analysis

Selects, configures, and integrates a static analysis tool for the project's language. Covers tool selection, rule configuration, CI integration, fixing existing violations, and pre-commit hook setup. Invoked when the user asks to add linting, set up static analysis, or configure a code quality tool.

soulcodex/agentic · 65 tokens

code-review-architecture

Architecture-focused code review covering hexagonal boundary violations, DDD anti-patterns, CQRS misuse, and microservices coupling issues. Applied in addition to the language-specific review skill when architecture markers are detected. Invoked when reviewing hexagonal architectures, DDD patterns, or microservices…

soulcodex/agentic · 63 tokens

code-review-go

Deep Go-specific code review covering goroutine lifecycle, data races, error wrapping, domain modeling, and interface design. Applied in addition to the generic code-review skill when Go code is detected. Invoked when reviewing Go PRs, Go code changes, or performing Go-specific quality checks.

soulcodex/agentic · 61 tokens

code-review-php

Deep PHP-specific code review covering strict types, PHPStan compliance, PSR standards, domain modeling, and ORM boundary violations. Applied in addition to the generic code-review skill when PHP code is detected. Invoked when reviewing PHP PRs, Symfony/Laravel changes, or performing PHP-specific quality checks.

soulcodex/agentic · 66 tokens