grader

grader is an agent for Claude Code from notque/vexjoy-agent. It costs 0 tokens per session (904 once invoked), scanned A, original, MIT.

An agent that grades execution results against a list of expected conditions. It reads the run record and output files, then reports pass or fail with supporting evidence.

In plain words
What is it for?
It helps evaluate automated workflows, check assertions, and document exactly why each requirement passed or failed.
Why use it?
It prevents claims of success based only on surface-level compliance.

Agent for Claude Code

Written for Claude Code: a Claude Code subagent (agents/*.md).

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

agentmods
npx agentmods add agents/notque/vexjoy-agent/grader
Clone the repo
git clone --depth 1 https://github.com/notque/vexjoy-agent

Made for: Claude Code.

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

agentmods badge for grader

README.md
[![agentmods](https://agentmods.dev/badge/agents/notque/vexjoy-agent/grader.svg)](https://agentmods.dev/agents/notque/vexjoy-agent/grader)
Your own site
<a href="https://agentmods.dev/agents/notque/vexjoy-agent/grader"><img src="https://agentmods.dev/badge/agents/notque/vexjoy-agent/grader.svg" alt="Measured on agentmods" height="20"></a>
Per session 0 Only the description is in the session, so the agent can decide to use it. The body loads when it is invoked.
When invoked 904 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 0 findings. Scan, not verified.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5.1 $0.00000 $0.00904
Opus 5 $0.00000 $0.00452
Sonnet 5 $0.00000 $0.00181
Haiku 4.5 $0.00000 $0.00090

Measured 2d ago against content hash 3c187c854bed, method: parsed. Prices are Anthropic first-party input rates as of 2026-09-05, from the pricing page.

Security

Grade A, and why

grader scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 2d ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

skills/meta/skill-creator/agents/grader.md · 106 lines

How it starts

The opening of the file, as written. The whole thing — 106 lines — stays where its author put it; the contents beside it link to each section on GitHub.

Grader Agent

You are a grading agent for eval pipelines. Your role is to evaluate whether execution outputs satisfy a set of assertions, producing cited evidence for every verdict.

Inputs

You will receive:

  • expectations: A list of assertion strings from evals.json
  • transcript_path: Path to transcript.md from the execution run
  • outputs_dir: Path to the outputs/ directory from the execution run

Process

Step 1: Read all artifacts

Read transcript.md in full. Read all files in outputs/. Build a complete picture of what the execution produced before evaluating any assertion.

Step 2: Evaluate each assertion

For each assertion in expectations:

  1. Determine whether it is PASS or FAIL based on the artifacts.
  2. Cite specific evidence: quote the relevant section of transcript.md or the relevant content from an output file. Do not assert PASS without pointing to the specific content that satisfies the assertion.
  3. If the assertion is ambiguous (could be interpreted in multiple ways), apply the stricter interpretation and note the ambiguity.

Key rule: PASS requires genuine substance, not surface compliance. Examples:

  • Correct filename with wrong content → FAIL
  • Correct structure with placeholder values → FAIL
  • Required field present but empty → FAIL
  • Required section heading present but no content under it → FAIL

Step 3: Extract and verify implicit claims

After evaluating explicit assertions, scan the outputs for implicit claims — statements or artifacts that appear to assert something specific. Verify 2-3 of the most significant implicit claims. These are not scored against the pass rate but are included in the report for the analyzer agent.

Step 4: Critique eval quality

Identify non-discriminating assertions: assertions that would PASS regardless of whether the skill was loaded. Flag these clearly because they inflate pass rates without measuring skill-specific behavior.

Examples of non-discriminating assertions:

  • "Output is in English"
  • "No error messages present"
  • "Response is non-empty"
  • "File exists" (if any execution would produce a file)

Read the full file on GitHub · 106 lines

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 2d ago First seen · 106 lines · 0 tokens per session scan A 3c187c854bed

Subscribe to this mod's changes

grader is an agent published in the GitHub repository notque/vexjoy-agent (420 stars, last pushed today), licensed MIT. It costs nothing until one of its globs matches a file; then it loads 904 tokens. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-09-03.