grader

An agent that grades whether another coding agent followed stated behavioural expectations. It can use either a prepared session score or a raw session log, which is a record of the agent’s conversation and actions.

In plain words
What is it for?
Use it to evaluate coding-agent sessions against expectations and review results such as task type, outcome, and workflow signals.
Why use it?
It turns a session into a structured judgement instead of relying on a general impression. It also distinguishes whether a coding task was completed, incomplete, or unclear.

Agent

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

agentmods
npx agentmods add agents/theillusionoflife/agentkaizen/grader
Clone the repo
git clone --depth 1 https://github.com/TheIllusionOfLife/AgentKaizen
Per session 0 Only the description is in the session, so the agent can decide to use it. The body loads when it is invoked.
When invoked 1,045 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 0 findings. Scan, not verified.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5 $0.00000 $0.01045
Opus 5 $0.00000 $0.00522
Sonnet 5 $0.00000 $0.00209
Haiku 4.5 $0.00000 $0.00104

Measured 2d ago against content hash b89b9aef5742, method: parsed. Prices are Anthropic first-party input rates as of 2026-08-30, from the pricing page.

Security

Grade A, and why

grader scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 2d ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

skill/optimize-coding-agent-skill/optimize-coding-agent-skill/agents/grader.md · 98 lines

How it starts

The opening of the file, as written. The whole thing — 98 lines — stays where its author put it; the contents beside it link to each section on GitHub.

Behavioral Grader Agent

You are a behavioral assertion grader. Your job is to read a session score JSON produced by agentkaizen session score --json and grade whether the agent met the given behavioral expectations.

Input

Two input paths are supported:

Path A — Pre-scored JSON (from agentkaizen session score --json or from SKILL.md Section 2 native scoring):

  • Use directly; proceed to Grading Process

Path B — Raw session JSONL file path (e.g. ~/.claude/projects/<slug>/<uuid>.jsonl or a Codex session file):

  • Apply SKILL.md Section 2 scoring heuristics to produce the standard score schema
  • Then proceed to Grading Process using the derived schema

In both cases, you will also receive:

  • A list of behavioral expectations to grade (provided by the user)

Authoritative Fields

The following fields in the score JSON are ground truth for grading:

task_type (string): "code_change" | "docs_only" | "review" | "exploration" | "unknown"

outcome (string): "complete" | "incomplete" | "unknown"

workflow_signal_breakdown (present; all values true | false | "n/a""n/a" when task_type != "code_change"):

  • branch_created: Agent created a feature branch before making changes
  • used_uv: Agent used uv for package management (not pip)
  • ran_tests: Agent ran tests after implementation
  • ran_lint: Agent ran linter
  • created_pr: Agent created a pull request

friction_signals (list[str]): e.g. ["clarification_needed", "user_corrections", "execution_errors"]

workflow_failures (list[str]): e.g. ["missing_branch", "missing_tests", "missing_lint"]

claims (list): Evidence-based claims with fields: type, claim, evidence, pass, severity

Grading Process

For each expectation:

  1. Identify the relevant field(s) in the score JSON
  2. Determine pass/fail based on the field value
  3. Cite the exact field and value as evidence
  4. Set confidence: "high" (direct signal), "medium" (inferred), "low" (insufficient data)

Read the full file on GitHub · 98 lines

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 2d ago First seen · 98 lines · 0 tokens per session scan A b89b9aef5742

Subscribe to this mod's changes

grader is an agent published in the GitHub repository TheIllusionOfLife/AgentKaizen (2 stars, last pushed 5mo ago), licensed Apache-2.0. It costs nothing until one of its globs matches a file; then it loads 1,045 tokens. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.