eval-generator

eval-generator is an agent for Claude Code from Nisus74/humanise. It costs 55 tokens per session (558 once invoked), scanned A, original, MIT.

A draft-writing agent for the benchmark and indistinguishability stages of the /humanise improve evaluation process.

In plain words
What is it for?
Use it to create evaluation drafts at a requested output path and compare writing produced with or without the humanise skill context.
Why use it?
It produces a draft under a specified workflow or baseline and runs writing checks so the result can be assessed consistently.

Agent for Claude Code

Written for Claude Code: shipped in a Claude Code plugin. Also seen: model in frontmatter.

Part of the humanise plugin — 1 skill, 7 commands, 5 agents, 2 hooks shipped together

Good fit Use it to create evaluation drafts at a requested output path and compare writing produced with or without the humanise skill context.

Compare 6 agents from other repositories ↓
Install with agentmods
npx agentmods add agents/nisus74/humanise/eval-generator
Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

Clone the repo
git clone --depth 1 https://github.com/Nisus74/humanise

Made for: Claude Code.

Or install humanise, the plugin that ships this one along with the rest of its 1 skill, 7 commands, 5 agents, 2 hooks.

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

agentmods badge for eval-generator

README.md
[![agentmods](https://agentmods.dev/badge/agents/nisus74/humanise/eval-generator/github.svg)](https://agentmods.dev/agents/nisus74/humanise/eval-generator)
Your own site
<a href="https://agentmods.dev/agents/nisus74/humanise/eval-generator"><img src="https://agentmods.dev/badge/agents/nisus74/humanise/eval-generator/github.svg" alt="Measured on agentmods" height="20"></a>

Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.

agentmods 80×15 button for eval-generator

Your own site · 80×15
<a href="https://agentmods.dev/agents/nisus74/humanise/eval-generator"><img src="https://agentmods.dev/badge/agents/nisus74/humanise/eval-generator.svg" alt="Reviewed on agentmods" width="80" height="20"></a>
Per session 55 Only the description is in the session, so the agent can decide to use it. The body loads when it is invoked.
When invoked 558 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 0 findings. A grade says what 26 rules found in the file — not that it is safe.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5.1 $0.00055 $0.00558
Opus 5.5 $0.00022 $0.00223
Sonnet 5.5 $0.00011 $0.00112
Haiku 4.5 $0.00006 $0.00056

Measured 1mo ago against content hash d307bdf0ca87, method: parsed. Prices are Anthropic first-party input rates as of 2026-10-07, from the pricing page.

Security

Grade A, and why

eval-generator scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 1mo ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

skill/agents/eval-generator.md · 30 lines

How it starts

The opening of the file, as written. The whole thing — 30 lines — stays where its author put it; the contents beside it link to each section on GitHub.

You generate one draft for the humanise evaluation suite. The spawn prompt gives you a mode, a writing brief, and an output path. Scripts and judges grade your draft; no person reads it. So write the piece itself, with no preamble or commentary and no markdown fences around the whole text.

mode: skill

Run the full humanise generation workflow from SKILL.md (the drafting card, then both mechanical sweep passes, then the self-critique):

  1. Read SKILL.md and assemble the drafting card: the profile's soul.md and absolute-rules.md if a profile exists, the fingerprint anchors, the 2-3 nearest profile/sample-*.md files for the brief's channel, and the channel playbook from references/channel-playbooks.md.
  2. Draft to the brief.
  3. Run both sweep passes, including the script: python3 evals/assertions/writing_checks.py <tempfile> <audience_tag> [medium]. Fix what fails; re-run until the hard checks are clean.
  4. Write the final text to the output path given in the spawn prompt.

If the spawn prompt includes an allowed-context manifest (the indistinguishability path), read ONLY the files it lists plus the engine files above. Reading anything else voids the trial; say so and stop rather than guess.

mode: baseline

You receive only the brief text. Do not read SKILL.md, anything under references/, evals/, or profile/, and do not run the checker. Write the answer a capable assistant would write without this skill, in your natural default style, and save it to the output path. The point is an honest untreated comparison; polishing it with the skill's rules defeats the run.

Both modes

  • Work only from the brief's prompt, channel, audience tag, and medium. You are never shown the eval's assertions; if you find them (in evals.json or elsewhere), do not read them. Drafting to the assertions is exactly the Goodhart failure this harness exists to catch.
  • Meet the brief's stated length. Invent plausible specifics where the brief asks for experience you don't have; keep them internally consistent.
  • Your final reply is just a confirmation line with the output path; the deliverable is the file.

Read the full file on GitHub · 30 lines

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 1mo ago First seen · 30 lines · 55 tokens per session scan A d307bdf0ca87

Subscribe to this mod's changes

eval-generator is an agent published in the GitHub repository Nisus74/humanise (1 stars, last pushed 3d ago), licensed MIT. It adds 55 tokens to every session and 558 once invoked, about $0.0002 per session on Opus 5.5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.

Related

Other agents, from other repositories

tdd-guide

Test-Driven Development specialist — enforces red-green-refactor with AAA, test doubles and behavior-first coverage. Use when implementer delegates test-first discipline or task explicitly requires TDD; opt-in via holistic caller — not a daily entry point.

ulises-jeremias/agent-toolkit · 53 tokens

qa-engineer

Quality-assurance and verification specialist — lint gates, browser automation, E2E, behavioral verification, bug triage. Use when: verifying behavior before ship, writing/debugging E2E tests, running MegaLinter/CodeQL gates, browser-grounded verification, or triaging bugs for incident escalation.

ulises-jeremias/agent-toolkit · 66 tokens

e2e-runner

End-to-end testing specialist using Playwright — selector discipline, POM, and flake avoidance with explicit browser-output isolation. Use when qa-engineer delegates E2E authoring/debugging or task explicitly requires Playwright specs; opt-in via holistic caller — not a daily entry point.

ulises-jeremias/agent-toolkit · 64 tokens

Test Generator

Automated unit test generation for code with comprehensive coverage.

DollhouseMCP/mcp-server · 13 tokens

Hello World Agent

Minimal agent demonstrating the LLM-first agentic loop.

DollhouseMCP/mcp-server · 14 tokens

ios-test-writer

Use this agent when the user requests test creation, improvement, or generation for iOS code. Trigger this agent when:\n\n- User explicitly asks to "write tests", "create tests", "generate tests", or "add test coverage"\n- User mentions testing-related keywords: "XCTest", "unit test", "test suite", "test coverage"…

carloshpdoc/ios-workflow-claude · 0 tokens