What is eval-trigger?

A command that tests whether an Autology skill is triggered by suitable user requests. It runs isolated evaluations using predefined queries and checks whether the skill starts.

Use it to run trigger evaluations for an Autology skill and inspect whether each test query activates it.

It helps find descriptions that fail to make the agent invoke the intended skill, before relying on that skill in real work.

eval-trigger — Command by Curt-Park

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

agentmods

npx agentmods add commands/curt-park/autology/eval-trigger

Clone the repo

git clone --depth 1 https://github.com/Curt-Park/autology

Made for: Claude Code.

Per session 34 Only the description is in the session, so the agent can decide to use it. The body loads when it is invoked.

When invoked 1,662 The whole file, excluding the scripts and references it only reads on demand.

Security scan E 3 findings. Scan, not verified.

Origin original No closer match found in the catalogue.

Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

Model	Per session	Once invoked
Fable 5	$0.00034	$0.01662
Opus 5	$0.00017	$0.00831
Sonnet 5	$0.00007	$0.00332
Haiku 4.5	$0.00003	$0.00166

Measured 2d ago against content hash 04cdd6944039, method: parsed. Prices are Anthropic first-party input rates as of 2026-08-30, from the pricing page.

Security

Grade E, and why

eval-trigger scanned grade E with 3 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 2d ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Harvests environment variableshighData exfiltration

Enumerating or grepping the environment for keys collects credentials unrelated to what the mod says it does.

env = {k: v for k, v in os.environ.items() if k not in UNSET}

Recursive force deletehighDestructive command

rm -rf with a variable or a broad path is one typo away from removing the wrong tree.

rm -rf /tmp/trigger-eval-$ARGUMENTS

Runs shell commandslowCapability

Expected in a hook, worth knowing in a rule or an instructions file.

2. `claude -p "<query>"` runs via Python `subprocess.Popen` with `--plugin-dir <stub>` and `--setting-sources ''` — subprocess sees exactly one skill (plus Claude Code built-ins)

.claude/commands/eval-trigger.md · 168 lines

How it starts

The opening of the file, as written. The whole thing — 168 lines — stays where its author put it; the contents beside it link to each section on GitHub.

Autology Trigger Eval

Empirically test whether the $ARGUMENTS skill description causes Claude to actually invoke it — using an isolated claude -p subprocess with only the target skill visible.

How it works

For each query in trigger_evals.json:

A stub plugin dir is created containing only the target skill's SKILL.md
claude -p "<query>" runs via Python subprocess.Popen with --plugin-dir <stub> and --setting-sources '' — subprocess sees exactly one skill (plus Claude Code built-ins)
stream-json is parsed line-by-line; trigger detected via stream_event/content_block_start → process killed immediately before skill executes
All queries run in parallel via ThreadPoolExecutor

Why the runner script must be executed in the background:

claude -p deadlocks when its parent process is the blocked Bash tool: the subprocess tries to connect to the parent Claude Code process via CLAUDE_CODE_SSE_PORT, but the parent is waiting for the Bash tool to finish
Running run.py in the background (python3 -u run.py > run.log 2>&1 &) frees the parent, breaking the deadlock
The Bash tool then polls run.log for completion

Why CLAUDECODE, ANTHROPIC_API_KEY, CLAUDE_CODE_SSE_PORT must be unset:

CLAUDECODE — prevents nested Claude Code detection
ANTHROPIC_API_KEY — forces OAuth (claude.ai subscription); Max subscribers without API credits get a silent billing_error otherwise
CLAUDE_CODE_SSE_PORT — prevents the subprocess from attempting IPC with the parent

Why --setting-sources '' is needed:

Skips global plugin settings; only --plugin-dir skills are visible
Built-in skills (keybindings-help, simplify, loop, claude-api) are always present regardless — but these occupy unrelated domains and don't compete with autology skills in practice

Step 1: Read the eval set

Read:

skills/$ARGUMENTS/SKILL.md — note the description field (the only thing Claude sees)
skills/$ARGUMENTS/evals/trigger_evals.json — queries with should_trigger labels

Read the full file on GitHub · 168 lines

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

2d ago First seen · 168 lines · 0 tokens per session scan E 04cdd6944039

Subscribe to this mod's changes

eval-trigger is a command published in the GitHub repository Curt-Park/autology (24 stars, last pushed 5mo ago), licensed MIT. It adds 34 tokens to every session and 1,662 once invoked, about $0.0002 per session on Opus 5. A static security scan graded it E with 3 findings (harvests environment variables, recursive force delete, runs shell commands). No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.

Building agents claude-code decision-tracking knowledge-management living-documentation mcp-server organizational-learning

eval-trigger

Getting it into your agent

What it costs to keep this loaded

Grade E, and why

Harvests environment variableshighData exfiltration

Recursive force deletehighDestructive command

Runs shell commandslowCapability

How it starts

Autology Trigger Eval

How it works

Step 1: Read the eval set

What this file has done since we first saw it

Other commands, from other repositories

configure

checkpoint

decision

search

resume

add-entry