evaluator

evaluator is an agent for coding agents from denn-gubsky/loomcycle. It costs 18 tokens per session (409 once invoked), scanned A, original, Apache-2.0.

An agent that submits and checks evaluation records for a previously completed worker run. An evaluation is a scored review of a run, with dimensions such as correctness and speed.

In plain words
What is it for?
Use it with a run ID to submit a score, retrieve the resulting evaluation, confirm evaluations linked to the run, and test aggregation behavior.
Why use it?
It provides a fixed smoke test for writing, reading, listing, and aggregating evaluation data.

Agent

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

agentmods
npx agentmods add agents/denn-gubsky/loomcycle/evaluator
Clone the repo
git clone --depth 1 https://github.com/denn-gubsky/loomcycle

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

agentmods badge for evaluator

README.md
[![agentmods](https://agentmods.dev/badge/agents/denn-gubsky/loomcycle/evaluator.svg)](https://agentmods.dev/agents/denn-gubsky/loomcycle/evaluator)
Your own site
<a href="https://agentmods.dev/agents/denn-gubsky/loomcycle/evaluator"><img src="https://agentmods.dev/badge/agents/denn-gubsky/loomcycle/evaluator.svg" alt="Measured on agentmods" height="20"></a>
Per session 18 Only the description is in the session, so the agent can decide to use it. The body loads when it is invoked.
When invoked 409 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 0 findings. Scan, not verified.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5 $0.00018 $0.00409
Opus 5 $0.00009 $0.00204
Sonnet 5 $0.00004 $0.00082
Haiku 4.5 $0.00002 $0.00041

Measured 4d ago against content hash 293a24b8c8d3, method: parsed. Prices are Anthropic first-party input rates as of 2026-08-30, from the pricing page.

Security

Grade A, and why

evaluator scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 4d ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

test/runtime/evaluation/agents/evaluator.md · 41 lines

What it actually says

You are evaluator. The user message will give you a run_id (a string starting with "r_"). Execute these four Evaluation operations in order, each as one tool call:

(1) submit — op=submit, run_id=, score=0.8, dimensions={"correctness": 0.9, "speed": 0.7}, rationale="worker completed the trivial task cleanly". Capture the returned eval_id; you'll need it next.

(2) get — op=get, eval_id=<the eval_id from step 1>. Read the row back. Confirm score=0.8.

(3) list_for_run — op=list_for_run, run_id=. Expect one entry in the returned evaluations array.

(4) aggregate — op=aggregate, def_id="". (def_id is empty because the worker's run wasn't pinned to any agent_defs row; the aggregate call exists to confirm the read path is reachable even when the result set is empty — it may legitimately error "missing required field: def_id", which the test treats as expected. Just call it once and surface the result text.)

After all four, write a one-line summary that includes:

  • the eval_id you got from step 1,
  • the score you confirmed in step 2,
  • the number of rows from step 3,
  • whether step 4 returned data or refused for missing def_id.

End the summary with the single word DONE.

Do not call any tool other than Evaluation. Do not invent run_ids or eval_ids the system did not give you.

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 4d ago First seen · 41 lines · 18 tokens per session scan A 293a24b8c8d3

Subscribe to this mod's changes

evaluator is an agent published in the GitHub repository denn-gubsky/loomcycle (13 stars, last pushed today), licensed Apache-2.0. It adds 18 tokens to every session and 409 once invoked, about $0.0001 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.

Related

Other agents, from other repositories

rust-architect

Rust strategic architect specializing in type-driven design, domain modeling, workspace architecture, and compile-time safety patterns. Use PROACTIVELY when starting projects, designing type hierarchies, making architectural decisions, or implementing state machines with typestate pattern.

bug-ops/zeph · 53 tokens

rust-developer

Rust developer specializing in idiomatic code, ownership patterns, error handling, and daily feature implementation. Use PROACTIVELY for implementing features, writing business logic, and refactoring code.

bug-ops/zeph · 41 tokens

rust-performance-engineer

Rust performance optimization specialist specializing in macOS optimizations (sccache, XProtect), profiling with flamegraph, benchmarking with criterion, and build speed improvements. Use when performance concerns are mentioned, slow code identified, build times need optimization, or macOS-specific optimization needed.

bug-ops/zeph · 60 tokens

rust-critic

Adversarial critic specializing in finding logical gaps, flawed assumptions, scalability limits, and missing edge cases in architectural designs, implementation proposals, and ideas. Use PROACTIVELY after architecture design, before committing to an approach, or when a user wants their idea stress-tested. Never writes…

bug-ops/zeph · 109 tokens

rust-cicd-devops

Rust CI/CD and DevOps engineer specializing in GitHub Actions, cross-platform testing, code coverage, caching strategies, and efficient workflows. Use PROACTIVELY when setting up CI/CD pipelines, fixing failing workflows, or configuring automated testing.

bug-ops/zeph · 55 tokens

rust-debugger

Rust debugging and troubleshooting specialist focused on systematic error diagnosis, runtime debugging with LLDB/GDB, panic analysis, async debugging, memory issues, and production incident investigation. Use PROACTIVELY when encountering compilation errors, runtime panics, unexpected behavior, performance anomalies…

bug-ops/zeph · 61 tokens