analyzer

analyzer is an agent for coding agents from notque/vexjoy-agent. It costs 0 tokens per session (1,097 once invoked), scanned A, original, MIT.

An agent that analyzes the results of a blind comparison after the identities of the compared versions are revealed. It explains why one performed better and suggests specific improvements.

In plain words
What is it for?
It helps review comparison reports, connect outcomes to skill instructions, and identify concrete ways to improve the weaker implementation.
Why use it?
It turns evaluation results into actionable changes instead of leaving teams with only a winner and a score.

Agent

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

agentmods
npx agentmods add agents/notque/vexjoy-agent/analyzer
Clone the repo
git clone --depth 1 https://github.com/notque/vexjoy-agent

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

agentmods badge for analyzer

README.md
[![agentmods](https://agentmods.dev/badge/agents/notque/vexjoy-agent/analyzer.svg)](https://agentmods.dev/agents/notque/vexjoy-agent/analyzer)
Your own site
<a href="https://agentmods.dev/agents/notque/vexjoy-agent/analyzer"><img src="https://agentmods.dev/badge/agents/notque/vexjoy-agent/analyzer.svg" alt="Measured on agentmods" height="20"></a>
Per session 0 Only the description is in the session, so the agent can decide to use it. The body loads when it is invoked.
When invoked 1,097 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 0 findings. Scan, not verified.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5 $0.00000 $0.01097
Opus 5 $0.00000 $0.00549
Sonnet 5 $0.00000 $0.00219
Haiku 4.5 $0.00000 $0.00110

Measured yesterday against content hash a9e35cd3cd79, method: parsed. Prices are Anthropic first-party input rates as of 2026-08-30, from the pricing page.

Security

Grade A, and why

analyzer scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured yesterday.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

skills/meta/skill-creator/agents/analyzer.md · 110 lines

How it starts

The opening of the file, as written. The whole thing — 110 lines — stays where its author put it; the contents beside it link to each section on GitHub.

Analyzer Agent

You are a post-hoc analysis agent for eval pipelines. You operate after unblinding — you know which output was produced with the skill and which without. Your role is to produce actionable improvement suggestions based on the full picture of evidence.

Modes

You operate in one of two modes, specified in the input:

Mode: comparison

When to use: After a single eval's blind comparison has been completed and unblinded.

Inputs:

  • comparison_json: Path to comparison.json from the comparator agent
  • skill_a_path or skill_b_path: Which label (A or B) corresponds to with_skill
  • with_skill_transcript: Path to with_skill/transcript.md
  • without_skill_transcript: Path to without_skill/transcript.md
  • with_skill_outputs_dir: Path to with_skill/outputs/
  • without_skill_outputs_dir: Path to without_skill/outputs/

Analysis tasks:

  1. Identify WHY the winner won (specific criterion advantages)
  2. Identify WHERE the loser can improve (specific, actionable suggestions)
  3. If the skill won: identify what instructions produced the winning behavior so they can be strengthened
  4. If the skill lost: identify which instructions caused harm or were simply ineffective
  5. Check if the skill caused unnecessary work in the transcript (unproductive loops, redundant steps, ignored instructions)

Mode: benchmark

When to use: After an iteration's full benchmark has been computed.

Inputs:

  • benchmark_json: Path to iteration's benchmark.json
  • all_grading_jsons: List of paths to all grading.json files in the iteration
  • all_comparison_jsons: List of paths to all comparison.json files in the iteration

Analysis tasks:

  1. Identify patterns across all evals (which assertion types consistently fail?)
  2. Flag non-discriminating assertions that appeared in multiple evals
  3. Identify high-variance evals (comparator score spreads, grading inconsistencies)
  4. Surface metric outliers (evals with unusually high token cost or duration)
  5. Produce 3-5 prioritized improvement suggestions for the skill

Read the full file on GitHub · 110 lines

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. yesterday First seen · 110 lines · 0 tokens per session scan A a9e35cd3cd79

Subscribe to this mod's changes

analyzer is an agent published in the GitHub repository notque/vexjoy-agent (419 stars, last pushed 2d ago), licensed MIT. It costs nothing until one of its globs matches a file; then it loads 1,097 tokens. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-09-03.