analyzer

analyzer is an agent for Claude Code from EntityProcess/agentv. It costs 55 tokens per session (2,277 once invoked), scanned A, original, MIT.

A read-only reviewer for AgentV evaluation results and their configuration. AgentV evaluations test how well an AI agent completes tasks, while JSONL is a text format with one JSON record per line.

In plain words
What is it for?
Use it to review evaluation output, inspect EVAL.yaml settings, identify flaky tests, and suggest more predictable grading checks.
Why use it?
It helps find weak or inconsistent checks, unnecessary use of AI-based grading, and evaluation setups that cost more or produce less reliable results.

Agent for Claude Code

Written for Claude Code: a Claude Code subagent (agents/*.md). Also seen: model in frontmatter.

Good fit Use it to review evaluation output, inspect EVAL.yaml settings, identify flaky tests, and suggest more predictable grading checks.

Compare 6 agents from other repositories ↓
Install with agentmods
npx agentmods add agents/entityprocess/agentv/analyzer
Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

Clone the repo
git clone --depth 1 https://github.com/EntityProcess/agentv

Made for: Claude Code.

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

agentmods badge for analyzer

README.md
[![agentmods](https://agentmods.dev/badge/agents/entityprocess/agentv/analyzer/github.svg)](https://agentmods.dev/agents/entityprocess/agentv/analyzer)
Your own site
<a href="https://agentmods.dev/agents/entityprocess/agentv/analyzer"><img src="https://agentmods.dev/badge/agents/entityprocess/agentv/analyzer/github.svg" alt="Measured on agentmods" height="20"></a>

Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.

agentmods 80×15 button for analyzer

Your own site · 80×15
<a href="https://agentmods.dev/agents/entityprocess/agentv/analyzer"><img src="https://agentmods.dev/badge/agents/entityprocess/agentv/analyzer.svg" alt="Reviewed on agentmods" width="80" height="20"></a>
Per session 55 Only the description is in the session, so the agent can decide to use it. The body loads when it is invoked.
When invoked 2,277 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 0 findings. A grade says what 26 rules found in the file — not that it is safe.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5.1 $0.00055 $0.02277
Opus 5.5 $0.00022 $0.00911
Sonnet 5.5 $0.00011 $0.00455
Haiku 4.5 $0.00006 $0.00228

Measured 1mo ago against content hash 647839d874df, method: parsed. Prices are Anthropic first-party input rates as of 2026-10-07, from the pricing page.

Security

Grade A, and why

analyzer scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 1mo ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

skills-data/agentv-bench/agents/analyzer.md · 178 lines

How it starts

The opening of the file, as written. The whole thing — 178 lines — stays where its author put it; the contents beside it link to each section on GitHub.

You are an eval-quality analyst for AgentV. Your job is to read JSONL evaluation results and the corresponding EVAL.yaml config, then produce a structured report of improvement opportunities. You are read-only — never modify any files.

You will receive these parameters:

  • results-file: Path to a .jsonl results file (from agentv eval or .agentv/results/)
  • eval-path (optional): Path to the EVAL.yaml file for additional context

Analysis Process

Step 1: Load Results

Read every line of the JSONL results file. Each line is a JSON object with:

  • test_id, suite, score, assertions, reasoning, target
  • scores (optional): Array of per-grader breakdowns with name, type, score, weight, pass, assertions, reasoning

If eval-path is provided, also read the EVAL.yaml to understand grader configurations.

Step 2: Deterministic-Upgrade Analysis

For each grader entry in scores where type is "llm-rubric" or "rubrics", inspect the reasoning and assertions fields for patterns that indicate a deterministic assertion would suffice:

Signal Detection Suggested Upgrade
Reasoning cites exact substring match Reasoning contains phrases like "contains", "includes the text", "mentions [quoted string]" type: contains with value: "<extracted string>"
Score is always 0.0 or 1.0 across all test cases for this grader Collect scores per grader name; if all are binary type: equals or deterministic check — LLM is doing binary work
Reasoning references JSON validity "valid JSON", "parseable JSON", "well-formed JSON" type: is-json
Reasoning references format compliance "starts with", "begins with", "output starts with [string]" type: regex with value: "^<extracted prefix>"
Reasoning references ending pattern "ends with", "output ends with" type: regex with value: "<extracted suffix>$"
Reasoning matches regex-like pattern "matches pattern", "follows the format", explicit regex mention type: regex with value: "<extracted pattern>"
Reasoning checks field presence/value "field X is Y", "contains key", "has property" in JSON output type: field-accuracy with expected fields
All passed assertions are substring checks Every passed assertion entry quotes a specific string found in output Multiple type: contains assertions (one per value from passed assertions)

Read the full file on GitHub · 178 lines

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 1mo ago First seen · 178 lines · 55 tokens per session scan A 647839d874df

Subscribe to this mod's changes

analyzer is an agent published in the GitHub repository EntityProcess/agentv (15 stars, last pushed 6d ago), licensed MIT. It adds 55 tokens to every session and 2,277 once invoked, about $0.0002 per session on Opus 5.5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.

Related

Other agents, from other repositories

analyzer

Analyze blind comparison results to identify why the winner won and generate improvement suggestions for the losing skill. Also surfaces patterns in benchmark runs.

feiskyer/claude-code-settings · 29 tokens

thermo-nuclear-code-quality-review-subagent

Thermo-nuclear code quality audit (maintainability, structure, 1k-line rule, spaghetti, code-judo). Invoked through the Agent tool after a parent gathers diff and file contents. Loads rubric from the thermo-nuclear-code-quality-review skill in this plugin.

michael-denyer/pstack-claude · 68 tokens

alchemist

Creative technologist who sees the browser as an unexplored physics engine. Consult when building UI that needs to feel alive - scroll-driven reveals, morphing transitions, spatial animation systems, anything where the interaction itself IS the product. Thinks in weight, tension, and breath before thinking in code.…

drobins25/craft · 355 tokens

astronomical-instrumentation-scientist

Reasons from system-level error budgets, the diffraction limit and Strehl ratio, detector figures of merit, and resolving power through Zemax/Code V tolerancing, ETC radiometry, AO modeling, and on-sky standard-star commissioning while treating flexure drift, IR persistence, ghosts, and quasi-static speckles as…

K-Dense-AI/scientific-agents · 78 tokens

Explore

Fast agent specialized for exploring codebases. Use this when you need to quickly find files by patterns (eg. "src/components//.tsx"), search code for keywords (eg. "API endpoints"), or answer questions about the codebase (eg. "how do API endpoints work?"). When calling this agent, specify the desired thoroughness…

rojim666/SztuCode · 0 tokens

gate

API quality gates — linting, style enforcement, breaking change CI, and API governance.

tonone-ai/tonone · 19 tokens