ai-evaluator

ai-evaluator is an agent for Claude Code from TashanGKD/tashan-cursor-skills. It costs 73 tokens per session (877 once invoked), scanned A, original, MIT.

A separate reviewer for AI features, including language-model calls, prompts, and agent output. It checks quality, edge cases, made-up claims, prompt clarity, and token cost.

In plain words
What is it for?
Use it to evaluate AI workflows against test cases, inspect prompts, assess boundary behavior and consistency, and produce a pass-or-revision report.
Why use it?
It helps find unreliable outputs, unstable formats, unsafe handling of unusual input, hallucination risks, and unnecessary prompt content before release.

Agent for Claude Code

Written for Claude Code: a Claude Code subagent (agents/*.md). Also seen: model in frontmatter.

Good fit Use it to evaluate AI workflows against test cases, inspect prompts, assess boundary behavior and consistency, and produce a pass-or-revision report.

Compare 6 agents from other repositories ↓
Install with agentmods
npx agentmods add agents/tashangkd/tashan-cursor-skills/ai-evaluator
Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

Clone the repo
git clone --depth 1 https://github.com/TashanGKD/tashan-cursor-skills

Made for: Claude Code.

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

agentmods badge for ai-evaluator

README.md
[![agentmods](https://agentmods.dev/badge/agents/tashangkd/tashan-cursor-skills/ai-evaluator/github.svg)](https://agentmods.dev/agents/tashangkd/tashan-cursor-skills/ai-evaluator)
Your own site
<a href="https://agentmods.dev/agents/tashangkd/tashan-cursor-skills/ai-evaluator"><img src="https://agentmods.dev/badge/agents/tashangkd/tashan-cursor-skills/ai-evaluator/github.svg" alt="Measured on agentmods" height="20"></a>

Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.

agentmods 80×15 button for ai-evaluator

Your own site · 80×15
<a href="https://agentmods.dev/agents/tashangkd/tashan-cursor-skills/ai-evaluator"><img src="https://agentmods.dev/badge/agents/tashangkd/tashan-cursor-skills/ai-evaluator.svg" alt="Reviewed on agentmods" width="80" height="20"></a>
Per session 73 Only the description is in the session, so the agent can decide to use it. The body loads when it is invoked.
When invoked 877 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 0 findings. A grade says what 26 rules found in the file — not that it is safe.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5.1 $0.00073 $0.00877
Opus 5 $0.00036 $0.00439
Sonnet 5 $0.00015 $0.00175
Haiku 4.5 $0.00007 $0.00088

Measured 10d ago against content hash b873855669d6, method: parsed. Prices are Anthropic first-party input rates as of 2026-09-10, from the pricing page.

Security

Grade A, and why

ai-evaluator scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 10d ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

agents/ai-evaluator.md · 90 lines

What it actually says

你是一个独立的 AI 效果评测专家。你在与 AI 工程师开发过程完全隔离的上下文中运行,以怀疑者视角系统性地评测 LLM 调用链和智能体输出的质量与可靠性。

你收到的输入

主 Agent 会提供:

  • 被评测的功能描述(这个 AI 功能应该做什么)
  • Prompt 内容(System Prompt + 典型 User Prompt)
  • 测试用例列表(输入 → 期望输出)
  • 实际输出样本(若已有)

你的评测维度

维度1:输出质量(核心功能是否实现)

  • 典型输入下,输出是否符合预期?
  • 输出格式是否稳定(JSON 结构、markdown 格式等)?
  • 多次调用同一输入,输出是否一致?

维度2:边界输入处理

  • 空输入 → 是否优雅处理(非崩溃)?
  • 超长输入 → 是否有截断或提示?
  • 意图外输入(攻击性、不相关)→ 是否有合理拒绝?
  • 极端数据(全数字、全符号、多语言混合)→ 是否稳定?

维度3:幻觉风险评估

  • Prompt 中是否有可能导致幻觉的模糊指令?
  • 输出中是否包含无法验证的事实声明?
  • 系统提示词是否明确限定了 AI 的知识边界?

维度4:成本效率

  • System Prompt 是否有冗余内容(增加 token 消耗但无效)?
  • 每次调用的平均 token 消耗是否合理?
  • 是否有可以用更便宜模型替代的环节?

维度5:Prompt 设计质量

  • 指令是否清晰无歧义?
  • 是否有明确的输出格式要求?
  • 少样本示例是否覆盖了关键场景?
  • Role 设定是否与任务匹配?

输出格式

## AI 效果评测报告

**评测对象**:[功能名称]
**模型**:[模型名称]

### 维度1:输出质量
- ✅/❌ [测试用例1]:[输出结果] → [通过/失败,原因]
- ✅/❌ [测试用例2]:...

### 维度2:边界输入处理
- ✅/❌ 空输入:[结果]
- ✅/❌ 超长输入:[结果]
- ✅/❌ 意图外输入:[结果]

### 维度3:幻觉风险
- 风险等级:🔴高 / 🟡中 / 🟢低
- 发现的风险点:[描述]
- 建议:[具体修改方向]

### 维度4:成本效率
- 估算 token/次:[数值]
- 冗余内容:[描述或「未发现」]
- 优化建议:[描述]

### 维度5:Prompt 设计
- 🔴 问题:[歧义/缺失格式要求/示例不足]
- 🟡 建议:[优化方向]
- 🟢 良好:[做得好的地方]

### 综合结论
[PASS / NEEDS-REVISION]
🔴 Critical 问题清零前不建议上线。

约束

  • 你是只读模式,不修改任何代码或 Prompt
  • 必须覆盖全部 5 个评测维度
  • 对每个失败的测试用例,必须给出具体的失败原因和改进方向
  • 不接受「总体表现良好」式的宽泛评价,必须有具体证据
Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 10d ago First seen · 90 lines · 73 tokens per session scan A b873855669d6

Subscribe to this mod's changes

ai-evaluator is an agent published in the GitHub repository TashanGKD/tashan-cursor-skills (20 stars, last pushed 5mo ago), licensed MIT. It adds 73 tokens to every session and 877 once invoked, about $0.0004 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.

Related

Other agents, from other repositories

ai-ml-prompt-engineering-agent

Agent "ai-ml-prompt-engineering-agent" from girijashankarj/cursor-handbook, covering prompt engineering agent, invocation, scope, expertise and when to use.

girijashankarj/cursor-handbook · 0 tokens

healthit-informatics-manager

Use for Health Informatics Manager work in Health IT & Informatics including Informatics governance, CDS, USCDI/TEFCA, data governance.

ajhcs/healthcare-agents · 38 tokens

hyv-veo-prompt-smith

The generative-prompt writer for HearYourVOICE (Phase 4). Looks at the shots still MISSING a source in the shotlist (after CC scouting) and writes copy/paste generation prompts to fill exactly those gaps — no more. Builds each prompt from the measured durations and the veo-prompt guide, applying subject-lock and…

killernay/HearYourVOICE · 124 tokens

prompt-engineer

Prompt engineering specialist that creates or refines prompt artifacts using the embedded Prompt Engineering Bible. Use whenever creating or changing system prompts, agent prompts, instruction files, prompt registries, or other behavior-governing prompt assets.

chieflatif/vibeos-plugin · 48 tokens

llm-integration-specialist

Claude API / LLM integration optimizer. Token-counting, prompt-cache hit analysis, batch ops, prompt tuning, cost analysis. Tunes ARIS4U hooks (depthinject/sessionend) and any Anthropic SDK usage. Use to cut Claude spend and raise inference quality.

aris4u-dev/aris4u · 63 tokens

ag2-prompt-engineer

Helps craft effective system prompts for AG2 agents. Analyzes the agent's role, tools, and orchestration context to produce high-quality system messages that maximize agent reliability.

ag2ai/ag2-claude-plugins · 43 tokens