agent-eval-canvas

A report-building tool for examining one software-agent run from start to finish. It records what the agent did, which step affected the result, why it succeeded or failed, and what should change.

In plain words
What is it for?
Use it to analyze one evaluation case, including its request, expected and actual results, timeline, tool calls, errors, evidence, score, and cost.
Why use it?
It turns a complicated run history into a clear incident review instead of leaving the cause buried in tool calls and logs.

Skill for Claude CodeCodex

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

agentmods
npx agentmods add skills/gcwing/bitfun/agent-eval-canvas
Any agent
npx skills add GCWing/BitFun --skill agent-eval-canvas
Clone the repo
git clone --depth 1 https://github.com/GCWing/BitFun

Made for: Claude Code, Codex.

Per session 81 Skills are progressive disclosure: only the name and description are preloaded; the body loads when the skill is used.
When invoked 1,941 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 0 findings. Scan, not verified.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5 $0.00081 $0.01941
Opus 5 $0.00041 $0.00971
Sonnet 5 $0.00016 $0.00388
Haiku 4.5 $0.00008 $0.00194

Measured 3d ago against content hash a770fd1c9e47, method: parsed. Prices are Anthropic first-party input rates as of 2026-08-30, from the pricing page.

Security

Grade A, and why

agent-eval-canvas scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 3d ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

src/crates/assembly/core/builtin_skills/agent-eval-canvas/SKILL.md · 218 lines

How it starts

The opening of the file, as written. The whole thing — 218 lines — stays where its author put it; the contents beside it link to each section on GitHub.

Agent Eval Canvas

Use this skill to produce a session-scoped BitFun Canvas for a single agent case. The report should read like an incident review or clinical diagnosis, not like an aggregate benchmark summary. Its job is to explain what happened in one run, which step determined success or failure, why, and what to fix next.

Read and follow bitfun-canvas first. It defines the Canvas tool workflow, source rules, SDK surface, design constraints, and final response requirements.

Inputs

Gather the case evidence before writing TSX:

  • Case ID, task type, original user request, expected result, actual result, final verdict, score, model, prompt version, tools, max steps, timeout, and environment facts when available.
  • Full or summarized trajectory: LLM calls, tool calls, handoffs, guardrail events, custom events, observations, retries, and final answer.
  • Tool call details: tool name, call timing, arguments, raw or summarized result, error state, whether the result was used, and recovery behavior.
  • Evidence artifacts: files, retrieved sources, code snippets, tables, judge output, screenshots, or execution logs that support the verdict.
  • Cost facts: total latency, steps, LLM calls, tool calls, tokens, repeated calls, invalid steps, budget/timeout status.
  • Safety facts: risky commands, permission prompts, sensitive data, production side effects, prompt injection exposure, and guardrail behavior.

If the trace or expected result is missing, ask for it unless the user clearly wants a best-effort report from partial evidence. In a best-effort report, mark unsupported fields as unavailable and keep root-cause claims conservative.

Canvas Structure

Lead with diagnosis, then evidence. The first screen should answer: did the case pass, what was the decisive step, and what should change next.

  1. Case header: Case ID, task type, verdict, score, run configuration, and a one-sentence conclusion.
  2. Diagnostic summary: final outcome, critical failure or success step, primary root cause, downstream propagation, confidence, and top repair.
  3. Trajectory timeline: compact step table or swimlane showing behavior type, agent action, input/observation, reasonableness, tag, and issue.
  4. Step-level evaluation: per-step cards or rows with labels: OK, WARN, ERROR, ROOT_CAUSE, and PROPAGATED.
  5. Critical step deep dive: why this step changed the run, what evidence proves it, why later errors are downstream, and what should have happened.
  6. Root cause and propagation: category breakdown plus a clear chain from first error to final result.
  7. Tool and evidence analysis: tool choice, parameters, results, result use, recovery behavior, unsupported claims, stale evidence, and source fit.
  8. Cost, efficiency, and safety: latency, tokens, duplicate work, invalid steps, risky actions, sensitive data, and confirmation/guardrail handling.
  9. Repair plan and regression: fixes grouped by Prompt, Planner, Tool schema, Tool result, Memory/RAG, Runtime, Evaluator, and Guardrail; include next-run pass criteria and steps to watch.

Read the full file on GitHub · 218 lines

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 3d ago First seen · 218 lines · 81 tokens per session scan A a770fd1c9e47

Subscribe to this mod's changes

agent-eval-canvas is a skill published in the GitHub repository GCWing/BitFun (1,871 stars, last pushed 3d ago), licensed MIT. It adds 81 tokens to every session and 1,941 once invoked, about $0.0004 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.

Related

Other skills, from other repositories

git-delegation

将所有 git 操作委托给 Manager 执行。Worker 无法直接访问 git credentials,因此任何需要认证的 git 操作(clone、push、fetch 等)都需要通过此机制委托给 Manager。.

agentscope-ai/AgentTeams · 51 tokens

agentteams-migrate

Analyze current OpenClaw setup and generate a migration package (ZIP) for importing into AgentTeams as a managed Worker.

agentscope-ai/AgentTeams · 28 tokens

workerflow-internal-workflow

Use when a QwenPaw-backed Worker needs to decide whether to do work directly, use native subagents for internal parallelism, or create a temporary QwenPaw agent with a custom AGENTS.md and skills.

agentscope-ai/AgentTeams · 52 tokens

find-skills

Discover and install agent skills from the open ecosystem. Use when you encounter an unfamiliar domain, framework, or workflow that you lack specialized knowledge about, or when your coordinator suggests searching for skills before starting a task.

agentscope-ai/AgentTeams · 46 tokens

higress-gateway-management

Manage the Higress AI Gateway via its Console API (consumers, routes, AI providers, MCP servers). Use when creating consumers, configuring routes, or managing AI gateway settings.

agentscope-ai/AgentTeams · 43 tokens

worker-management

Use when admin requests hand-creating or resetting a Worker, starting/stopping a Worker, managing Worker skills, enabling peer mentions, or opening a QwenPaw console. Use agentteams-find-worker only as a helper for Nacos-backed market import or when task assignment needs you to discover a suitable Worker.

agentscope-ai/AgentTeams · 65 tokens