patpat-eval

patpat-eval is a skill for Cursor from goiltpatpat/patpat. It costs 35 tokens per session (570 once invoked), scanned A, original, MIT.

A testing process for checking whether an agent skill activates and behaves correctly. It uses isolated trials with prompts that should trigger the skill and similar prompts that should not.

In plain words
What is it for?
Use it after significant skill changes or when unsure about routing. It helps define evaluation criteria, run controlled trials, inspect outputs, and decide whether a skill is ready.
Why use it?
It helps separate actual behavior from claims or confidence. This makes it easier to find incorrect triggering, weak results, scope problems, and cleanup issues.

Skill for Cursor

Written for Cursor: shipped in a Cursor plugin. Also seen: mentions Codex.

Part of the patpat plugin — 22 skills, 1 agent, 3 hooks shipped together

Good fit Use it after significant skill changes or when unsure about routing. It helps define evaluation criteria, run controlled trials, inspect outputs, and decide whether a skill is ready.

Compare 6 skills from other repositories ↓
Install with agentmods
npx agentmods add skills/goiltpatpat/patpat/patpat-eval
Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

Any agent
npx skills add goiltpatpat/patpat --skill patpat-eval
Clone the repo
git clone --depth 1 https://github.com/goiltpatpat/patpat

Made for: Cursor.

Or install patpat, the plugin that ships this one along with the rest of its 22 skills, 1 agent, 3 hooks.

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

agentmods badge for patpat-eval

README.md
[![agentmods](https://agentmods.dev/badge/skills/goiltpatpat/patpat/patpat-eval/github.svg)](https://agentmods.dev/skills/goiltpatpat/patpat/patpat-eval)
Your own site
<a href="https://agentmods.dev/skills/goiltpatpat/patpat/patpat-eval"><img src="https://agentmods.dev/badge/skills/goiltpatpat/patpat/patpat-eval/github.svg" alt="Measured on agentmods" height="20"></a>

Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.

agentmods 80×15 button for patpat-eval

Your own site · 80×15
<a href="https://agentmods.dev/skills/goiltpatpat/patpat/patpat-eval"><img src="https://agentmods.dev/badge/skills/goiltpatpat/patpat/patpat-eval.svg" alt="Reviewed on agentmods" width="80" height="20"></a>
Per session 35 Skills are progressive disclosure: only the name and description are preloaded; the body loads when the skill is used.
When invoked 570 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 0 findings. A grade says what 26 rules found in the file — not that it is safe.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5.1 $0.00035 $0.00570
Opus 5 $0.00017 $0.00285
Sonnet 5 $0.00007 $0.00114
Haiku 4.5 $0.00003 $0.00057

Measured 5d ago against content hash 5268ef250503, method: parsed. Prices are Anthropic first-party input rates as of 2026-09-09, from the pricing page.

Security

Grade A, and why

patpat-eval scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 5d ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

skills/patpat-eval/SKILL.md · 21 lines

What it actually says

Patpat Eval

When invoked directly, read the operating protocol in full. Do not load the router.

Read proof over proxy and apply the behavioral evaluation playbook.

Define the target behavior and rubric before running a trial. Include at least one prompt that should trigger the skill and one neighboring prompt that should not. Run trials in isolated temporary workspaces with equivalent context, organic names, and no hidden access to the expected conclusion.

Judge produced artifacts, commands, observations, scope control, and cleanup. Do not treat an agent's explanation or confidence as evidence. Record environmental limits and keep comparisons sequential unless isolation and integration proof have earned parallel execution.

Freeze the rubric before the first trial. Record PASS only when inspectable evidence satisfies every predeclared criterion; record FAIL when observed behavior violates any criterion and INCONCLUSIVE when required evidence is missing or uninspectable. Never weaken or reinterpret the rubric after observing output. Do not rewrite a failed trial as a pass; record a corrected candidate as a new trial.

Promote the skill only when structural validation passes and the behavioral evidence receives PASS under the frozen rubric.

For a revision-bound Codex contract canary, run ../../scripts/probe_codex_behavior.py manually against a clean committed revision and an explicit requested model. Keep the generated JSONL and private receipt.json outside every Git worktree. The probe also writes a strict allowlisted attestation.json bound to the receipt and raw-evidence digests; inspect it before explicitly posting or uploading it. Generation alone does not make the attestation external evidence. Promote is the gate: scripts/publish_codex_attestation.py re-hashes raw receipt.json and event streams; generation is still not external. External inspectability is the Actions artifact, not a Git blob. The probe checks bounded task behavior, observed mutation-capable commands, and response-shape conformance; it does not prove independent review, runtime enforcement, host-attested skill activation, every route, the resolved provider snapshot, or every possible transient side effect. Its producer timestamps are wall-clock observations, not trusted timestamps. Apply its result only to the recorded Codex version, requested model selection, Patpat revision, and Git tree. Do not transfer it to Cursor, Grok, Antigravity, or a later commit.

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 5d ago Changed · +4 lines 5268ef250503
  2. 9d ago First seen · 17 lines · 35 tokens per session scan A 3c5983c75542

Subscribe to this mod's changes

patpat-eval is a skill published in the GitHub repository goiltpatpat/patpat (1 stars, last pushed 3d ago), licensed MIT. It adds 35 tokens to every session and 570 once invoked, about $0.0002 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.