Getting it into your agent
It runs from inside its repository, so the clone comes first — what it calls does not travel with the file alone.
git clone --depth 1 https://github.com/rjmurillo/ai-agentsnpx agentmods add skills/rjmurillo/ai-agents/ai-agents-empirical-probe-toolkitWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/rjmurillo/ai-agents/ai-agents-empirical-probe-toolkit)<a href="https://agentmods.dev/skills/rjmurillo/ai-agents/ai-agents-empirical-probe-toolkit"><img src="https://agentmods.dev/badge/skills/rjmurillo/ai-agents/ai-agents-empirical-probe-toolkit/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/rjmurillo/ai-agents/ai-agents-empirical-probe-toolkit"><img src="https://agentmods.dev/badge/skills/rjmurillo/ai-agents/ai-agents-empirical-probe-toolkit.svg" alt="Reviewed on agentmods" width="80" height="20"></a>- NVIDIA SkillSpector warn
SkillSpector: 2 findings, up to high
These are SkillSpector’s own severities. On a checked sample its high-severity flags on skills were ~96% false positives — a documented command, a public API, a “never do X” rule — so we show them as a caution to read, not a verdict. Why →
- high Excessive Agency · line 101 Skill selects an external model or provider that may use a different account or billing plan than the operator expects. Undisclosed model switches can cause unexpected cost or quota consumption.Fix: Remove the model/provider override or disclose it prominently and require explicit operator approval before invoking an external coding CLI or billed model.
- medium Excessive Agency · line 163 Skill enables autonomous high-impact decisions without human-in-the-loop verification. Critical operations (destructive commands, financial transactions, data deletion) should require explicit user confirmation.Fix: Add human-in-the-loop confirmation for destructive, irreversible, or high-impact operations. Never auto-execute commands that modify files, send data, or alter system state.
What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00119 | $0.03491 |
| Opus 5 | $0.00060 | $0.01746 |
| Sonnet 5 | $0.00024 | $0.00698 |
| Haiku 4.5 | $0.00012 | $0.00349 |
Grade A, and why
ai-agents-empirical-probe-toolkit scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 8d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 183 lines — stays where its author put it; the contents beside it link to each section on GitHub.
AI Agents Empirical Probe Toolkit
This skill is the method library for replacing assumption with measurement. Every recipe exists because an assumption shipped here and became an incident. The house rule, from the #2290 payload-format incident: the cost of a probe is 15 minutes; the cost of assumption is a P0 (.serena/memories/copilot-hooks-observations.md).
Two terms used throughout:
- Probe: a minimal disposable artifact (a hook that dumps its environment, a script that captures stdin) run against the real pinned tool to observe what the tool actually does, instead of what its docs say.
- Negative control: a deliberately broken input run through the same harness as the passing case, proving the test or probe CAN fail. A check that cannot fail proves nothing.
Triggers
probe the runtime contractcalibrate this guardprove it empiricallyadd a negative controldocs versus reality audit
Recipe Selector
| Situation | Recipe | Related skill |
|---|---|---|
| You are about to rely on undocumented tool behavior (cwd, env vars, stdin format) | 1. Runtime-contract probe | agent-harness-reference for settled contracts |
| You are shipping a detector, guard, or numeric threshold | 2. Guard/threshold calibration | ship your own post-ship telemetry; no shared monitor exists (ADR-084, issue #5154) |
| You changed a prompt, rule, or agent and claim it behaves better | 3. Behavioral A/B via eval harness | benchmark-models for cross-model comparison |
| You are about to write a command, path, or "matches X" claim into a doc or docstring | 4. Docs-vs-reality audit | doc-accuracy for full doc audits |
| A CI job failed on your PR | 5. Reproduce-on-main discriminator | ai-agents-debugging-playbook for symptom triage |
| You are writing a test for a generated artifact or contract | 6. Negative-control test design | ai-agents-validation-and-qa for the evidence bar |
What ships with it
3 files beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 8d ago First seen · 183 lines · 119 tokens per session scan A acb5f3ee4042
ai-agents-empirical-probe-toolkit is a skill published in the GitHub repository rjmurillo/ai-agents (45 stars, last pushed yesterday), licensed MIT. It adds 119 tokens to every session and 3,491 once invoked, about $0.0006 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-09-03.
Other skills, from other repositories
ios-simulator
Verify and debug native, React Native, Expo, or Flutter apps on an iOS Simulator with agent-device. Use when an agent needs to launch an app, inspect its live UI, tap, type, scroll, validate a code change, collect failure evidence, or reproduce a workflow on an iPhone or iPad Simulator.
diagnostic-first-refactoring
Analyze codebase structure before making changes — the "Surgeon's Scan" pattern.
debugging
A structured method for finding and fixing software bugs. It starts by writing a test that reproduces the failure, then investigates its underlying cause before making a small fix.
backpropagation
Trace runtime bugs back to spec gaps — identify missing acceptance criteria, update specs, generate regression tests, and detect patterns.
staged-transformation-pipeline
Monolithic transformations are hard to test and debug.
webapp-testing
Toolkit for interacting with and testing local web applications using Playwright. Supports verifying frontend functionality, debugging UI behavior, capturing browser screenshots, and viewing browser logs.