Getting it into your agent
It runs from inside its repository, so the clone comes first — what it calls does not travel with the file alone.
git clone --depth 1 https://github.com/rjmurillo/ai-agentsnpx agentmods add skills/rjmurillo/ai-agents/ai-agents-empirical-probe-toolkitWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/rjmurillo/ai-agents/ai-agents-empirical-probe-toolkit)<a href="https://agentmods.dev/skills/rjmurillo/ai-agents/ai-agents-empirical-probe-toolkit"><img src="https://agentmods.dev/badge/skills/rjmurillo/ai-agents/ai-agents-empirical-probe-toolkit/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/rjmurillo/ai-agents/ai-agents-empirical-probe-toolkit"><img src="https://agentmods.dev/badge/skills/rjmurillo/ai-agents/ai-agents-empirical-probe-toolkit.svg" alt="Reviewed on agentmods" width="80" height="20"></a>- NVIDIA SkillSpector warn
SkillSpector: 2 findings, up to high
These are SkillSpector’s own severities. On a checked sample its high-severity flags on skills were ~96% false positives — a documented command, a public API, a “never do X” rule — so we show them as a caution to read, not a verdict. Why →
- high Excessive Agency · line 101 Skill selects an external model or provider that may use a different account or billing plan than the operator expects. Undisclosed model switches can cause unexpected cost or quota consumption.Fix: Remove the model/provider override or disclose it prominently and require explicit operator approval before invoking an external coding CLI or billed model.
- medium Excessive Agency · line 163 Skill enables autonomous high-impact decisions without human-in-the-loop verification. Critical operations (destructive commands, financial transactions, data deletion) should require explicit user confirmation.Fix: Add human-in-the-loop confirmation for destructive, irreversible, or high-impact operations. Never auto-execute commands that modify files, send data, or alter system state.
What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00119 | $0.03569 |
| Opus 5.5 | $0.00048 | $0.01428 |
| Sonnet 5.5 | $0.00024 | $0.00714 |
| Haiku 4.5 | $0.00012 | $0.00357 |
Grade A, and why
ai-agents-empirical-probe-toolkit scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 15d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 190 lines — stays where its author put it; the contents beside it link to each section on GitHub.
AI Agents Empirical Probe Toolkit
This skill is the method library for replacing assumption with measurement. Every recipe exists because an assumption shipped here and became an incident. The house rule, from the #2290 payload-format incident: the cost of a probe is 15 minutes; the cost of assumption is a P0 (.serena/memories/copilot-hooks-observations.md).
Two terms used throughout:
- Probe: a minimal disposable artifact (a hook that dumps its environment, a script that captures stdin) run against the real pinned tool to observe what the tool actually does, instead of what its docs say.
- Negative control: a deliberately broken input run through the same harness as the passing case, proving the test or probe CAN fail. A check that cannot fail proves nothing.
Triggers
probe the runtime contractcalibrate this guardprove it empiricallyadd a negative controldocs versus reality audit
Recipe Selector
| Situation | Recipe | Related skill |
|---|---|---|
| You are about to rely on undocumented tool behavior (cwd, env vars, stdin format) | 1. Runtime-contract probe | agent-harness-reference for settled contracts |
| You are shipping a detector, guard, or numeric threshold | 2. Guard/threshold calibration | ship your own post-ship telemetry; no shared monitor exists (ADR-084, issue #5154) |
| You changed a prompt, rule, or agent and claim it behaves better | 3. Behavioral A/B via eval harness | benchmark-models for cross-model comparison |
| You are about to write a command, path, or "matches X" claim into a doc or docstring | 4. Docs-vs-reality audit | doc-accuracy for full doc audits |
| A CI job failed on your PR | 5. Reproduce-on-main discriminator | ai-agents-debugging-playbook for symptom triage |
| You are writing a test for a generated artifact or contract | 6. Negative-control test design | ai-agents-validation-and-qa for the evidence bar |
What ships with it
2 files beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 15d ago Changed · +1 lines a0efe2e4d6df
- 26d ago Changed a10fc6c80495
- 1mo ago Changed · +6 lines 3eff413eca82
- 1mo ago Changed fec95bdd2508
- 1mo ago Changed 5c1acda016af
- 1mo ago First seen · 183 lines · 119 tokens per session scan A acb5f3ee4042
ai-agents-empirical-probe-toolkit is a skill published in the GitHub repository rjmurillo/ai-agents (49 stars, last pushed today), licensed MIT. It adds 119 tokens to every session and 3,569 once invoked, about $0.0005 per session on Opus 5.5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-09-03.
Other skills, from other repositories
analyze-failures
Triage Katalon True Platform/TestOps test failures and file defects. Use when you need to investigate failed test results, classify each failure as product defect vs automation defect vs environment/data issue, cluster failures by common signature, find likely root cause from execution data, and optionally create…
review
Review code changes, pull requests, patches, or a scoped code area for actionable correctness, security, compatibility, and test risks with file and line evidence. Use for review or audit requests; do not use for general proofreading, feature implementation, or debugging a reported failure when the user wants a fix.
debug
Diagnose a concrete failure, regression, crash, hang, flaky test, or incorrect runtime behavior by reproducing it, testing hypotheses, and identifying the evidence-backed root cause. Use when symptoms or failing output exist; do not use for feature implementation without a failure, general code review, or a conceptual…
phoenix-review
Review completed work against the Intent Contract's acceptance criteria using objective evidence, not opinion — re-run every check, confirm no regressions, inspect the tamper-evident trace, and surface only real issues (bugs, unmet criteria, regressions). Use before shipping, after a feature is built, or when the user…
extension-audit-methodology
Systematic 5-dimension audit framework for VS Code extensions — debug hygiene, dead code, performance, menu validation, dependency cleanup.
staged-transformation-pipeline
Monolithic transformations are hard to test and debug.