Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add zjunlp/Mechanist --skill experiment-auditgit clone --depth 1 https://github.com/zjunlp/MechanistWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/zjunlp/mechanist/experiment-audit)<a href="https://agentmods.dev/skills/zjunlp/mechanist/experiment-audit"><img src="https://agentmods.dev/badge/skills/zjunlp/mechanist/experiment-audit.svg" alt="Measured on agentmods" height="20"></a>- NVIDIA SkillSpector warn
SkillSpector: 9 findings, up to high
These are SkillSpector’s own severities. On a checked sample its high-severity flags on skills were ~96% false positives — a documented command, a public API, a “never do X” rule — so we show them as a caution to read, not a verdict. Why →
- high Agent Snooping · line 53 Skill accesses MCP server configuration files (mcp.json). MCP configs contain server URLs, authentication tokens, and tool definitions — reading them allows the skill to discover and potentially abuse other tool integrations.Fix: Remove all code or instructions that read MCP configuration files (mcp.json). MCP server details should be managed by the agent runtime, not read by individual skills.
- high Agent Snooping · line 54 Skill reads from agent configuration directories (.claude/, .codex/, .gemini/). These directories may contain API keys, personal settings, and other credentials that the skill has no legitimate need to access.Fix: Remove all code or instructions that access agent configuration directories (.claude/, .codex/, .gemini/). If configuration values are needed, pass them explicitly as parameters or environment variabl
- high Agent Snooping · line 66 Skill reads from agent configuration directories (.claude/, .codex/, .gemini/). These directories may contain API keys, personal settings, and other credentials that the skill has no legitimate need to access.Fix: Remove all code or instructions that access agent configuration directories (.claude/, .codex/, .gemini/). If configuration values are needed, pass them explicitly as parameters or environment variabl
- high Agent Snooping · line 67 Skill reads from agent configuration directories (.claude/, .codex/, .gemini/). These directories may contain API keys, personal settings, and other credentials that the skill has no legitimate need to access.Fix: Remove all code or instructions that access agent configuration directories (.claude/, .codex/, .gemini/). If configuration values are needed, pass them explicitly as parameters or environment variabl
- high Agent Snooping · line 68 Skill reads from agent configuration directories (.claude/, .codex/, .gemini/). These directories may contain API keys, personal settings, and other credentials that the skill has no legitimate need to access.Fix: Remove all code or instructions that access agent configuration directories (.claude/, .codex/, .gemini/). If configuration values are needed, pass them explicitly as parameters or environment variabl
- high Agent Snooping · line 69 Skill reads from agent configuration directories (.claude/, .codex/, .gemini/). These directories may contain API keys, personal settings, and other credentials that the skill has no legitimate need to access.Fix: Remove all code or instructions that access agent configuration directories (.claude/, .codex/, .gemini/). If configuration values are needed, pass them explicitly as parameters or environment variabl
- high Agent Snooping · line 70 Skill reads from agent configuration directories (.claude/, .codex/, .gemini/). These directories may contain API keys, personal settings, and other credentials that the skill has no legitimate need to access.Fix: Remove all code or instructions that access agent configuration directories (.claude/, .codex/, .gemini/). If configuration values are needed, pass them explicitly as parameters or environment variabl
- high Agent Snooping · line 78 Skill reads from agent configuration directories (.claude/, .codex/, .gemini/). These directories may contain API keys, personal settings, and other credentials that the skill has no legitimate need to access.Fix: Remove all code or instructions that access agent configuration directories (.claude/, .codex/, .gemini/). If configuration values are needed, pass them explicitly as parameters or environment variabl
- medium Rogue Agent · line 36 Skill establishes unauthorized persistence across sessions via cron jobs, startup scripts, or state files. Session persistence allows an attacker to maintain access beyond the current interaction.Fix: Remove any persistence mechanisms (cron jobs, startup scripts, state files). Skills should not maintain state across sessions without explicit user consent.
What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00115 | $0.04347 |
| Opus 5 | $0.00057 | $0.02174 |
| Sonnet 5 | $0.00023 | $0.00869 |
| Haiku 4.5 | $0.00012 | $0.00435 |
Grade A, and why
experiment-audit scanned grade A with 2 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 8d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Reads agent configuration directorieslowAgent snooping
.claude/, .codex/, .gemini/ hold keys, settings and other credentials a mod has no legitimate need for.
2. **User MCP config** — `~/.claude/settings.json`, same field. Downgraded: this mod is about security review, or the phrase is quoted, so it is likely naming the pattern rather than instructing it.
Reads MCP configurationlowAgent snooping
mcp.json carries server URLs and auth tokens; reading it lets a mod discover and abuse other integrations.
1. **Project MCP config** — `${PROJECT_ROOT}/.mcp.json`, field `mcpServers["llm-chat"].env.{LLM_MODEL,LLM_BASE_URL,LLM_API_KEY}`. Downgraded: this mod is about security review, or the phrase is quoted, so it is likely naming the pattern rather than instructing it.
How it starts
The opening of the file, as written. The whole thing — 354 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Experiment Audit: Per-Claim Cross-Model Integrity Verification
Audit the experimental process for one claim: $ARGUMENTS
Why This Exists
LLM agents can produce fraudulent experimental results through:
- Fake ground truth — creating synthetic "reference" from model outputs, then reporting high agreement as performance
- Score normalization — dividing metrics by the model's own max to get 0.99+
- Phantom results — claiming numbers from files that don't exist or functions never called
- Insufficient scope — reporting 2-scene pilots as "comprehensive evaluation"
These are NOT intentional deception — they are failure modes of optimizing agents that lack integrity constraints. This skill adds that constraint, once per claim: each invocation scopes to one claim's runs and returns a PASS/WARN/FAIL verdict on that claim's experimental process.
Core Principle
The executor (Claude) collects file paths scoped to the target claim. The external LLM reviewer reads code and judges integrity. The executor does NOT participate in integrity judgment.
This follows shared-references/reviewer-independence.md and shared-references/experiment-integrity.md.
Constants
- REVIEWER_BACKEND =
llm-chat— External LLM reviewer via llm-chat MCP (model defers toLLM_MODELenv). Always ask the external reviewer for strict, high-rigor feedback. Override with— reviewer: oracle-provia Oracle MCP.
Arguments
<experiment-dir-or-results-path>(positional, required) — directory containing the experiment artifacts to audit. Typicallyrefine-logs/(for main-experiment audit) orverify/<claim_dir>/variants/(for variant audit).— claim <Cx>(required) — the claim whose experimental process is being audited. The skill is inherently per-claim: each invocation produces ONE PASS/WARN/FAIL verdict for the named claim, derived from auditing only that claim's linked milestones/runs. To audit N claims, call the skill N times.— output-dir <path>(optional) — directory to writeEXPERIMENT_AUDIT.{md,json}into. Defaults to current working directory. When given, the skill creates<path>if needed and writes directly to<path>/EXPERIMENT_AUDIT.mdand<path>/EXPERIMENT_AUDIT.json; callers do not need a follow-upmv.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 8d ago First seen · 354 lines · 115 tokens per session scan A 85640ecd9cf2
experiment-audit is a skill published in the GitHub repository zjunlp/Mechanist (72 stars, last pushed 12d ago), licensed MIT. It adds 115 tokens to every session and 4,347 once invoked, about $0.0006 per session on Opus 5. A static security scan graded it A with 2 findings (reads agent configuration directories, reads mcp configuration). No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other skills, from other repositories
nanoresearch-writing
Draft a LaTeX research paper from all previous stage outputs.
nanoresearch-experiment
Generate a Python code skeleton from an experiment blueprint.
nanoresearch-ideation
Search academic literature and generate research hypotheses.
owasp-top-10-testing
Test an application against the OWASP Top 10 with Strix — autonomous AI agents that attempt real exploits for each category of the current OWASP Top 10:2025 (broken access control including SSRF, security misconfiguration, software supply chain failures, cryptographic failures, injection, insecure design…
application-security-testing
Application security testing (AppSec) across a whole product with Strix — decide which asset needs which test (source code, running web app, API, CI pipeline), run it, and turn the results into a ranked remediation plan. Autonomous agents exploit and prove each issue instead of emitting static-analysis alerts, so the…
potpie-debug-memory
Use while debugging or troubleshooting failures, flaky tests, incidents, production alerts, CI failures, local dev setup issues, repeated bugs, prior fixes, failed attempts, and verification history.