Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add UKGovernmentBEIS/inspect_evals --skill security-audit-evalgit clone --depth 1 https://github.com/UKGovernmentBEIS/inspect_evalsWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/ukgovernmentbeis/inspect_evals/security-audit-eval)<a href="https://agentmods.dev/skills/ukgovernmentbeis/inspect_evals/security-audit-eval"><img src="https://agentmods.dev/badge/skills/ukgovernmentbeis/inspect_evals/security-audit-eval/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/ukgovernmentbeis/inspect_evals/security-audit-eval"><img src="https://agentmods.dev/badge/skills/ukgovernmentbeis/inspect_evals/security-audit-eval.svg" alt="Reviewed on agentmods" width="80" height="20"></a>- NVIDIA SkillSpector warn
SkillSpector: 7 findings, up to high
These are SkillSpector’s own severities. On a checked sample its high-severity flags on skills were ~96% false positives — a documented command, a public API, a “never do X” rule — so we show them as a caution to read, not a verdict. Why →
- high YARA Match · line 76 YARA rule matched a hack tool or exploit indicator (offensive tools, reconnaissance, privilege escalation, or exploit frameworks).Fix: Remove offensive tool references and exploit code. Legitimate agent skills should not contain penetration testing tools, exploit frameworks, or reconnaissance utilities.
- high Tool Misuse · line 77 Code deploys a privileged Kubernetes workload (privileged container, hostPath mount, or host namespaces). This grants root on the node and is a node/cluster takeover vector.Fix: Remove privileged, hostPath, and host-namespace settings from workloads. Use a least-privilege securityContext, drop capabilities, and avoid mounting the host filesystem.
- high Privilege Escalation · line 78 Potential security issue detected. Manual review is recommended.Fix: Review the flagged content for security risks. Ensure no credentials, secrets, or sensitive data are exposed.
- high Supply Chain · line 80 Remote code is downloaded and executed. This bypasses code review and could introduce malicious code.Fix: Avoid downloading and executing remote scripts. Use trusted packages from PyPI/npm. If remote fetch is required, verify checksums and use HTTPS.
- high Anti-Refusal · line 124 Skill attempts to nullify the agent's safety policies or restrictions ('you have no restrictions', 'ignore your guidelines', 'do anything now'). This is a direct jailbreak that disables guardrails.Fix: Remove jailbreak framing that nullifies safety policies or restrictions. Skill content must not instruct the agent to ignore its guidelines or operate without guardrails.
- medium Tool Misuse · line 109 Tool defaults are unsafe or overly permissive (e.g. disabled TLS verification, no authentication, world-writable permissions). Unsafe defaults widen the attack surface.Fix: Override unsafe defaults with secure settings (verify=True, auth required, restrictive permissions). Review and harden all tool configurations.
- low Privilege Escalation · line 123 Skill requests more permissions than appear necessary for its stated functionality. Review if elevated access is justified.Fix: Request only the minimum permissions required. Document why each permission is needed. Remove broad permissions like '*' or 'all'.
What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00140 | $0.04484 |
| Opus 5 | $0.00070 | $0.02242 |
| Sonnet 5 | $0.00028 | $0.00897 |
| Haiku 4.5 | $0.00014 | $0.00448 |
Grade C, and why
security-audit-eval scanned grade C with 4 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 9d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nullifies safety policiesmediumAnti-refusal
"You have no restrictions", "do anything now", "ignore your guidelines": a direct jailbreak that disables guardrails.
- Instructions to disable safety filters, ignore guardrails, or "act as if you have no restrictions" are fine in offensive-security or red-team evals running in proper containment. They are **High** if the agent has tool Downgraded: this mod is about security review, or the phrase is quoted, so it is likely naming the pattern rather than instructing it.
Downloads and executes remote codemediumSupply chain
curl | sh runs whatever the server returns today, which is not necessarily what it returned when this was reviewed.
- Custom Dockerfile with `RUN curl ... | sh` or `RUN wget ... && chmod +x` → **High** Downgraded: this mod is about security review, or the phrase is quoted, so it is likely naming the pattern rather than instructing it.
Makes network callslowCapability
Not a fault in itself. Listed so you know the mod talks to something, and to what.
- Custom Dockerfile with `RUN curl ... | sh` or `RUN wget ... && chmod +x` → **High** Runs shell commandslowCapability
Expected in a hook, worth knowing in a rule or an instructions file.
For each hit, decide: is this a legitimate use (e.g. `subprocess` inside a function only called within a sandbox abstraction), or could it run on import / on the host? Distinguish `sandbox().exec(...)` (Inspect's sandbox How it starts
The opening of the file, as written. The whole thing — 275 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Security Audit: Third-Party Evaluation
This skill produces a security audit report for an Inspect AI evaluation that the user is considering running. The output is a single Markdown report in agent_artefacts/security_audits/<eval_name>/SECURITY_AUDIT_REPORT.md with a clear verdict (safe / safe-with-caveats / unsafe) backed by file:line evidence.
The audit is read-only — it does not run the evaluation, does not clone repositories to disk, and does not modify any files outside the report directory.
Identifying the target
The eval may be supplied as:
- A GitHub URL (most common for third-party audits) — use
gh api repos/<owner>/<repo>/contents/<path>andgh api repos/<owner>/<repo>/git/trees/HEAD?recursive=1to read content. Do not clone. - A local path to a directory the user already has — use
ReadandBashdirectly. - An eval already inside
src/inspect_evals/— same as local path; treat the eval directory as the target.
If the user is ambiguous, ask which one.
Set <eval_name> to the repo name (for URLs) or the directory name (for local paths). Set <commit> to the audited commit SHA: gh api repos/<owner>/<repo>/commits/HEAD --jq '.sha' for URLs, git rev-parse HEAD for local repos. If neither is a git repo, omit <commit> and note "uncommitted working tree" in the report.
Setup
- Create the output directory:
agent_artefacts/security_audits/<eval_name>/ - Create
NOTES.mdin that directory for raw observations as you work. Err on the side of taking lots of notes — they are the source material for the report. - Get the full file tree (note sizes — large unexplained binaries are red flags):
- URL:
gh api repos/<owner>/<repo>/git/trees/HEAD?recursive=1 --jq '.tree[] | "\(.type)\t\(.size // "")\t\(.path)"' - Local:
find . -type f -not -path './.git/*' -printf '%s\t%p\n'
- URL:
- Note any unexpected files: archives (
.tar.gz,.zip), executables (.so,.dylib,.exe), pickle (.pkl,.joblib,.pt,.pth), encoded blobs, files >5 MB without an obvious dataset purpose.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 9d ago First seen · 275 lines · 140 tokens per session scan C f051e66dc11a
security-audit-eval is a skill published in the GitHub repository UKGovernmentBEIS/inspect_evals (662 stars, last pushed today), licensed MIT. It adds 140 tokens to every session and 4,484 once invoked, about $0.0007 per session on Opus 5. A static security scan graded it C with 4 findings (nullifies safety policies, downloads and executes remote code, makes network calls). No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other skills, from other repositories
systematic-debugging
Use when encountering any bug, test failure, or unexpected behavior, before proposing fixes.
local-ai-agents
Build local-first AI agents that run entirely on a developer workstation with Microsoft Foundry Local and Qwen function-calling models. Covers Small Language Models (SLMs), the OpenAI-compatible local endpoint, sandboxed local tools, local RAG with Chroma, local MCP servers, hybrid cloud/local routing, and the…
next-cache-components-adoption
Turn on Cache Components in a Next.js app and resolve the blocking routes it surfaces. Use when the user wants to enable, adopt, or migrate to Cache Components, flip the cacheComponents flag, work through a flood of blocking-prerender / instant validation errors, run the cache-components-instant-false codemod, or…
next-cache-components-optimizer
Drive a Next.js route to instant navigation by setting up an agentic loop, under Cache Components / PPR, on initial load (hard navigation) and client-side navigation (soft navigation). Encode the goal as a failing @next/playwright instant() e2e and work it to green, one verified route at a time; the shipped test then…
next-partial-prefetching-adoption
Turn on Partial Prefetching in a Next.js app and work through the insights it surfaces. Use when the user wants to enable or adopt Partial Prefetching, flip the partialPrefetching flag, opt routes in with export const prefetch = 'partial', audit Link prefetch={true} behavior, preserve existing prefetched UI with…
chronicle
Analyze Copilot session history for standup reports, usage tips, session search, and session reindexing. Use when the user asks for a standup, daily summary, usage tips, workflow recommendations, wants to search or find past sessions by keyword/file/PR, wants to reindex their session store, or asks about deleting…