Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add adriannoes/awesome-agentic-ai --skill orchestrating-llm-attacks-with-pyritgit clone --depth 1 https://github.com/adriannoes/awesome-agentic-aiWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/adriannoes/awesome-agentic-ai/orchestrating-llm-attacks-with-pyrit)<a href="https://agentmods.dev/skills/adriannoes/awesome-agentic-ai/orchestrating-llm-attacks-with-pyrit"><img src="https://agentmods.dev/badge/skills/adriannoes/awesome-agentic-ai/orchestrating-llm-attacks-with-pyrit/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/adriannoes/awesome-agentic-ai/orchestrating-llm-attacks-with-pyrit"><img src="https://agentmods.dev/badge/skills/adriannoes/awesome-agentic-ai/orchestrating-llm-attacks-with-pyrit.svg" alt="Reviewed on agentmods" width="80" height="20"></a>- NVIDIA SkillSpector warn
SkillSpector: 2 findings, up to high
These are SkillSpector’s own severities. On a checked sample its high-severity flags on skills were ~96% false positives — a documented command, a public API, a “never do X” rule — so we show them as a caution to read, not a verdict. Why →
- high YARA Match · line 2 YARA rule matched a hack tool or exploit indicator (offensive tools, reconnaissance, privilege escalation, or exploit frameworks).Fix: Remove offensive tool references and exploit code. Legitimate agent skills should not contain penetration testing tools, exploit frameworks, or reconnaissance utilities.
- medium Data Exfiltration · line 95 Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.Fix: Verify the destination URL is trusted and necessary. Remove or replace with documented APIs. Ensure no secrets, tokens, or PII are transmitted.
What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00056 | $0.02603 |
| Opus 5 | $0.00028 | $0.01301 |
| Sonnet 5 | $0.00011 | $0.00521 |
| Haiku 4.5 | $0.00006 | $0.00260 |
Grade A, and why
orchestrating-llm-attacks-with-pyrit scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 7d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 226 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Orchestrating LLM Attacks with PyRIT
Legal and Authorized-Use Notice: PyRIT generates adversarial and potentially harmful prompts to test AI systems. Use it only against models and endpoints you own or are explicitly authorized to assess. Multi-turn orchestrators consume large numbers of tokens against both the target and the adversarial/scoring models; account for cost and terms of service. Unauthorized use is prohibited.
Overview
PyRIT (Python Risk Identification Tool for generative AI) is an open-source automation framework from Microsoft's AI Red Team, distributed at github.com/microsoft/PyRIT. Where a single-shot scanner sends one prompt and checks the answer, PyRIT automates multi-turn adversarial conversations: an attacker model and a scorer model collaborate in a loop to drive a target model toward a defined objective (for example, eliciting restricted content, leaking a system prompt, or making an agent perform an unauthorized tool call). This mirrors how real adversaries iterate against a chatbot rather than relying on one magic prompt.
PyRIT is built from composable primitives. Targets (pyrit.prompt_target) wrap the systems being probed and the helper models — OpenAIChatTarget, AzureMLChatTarget, HTTPTarget, and others. Orchestrators / attacks (pyrit.orchestrator) implement attack strategies; all multi-turn strategies subclass MultiTurnOrchestrator. The headline strategies are RedTeamingOrchestrator (a generic adversarial-chat loop), CrescendoOrchestrator (the Crescendo technique — start benign and escalate gradually so each turn looks reasonable in isolation), and TreeOfAttacksWithPruningOrchestrator (TAP — branch multiple attack lines in parallel, expand the branches the scorer rates as progressing, and prune dead ends). Scorers (pyrit.score) such as SelfAskTrueFalseScorer decide whether the objective was met and feed that judgment back into the loop. Converters mutate prompts (base64, translation, ASCII art) to evade filters, and memory persists every turn for later analysis.
What ships with it
4 files beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 7d ago First seen · 226 lines · 56 tokens per session scan A 16212f596b6d
orchestrating-llm-attacks-with-pyrit is a skill published in the GitHub repository adriannoes/awesome-agentic-ai (57 stars, last pushed 13d ago), licensed MIT. It adds 56 tokens to every session and 2,603 once invoked, about $0.0003 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-09-03.
Other skills, from other repositories
agent-safety
Use when bounding an LLM agent that already runs — scoping its task domain, gating tools to least privilege, defending against prompt injection in untrusted web/email/RAG text, requiring human approval on irreversible actions, capping runtime and cost, or triaging what it already did. NOT building the loop, tools, or…
llm-prompt-injection
Use when testing an authorized LLM application for prompt injection, system-prompt exposure, unsafe tool use, or RAG data-boundary failures.
opfor-run
Run red-team attacks and generate a report for an agent target.
opfor-setup
Set up an agent or chatbot target for Opfor red-teaming.
ai-security-review
Activate when the user asks to audit, review, scan, or assess the security of an AI application, agent, chatbot, or LLM-powered system. Also activate when the user mentions prompt injection, data exfiltration, guardrail bypass, red-teaming, AI SBOM, cognitive policy, OWASP LLM Top 10, NIST AI RMF, or EU AI Act…
sbom-analysis
Activate when the user opens, mentions, or asks questions about a .sbom.json file, an AI Bill of Materials, an aibom.json, or asks about what AI components an application uses. Also activate when the user asks about component dependencies, LLM model usage, tool permissions, datastore access, or the attack surface of…