Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx agentmods add skills/mathews-tom/armory/rag-auditornpx skills add Mathews-Tom/armory --skill rag-auditorgit clone --depth 1 https://github.com/Mathews-Tom/armoryWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/mathews-tom/armory/rag-auditor)<a href="https://agentmods.dev/skills/mathews-tom/armory/rag-auditor"><img src="https://agentmods.dev/badge/skills/mathews-tom/armory/rag-auditor.svg" alt="Measured on agentmods" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00076 | $0.01966 |
| Opus 5 | $0.00038 | $0.00983 |
| Sonnet 5 | $0.00015 | $0.00393 |
| Haiku 4.5 | $0.00008 | $0.00197 |
Grade A, and why
rag-auditor scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured today.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 191 lines — stays where its author put it; the contents beside it link to each section on GitHub.
RAG Auditor
Systematic RAG pipeline evaluation across the full retrieval-generation chain: designs evaluation query sets, measures retrieval metrics (Precision@K, Recall@K, MRR), evaluates generation quality (groundedness, completeness, hallucination rate), diagnoses component-level failures, and recommends targeted improvements.
Reference Files
| File | Contents | Load When |
|---|---|---|
references/retrieval-metrics.md |
Precision@K, Recall@K, MRR, NDCG definitions and calculation | Always |
references/generation-metrics.md |
Groundedness, completeness, hallucination detection methods | Generation evaluation needed |
references/failure-taxonomy.md |
RAG failure categories: retrieval, generation, chunking, embedding | Failure diagnosis needed |
references/diagnostic-queries.md |
Designing evaluation query sets, known-answer questions, difficulty levels | Evaluation setup |
Prerequisites
- Access to the RAG pipeline (or its outputs for post-hoc evaluation)
- A set of test queries with known-correct answers
- Understanding of the pipeline components (embedding model, retriever, generator)
Workflow
Phase 1: Pipeline Inventory
Document the RAG pipeline configuration:
- Document source — What documents are indexed? Format, count, size.
- Chunking — Strategy (fixed-size, semantic, paragraph), chunk size, overlap.
- Embedding — Model name and version, dimensionality.
- Vector store — Type (FAISS, Pinecone, Chroma, pgvector), index type.
- Retrieval — Method (similarity, hybrid, reranking), top-K parameter.
- Generation — Model, prompt template, context window usage.
Phase 2: Design Evaluation Queries
What ships with it
5 files beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- today First seen · 191 lines · 76 tokens per session scan A 633f6f5730a2
rag-auditor is a skill published in the GitHub repository Mathews-Tom/armory (316 stars, last pushed today), licensed MIT. It adds 76 tokens to every session and 1,966 once invoked, about $0.0004 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-09-05.
Other skills, from other repositories
agent-platform-rag-engine-management
Manage and query Agent Platform RAG Engine Corpora and retrieve grounded contexts using the Google GenAI SDK. Use when listing RAG corpora or files, inspecting a corpus, retrieving contexts, or generating content grounded in a RAG corpus. Do not use for standard database queries (use SQL/Spanner skills), Google…
graphify
Use for any question about a codebase, its architecture, file relationships, or project content — especially when graphify-out/ exists, where the question should be treated as a graphify query first. Turns any input (code, docs, papers, images, videos) into a persistent knowledge graph with god nodes, community…
google-cloud-solution-rag-enterprise-search-gke-sqldb
Discovers requirements, and generates architectural, design, and deployment guidance for a retrieval-augmented generation (RAG)-capable enterprise search system in Google Cloud. Use when users need a vector-enabled SQL database as the store and index for the embedding vectors, an open model and open-source inferencing…
jetson-inference-mem-tune
Pick the serving stack and per-runtime memory flags (vLLM, SGLang, llama.cpp, TensorRT Edge-LLM) for an LLM/VLM workload on any NVIDIA Jetson.
gpt-image-2
面向 GPT Image 2 的图像生成 / 编辑技能。可在 3 种环境下使用：(A) Garden 本地模式，通过 OpenAI 兼容接口直接出图并落盘；(B) Host-Native 模式，把本 Skill 当作提示词工程指引，把渲染好的 prompt 交给宿主 Agent 自带的图像工具出图；(C) Advisor 模式，宿主无任何图像工具时退化为高质量 prompt 顾问。涵盖 18 大类、80+ 个结构化模板，覆盖海报 / UI / 产品 / 信息图 / 学术图 / 技术架构图 / 漫画 / 头像 / 流程板 / 电影分镜 / IP 周边 / 编辑工作流等场景。.
kb-retriever
面向本地知识库目录的检索和问答助手。核心流程：(1)分层索引导航 (2)遇到PDF/Excel时必须先读取references学习处理方法 (3)处理文件后再检索。按文件类型组合使用 grep、Read、pdfplumber、pandas 进行渐进式检索，避免整文件加载。用户问题涉及"从知识库目录回答问题/检索信息/查资料"时使用。.