eval

eval is a skill for Claude Code from qxbyte/pluginhub. It costs 48 tokens per session (196 once invoked), scanned A, original, MIT.

A retrieval-accuracy checker for RagKit, a tool that searches a project's knowledge base through several search methods.

In plain words
What is it for?
Use it to run a known test set before or after changing search tokenization, weighting, or result-fusion settings, and compare the results by category.
Why use it?
It shows whether search changes improve or reduce the chance of finding the right information. Results include recall@k, which measures whether the answer appears in the first k results, and MRR, which rewards finding it near the top.

Skill for Claude Code

Written for Claude Code: shipped in a Claude Code plugin.

Needs its repository: it reads a path above its own folder, which exists only inside the repository. The line is sh ../../scripts/run.sh ../../scripts/ragkit.py \.

Part of the ragkit plugin — 5 skills, 1 hook shipped together

Good fit Use it to run a known test set before or after changing search tokenization, weighting, or result-fusion settings, and compare the results by category.

Compare 6 skills from other repositories ↓
Install

Getting it into your agent

It runs from inside its repository, so the clone comes first — what it calls does not travel with the file alone.

Clone the repo
git clone --depth 1 https://github.com/qxbyte/pluginhub
agentmods
npx agentmods add skills/qxbyte/pluginhub/eval

Made for: Claude Code.

Or install ragkit, the plugin that ships this one along with the rest of its 5 skills, 1 hook.

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

agentmods badge for eval

README.md
[![agentmods](https://agentmods.dev/badge/skills/qxbyte/pluginhub/eval/github.svg)](https://agentmods.dev/skills/qxbyte/pluginhub/eval)
Your own site
<a href="https://agentmods.dev/skills/qxbyte/pluginhub/eval"><img src="https://agentmods.dev/badge/skills/qxbyte/pluginhub/eval/github.svg" alt="Measured on agentmods" height="20"></a>

Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.

agentmods 80×15 button for eval

Your own site · 80×15
<a href="https://agentmods.dev/skills/qxbyte/pluginhub/eval"><img src="https://agentmods.dev/badge/skills/qxbyte/pluginhub/eval.svg" alt="Reviewed on agentmods" width="80" height="20"></a>
Per session 48 Skills are progressive disclosure: only the name and description are preloaded; the body loads when the skill is used.
When invoked 196 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 0 findings. A grade says what 26 rules found in the file — not that it is safe.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5.1 $0.00048 $0.00196
Opus 5.5 $0.00019 $0.00078
Sonnet 5.5 $0.00010 $0.00039
Haiku 4.5 $0.00005 $0.00020

Measured 1mo ago against content hash 2359c4734af8, method: parsed. Prices are Anthropic first-party input rates as of 2026-10-07, from the pricing page.

Security

Grade A, and why

eval scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 1mo ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

plugins/ragkit/skills/eval/SKILL.md · 17 lines

What it actually says

RagKit Eval

脚本在本插件的 scripts/ 目录(本 skill 目录的上两级);用本 skill 的 base directory 把下面的相对路径拼成绝对路径执行。

sh ../../scripts/run.sh ../../scripts/ragkit.py \
   eval --kb <知识库路径> [--evalset <file>] [--channels lexical,metadata]
  • --channels lexical,metadata = 无向量基线;与全通道对比即向量路增益。
  • 任何检索参数调优(分词/权重/RRF)都必须先跑 eval 留对照数字。
Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 1mo ago First seen · 17 lines · 48 tokens per session scan A 2359c4734af8

Subscribe to this mod's changes

eval is a skill published in the GitHub repository qxbyte/pluginhub (3 stars, last pushed 25d ago), licensed MIT. It adds 48 tokens to every session and 196 once invoked, about $0.0002 per session on Opus 5.5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.

Related

Other skills, from other repositories

index

Use when the user wants to index their vault for semantic search, rebuild the RAG index, or says "index", "reindex", or "rebuild index".

thoreinstein/obsidian-rag · 36 tokens

mcp-builder

DEPRECATED: This skill has been replaced by mcp-app-builder. Check if mcp-app-builder is available in the skills folder. If not, install it: npx skills install mcp-use/mcp-use --skill mcp-app-builder Use mcp-app-builder instead of this skill. Build Model Context Protocol (MCP) servers with mcp-use framework. Use when…

Shubhamsaboo/awesome-llm-apps · 123 tokens

thinking-out-loud

A contract for what the agent does when a long, messy, stream-of-consciousness ramble arrives (usually voice dictation): act on nothing until the echo brief is approved. The echo audits the entire transfer, mission, locked decisions and constraints, open questions, flips and parked tangents, with the model's…

Shubhamsaboo/awesome-llm-apps · 206 tokens

xberg-chunking

Use when splitting extracted text into chunks for LLM context windows or RAG ingestion. Covers chunk size, overlap, markdown/yaml/semantic chunkers, tokenizer-based sizing, and the standalone chunk command.

xberg-io/xberg · 43 tokens

xberg-extracting-keywords

Use when extracting keywords (YAKE/RAKE) from documents — and, secondarily, when detecting document language or generating embeddings for RAG and search. Covers the keyword config (and its feature gating), --detect-language, and the standalone embed command with real flags.

xberg-io/xberg · 58 tokens

xberg-picking-a-format

Use when choosing an output format for extracted documents — plain text, markdown, djot, HTML, JSON, or DocTags. Maps consumer (LLM, parser, archive) to the right --format / --content-format pair.

xberg-io/xberg · 51 tokens