eval

A retrieval-accuracy checker for RagKit, a tool that searches a project's knowledge base through several search methods.

In plain words
What is it for?
Use it to run a known test set before or after changing search tokenization, weighting, or result-fusion settings, and compare the results by category.
Why use it?
It shows whether search changes improve or reduce the chance of finding the right information. Results include recall@k, which measures whether the answer appears in the first k results, and MRR, which rewards finding it near the top.

Skill for Claude CodeCodex

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

agentmods
npx agentmods add skills/qxbyte/pluginhub/eval
Any agent
npx skills add qxbyte/pluginhub --skill eval
Clone the repo
git clone --depth 1 https://github.com/qxbyte/pluginhub

Made for: Claude Code, Codex.

Per session 48 Skills are progressive disclosure: only the name and description are preloaded; the body loads when the skill is used.
When invoked 196 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 0 findings. Scan, not verified.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5 $0.00048 $0.00196
Opus 5 $0.00024 $0.00098
Sonnet 5 $0.00010 $0.00039
Haiku 4.5 $0.00005 $0.00020

Measured yesterday against content hash 2359c4734af8, method: parsed. Prices are Anthropic first-party input rates as of 2026-08-30, from the pricing page.

Security

Grade A, and why

eval scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured yesterday.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

plugins/ragkit/skills/eval/SKILL.md · 17 lines

What it actually says

RagKit Eval

脚本在本插件的 scripts/ 目录(本 skill 目录的上两级);用本 skill 的 base directory 把下面的相对路径拼成绝对路径执行。

sh ../../scripts/run.sh ../../scripts/ragkit.py \
   eval --kb <知识库路径> [--evalset <file>] [--channels lexical,metadata]
  • --channels lexical,metadata = 无向量基线;与全通道对比即向量路增益。
  • 任何检索参数调优(分词/权重/RRF)都必须先跑 eval 留对照数字。
Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. yesterday First seen · 17 lines · 48 tokens per session scan A 2359c4734af8

Subscribe to this mod's changes

eval is a skill published in the GitHub repository qxbyte/pluginhub (3 stars, last pushed 27d ago), licensed MIT. It adds 48 tokens to every session and 196 once invoked, about $0.0002 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.

Related

Other skills, from other repositories

douyin-favorites-to-knowledge

将用户已授权账号中的抖音视频收藏或用户明确指定的喜欢列表配置并同步到本地 Markdown 或 Obsidian 知识库;默认收藏,只有用户明确说喜欢/点赞才切换来源。首次明确选择推荐的 SiliconFlow转录、本地 Whisper 或不转录。不得绕过登录、访问他人账号或泄露 Cookie 与私密数据。.

tars1230/douyin-favorites-to-knowledge · 92 tokens

knowledge-extractor

Entrevista a un experto de dominio (SME) para extraer su conocimiento técnico tácito y sintetizarlo en una Skill reutilizable. Se activa PROACTIVAMENTE ante 3 situaciones: (1) Frustración — el usuario se queja de código, arquitectura o falta de estándares; (2) Ambigüedad sin Reglas — se pide un refactor profundo pero…

Vasallo94/obsidian-mcp-server · 156 tokens

MCP Developer

Skill para desarrollar, mantener y extender el servidor MCP de Obsidian. Incluye patrones de código, arquitectura, testing y gestión de paquetes.

Vasallo94/obsidian-mcp-server · 33 tokens

Python Patterns

Buenas prácticas y patrones de desarrollo Python para el proyecto MCP. Incluye estándares de código, patrones arquitectónicos, y convenciones.

Vasallo94/obsidian-mcp-server · 29 tokens

Refactoring

Guía para refactorizar código Python de forma segura y efectiva. Incluye técnicas de refactoring, detección de code smells, y mejoras.

Vasallo94/obsidian-mcp-server · 33 tokens

Test Runner

Skill para ejecutar y gestionar tests. Incluye patrones de testing, fixtures comunes, y estrategias de debugging de tests fallidos.

Vasallo94/obsidian-mcp-server · 28 tokens