Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add nimadorostkar/Claude-Skills-collection --skill llm-evaluationgit clone --depth 1 https://github.com/nimadorostkar/Claude-Skills-collectionWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/nimadorostkar/claude-skills-collection/llm-evaluation)<a href="https://agentmods.dev/skills/nimadorostkar/claude-skills-collection/llm-evaluation"><img src="https://agentmods.dev/badge/skills/nimadorostkar/claude-skills-collection/llm-evaluation/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/nimadorostkar/claude-skills-collection/llm-evaluation"><img src="https://agentmods.dev/badge/skills/nimadorostkar/claude-skills-collection/llm-evaluation.svg" alt="Reviewed on agentmods" width="80" height="20"></a>- NVIDIA SkillSpector pass
What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00043 | $0.01350 |
| Opus 5 | $0.00022 | $0.00675 |
| Sonnet 5 | $0.00009 | $0.00270 |
| Haiku 4.5 | $0.00004 | $0.00135 |
Grade A, and why
llm-evaluation scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 12d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 126 lines — stays where its author put it; the contents beside it link to each section on GitHub.
LLM Evaluation
Purpose
Know whether an LLM feature is getting better or worse. Without evaluation, every prompt change is a guess, and the confidence that a change helped is indistinguishable from the confidence that it did not.
When to Use
- Before iterating on any prompt or model in production.
- Comparing models, prompts, or retrieval strategies.
- Setting up regression testing for an LLM feature.
- Deciding whether a quality complaint is real or anecdotal.
Capabilities
- Evaluation-set construction from real usage.
- Metric selection: exact match, similarity, rubric-based, task-specific.
- LLM-as-judge, with the controls that make it trustworthy.
- Regression testing in CI.
- Online evaluation and production monitoring.
Inputs
- Real inputs from actual usage, not invented ones.
- A definition of correct — which is the hard part.
- The current behavior, as a baseline.
Outputs
- An evaluation set that includes the hard cases.
- A metric that correlates with what users actually care about.
- A baseline score, and a gate that catches regressions.
Workflow
- Build the set from real usage — Fifty to two hundred real inputs, including the failures. An evaluation set of invented examples measures your imagination, not the system.
- Define correct precisely — For extraction, the exact expected output. For open-ended generation, a rubric with concrete criteria. "A good summary" is not a criterion; "mentions all three decisions and no facts absent from the source" is.
- Choose the cheapest sufficient metric — Exact match where possible. String or semantic similarity next. LLM-as-judge only where the output is genuinely open-ended.
- Validate the judge — Have a human grade fifty cases. If the judge disagrees with the human more than about 15% of the time, the judge is not measuring what you think.
- Baseline, then change one thing — Measure. Change one variable. Measure again on the same set. Anything else is not evidence.
- Gate in CI — A prompt change that drops the score below the threshold fails the build, exactly like any other regression.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 12d ago First seen · 126 lines · 43 tokens per session scan A 1bcf2f5ee721
llm-evaluation is a skill published in the GitHub repository nimadorostkar/Claude-Skills-collection (26 stars, last pushed 24d ago), licensed MIT. It adds 43 tokens to every session and 1,350 once invoked, about $0.0002 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other skills, from other repositories
datarobot-agent-assist
Use when the user wants to design, build, code, simulate, or deploy an AI agent (not a predictive model) to DataRobot; mentions agentspec.md, dr-assist, datarobot-agent-assist, dress rehearsal, swarm simulation, or the DataRobot agent template; wants to scaffold a LangGraph, CrewAI, LlamaIndex, NAT, or Base agent…
datarobot-agent-assist-build
Use when the user wants to design, build, code, or deploy an AI agent on DataRobot; mentions agentspec.md, dress rehearsal, the DataRobot agent template, LangGraph, CrewAI, LlamaIndex, NAT, Base agents, MCP servers, backend APIs, custom frontends, or the DataRobot CLI.
datarobot-predictions
Tools and guidance for making predictions with DataRobot deployments, including real-time predictions, batch scoring, prediction dataset generation, and prediction explanations (SHAP/XEMP). Use when making predictions, running batch scoring, generating prediction datasets, or explaining individual predictions from a…
datarobot-model-explainability
Tools and guidance for model explainability, prediction explanations, feature impact analysis, SHAP values, SHAP distributions, anomaly assessment, and model diagnostics. Use when analyzing model explanations, feature impact, SHAP values, SHAP distributions, anomaly assessment, or diagnosing model behavior.
datarobot-model-training
Comprehensive guidance for training models in DataRobot, including project creation, AutoML configuration, feature engineering, and model selection. Use when training models, creating AutoML projects, or selecting models in DataRobot.
datarobot-model-deployment
Tools and guidance for deploying DataRobot models, managing deployments, configuring prediction environments, and deployment operations. Use when deploying models, creating or updating deployments, or configuring prediction environments.