Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add michalstrnadel/source-to-skill --skill skill-evalgit clone --depth 1 https://github.com/michalstrnadel/source-to-skillWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/michalstrnadel/source-to-skill/skill-eval)<a href="https://agentmods.dev/skills/michalstrnadel/source-to-skill/skill-eval"><img src="https://agentmods.dev/badge/skills/michalstrnadel/source-to-skill/skill-eval/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/michalstrnadel/source-to-skill/skill-eval"><img src="https://agentmods.dev/badge/skills/michalstrnadel/source-to-skill/skill-eval.svg" alt="Reviewed on agentmods" width="80" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00117 | $0.00979 |
| Opus 5.5 | $0.00047 | $0.00392 |
| Sonnet 5.5 | $0.00023 | $0.00196 |
| Haiku 4.5 | $0.00012 | $0.00098 |
Grade A, and why
skill-eval scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 3d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 78 lines — stays where its author put it; the contents beside it link to each section on GitHub.
skill-eval
Check a skill against the source it was made from. The question is not "does the skill read well" but "can an agent holding only this skill answer what the source answers, and cite it correctly".
Usage: /skill-eval <skill-dir> (a path, or a skill name found under
~/.claude/skills/, ~/.agents/skills/, ~/.copilot/skills/ or the
project's .claude/skills/, .agents/skills/, .github/skills/).
Step 1 — Get the questions
If <skill-dir>/evals/evals.json exists (source-to-skill writes one while
the source is still in context), use it. Otherwise draft one:
- Read
<skill-dir>/source.json(topic skills:sources/*.source.json). Missing → say the skill was not made by source-to-skill and ask for the original source. - Re-extract the source with the source-to-skill extractor into a fresh
work dir (
python3 scripts/extract.py "<source>" [options from the manifest] --work-dir <tmp>/skill_eval, run from the source-to-skill skill folder). Tell the user theest_tokensand ask before reading it. - Draft 8 questions from
full_text.txt, never from the skill, spread across the source's segments: 3 concrete facts or numbers, 2 decision rules or "when should I", 2 that need two segments combined, and for video or audio 1 "where in the source is X" that needs a timestamp. Each gets 1 to 3 expectations: short statements a grader can check against an answer, each naming its segment.
Write it in the skill-creator format so the same file works there:
{
"skill_name": "<slug>",
"evals": [
{"id": 1, "prompt": "What learning-rate schedule did the paper use?",
"expected_output": "Linear warmup for 4000 steps, then inverse square root decay",
"files": [],
"expectations": ["States warmup_steps = 4000 (segment 5.3 Optimizer)",
"Says the rate then decays proportionally to step^-0.5"]}
]
}
Step 2 — Answer with only the skill
The answers must come from someone who has not read the source. If your host can run subagents, give one subagent the skill directory path and the eight prompts, and tell it to answer from those files only, citing file names and deep links. Without subagents, answer them yourself, reading only the skill's files, and say in the report that grading was not blind.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 3d ago First seen · 78 lines · 117 tokens per session scan A 651b6689c9a9
skill-eval is a skill published in the GitHub repository michalstrnadel/source-to-skill (8 stars, last pushed 4d ago), licensed MIT. It adds 117 tokens to every session and 979 once invoked, about $0.0005 per session on Opus 5.5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-10-27.
Other skills, from other repositories
133-java-testing-acceptance-tests
Use when you need to implement acceptance tests from maintainer-sanitized Gherkin scenario facts for framework-agnostic Java (no Spring Boot, Quarkus, Micronaut) — confirming @acceptance scenarios before coding, happy path with RestAssured, DB/Kafka test fixtures, WireMock for external REST only, and AT classes run by…
702-technologies-wiremock
Use when you need framework-agnostic WireMock guidance — stub design, JSON or programmatic mappings, precise request matching, response bodies and faults, classpath fixtures, isolation and reset between tests, verification of calls, dynamic ports and base URLs, and avoiding flaky stubs — without choosing Spring Boot…
054-design-tdd
Use when Java implementation work should be guided by Test-Driven Development, including maintaining a test list, choosing the next behavior, writing a failing test first, implementing only enough production code to pass, and refactoring while keeping tests green. This should trigger for requests such as Apply TDD…
131-java-testing-unit-testing
Use when you need to review, improve, or write Java unit tests — including migrating from JUnit 4 to JUnit 5, adopting AssertJ for fluent assertions, structuring tests with Given-When-Then, ensuring test independence, applying parameterized tests, mocking dependencies with Mockito, verifying boundary conditions…
132-java-testing-integration-testing
Use when you need to set up, review, or improve Java integration tests — including generating a BaseIntegrationTest.java with WireMock for HTTP stubs, detecting HTTP client infrastructure from import signals, injecting service coordinates dynamically via System.setProperty(), creating WireMock JSON mapping files with…
321-frameworks-spring-boot-testing-unit-tests
Use when you need to write unit tests for Spring Boot applications — including pure unit tests with @ExtendWith(MockitoExtension.class) for @Service/@Component, slice tests with @WebMvcTest and @MockitoBean for controllers, @JsonTest for JSON serialization, parameterized tests with @CsvSource/@MethodSource, test…