evaluate-skills

<essential_principles>

1. Don't Just Build -- Measure. Every skill should have evidence that it works. Eval-driven development means writing test cases before assuming effectiveness.

2. Parallel Execution. Spawn subagents to run with-skill and without-skill configurations simultaneously. Never run evals sequentially when parallel is possible.

3. Realistic Queries. Eval queries must include real-world noise: file paths, personal context, company names, URLs, backstory, typos, casual speech. Sanitized queries produce misleading results.

4. Statistical Rigor. Report mean, stddev, min, max. A single test case proves nothing. Use 8-20 cases per eval run. Calculate precision, recall, and accuracy for trigger tests.

5. Train/Test Split. When optimizing descriptions, always hold out a test set. Optimizing against all data overfits. Select the best variant by TEST performance, not train.

</essential_principles>

1. Trigger test -- Does the skill activate on the right queries and stay silent on the wrong ones?
2. Benchmark -- Compare skill vs. no-skill output quality, timing, and token usage
3. Compare versions -- Blind A/B comparison of two skill variants
4. Optimize description -- Automated loop to maximize trigger accuracy

If intent is clear from context, skip the question and route directly.

Intent-based routing:

• "test if this skill triggers", "is my skill activating" -> workflows/trigger-test.md
• "benchmark this skill", "measure improvement" -> workflows/benchmark.md
• "compare these two versions", "A/B test" -> workflows/compare.md
• "optimize the description", "skill not triggering enough" -> workflows/optimize-description.md

After reading the workflow, follow it exactly.

<quick_start>

1. Point to a skill: provide the skill path or name
2. Choose evaluation mode (trigger-test, benchmark, compare, optimize-description)
3. The skill generates realistic eval queries automatically
4. Subagents execute in parallel (with-skill vs without-skill)
5. Results are graded, aggregated, and reported with actionable findings </quick_start>

<reference_index>

• references/eval-workflow.md -- Complete eval methodology: query generation, execution, grading, aggregation, analysis
• references/benchmark-schemas.md -- JSON schemas for evals.json, grading.json, metrics.json, timing.json, benchmark.json, comparison.json
• references/description-optimization.md -- Train/test split methodology, extended thinking for proposals, iteration protocol, overfitting prevention </reference_index>

<workflows_index>

Workflow	When to use
workflows/trigger-test.md	Testing whether a skill activates on relevant queries
workflows/benchmark.md	Full quality benchmark comparing skill vs baseline
workflows/compare.md	Blind A/B comparison of two skill versions
workflows/optimize-description.md	Automated description optimization loop
</workflows_index>

<success_criteria>

• Trigger test: precision and recall both above 80% (target 90%+)
• Benchmark: skill configuration shows measurable improvement in pass rate over baseline
• Compare: clear winner identified with per-criterion scores and aggregate
• Optimize: description achieves higher test-set accuracy than original, without overfitting
• All eval runs produce structured reports with actionable next steps </success_criteria>