eval-harness-skill

What I do

I provide a structured evaluation framework to assess code, skills, and implementations:

1. Define Evaluation Criteria: Establish measurable quality benchmarks for code or skill output
2. Execute Evaluations: Run automated checks against defined criteria with scoring
3. Score Results: Generate quantitative scores with qualitative feedback
4. Identify Gaps: Highlight areas below threshold with specific improvement suggestions
5. Track Progress: Compare scores across iterations to measure improvement

When to use me

Use this skill when:

• You want to evaluate the quality of generated code against best practices
• You need to assess if a skill or agent is producing correct, high-quality output
• You want to benchmark code quality before and after refactoring
• You need to verify that implementation meets acceptance criteria
• You want to establish quality gates for PRs or deployments

Trigger phrases:

• "evaluate this code"
• "run eval"
• "score this implementation"
• "quality check"
• "benchmark against criteria"

Core Workflow

Step 1: Define Evaluation Criteria

Establish what "good" looks like for the evaluation target:

## Evaluation Criteria

### Correctness (0-10)
- Does the code do what it's supposed to do?
- Are edge cases handled?
- Are error paths covered?

### Code Quality (0-10)
- Follows language/framework conventions?
- Clean, readable, maintainable?
- Proper abstractions without over-engineering?

### Test Coverage (0-10)
- Are tests present and comprehensive?
- Do tests cover edge cases?
- Are tests maintainable?

### Performance (0-10)
- Efficient algorithms and data structures?
- No unnecessary computations?
- Proper caching where applicable?

### Security (0-10)
- Input validation?
- No injection vulnerabilities?
- Proper auth/authz?

Step 2: Configure Evaluation Scope

Define what to evaluate:

Scope	Description	Example
Single File	Evaluate one file	eval src/auth/login.ts
Module	Evaluate a directory/module	eval src/auth/
Skill Output	Evaluate what a skill produces	eval skill:tdd-workflow
PR Diff	Evaluate changes in a PR	eval pr:42
Acceptance Criteria	Check against issue requirements	eval issue:150