eval

Eval — Quality & Regression Gate

CRITICAL: Run HARNESS_DIR=$(epic path) first. Never use .harness/ in the project directory.

When to Trigger

• Before /ship creates a PR (automatic if eval.yaml exists)
• After /go completes a feature
• On explicit /eval command
• When user mentions "regression", "baseline", "eval suite", "quality check"
• CI: make eval or epic eval --json

Execution Modes

4 dimensions run in parallel where possible:

1. eval:correctness — Test pass rate, mutation score, assertion density
2. eval:performance — Throughput, latency, memory (opt-in)
3. eval:quality — Lint, code quality, LLM-as-judge
4. eval:regression — Baseline comparison, score deltas

---

Process

Step 0: Prerequisites

HARNESS_DIR=$(epic path)

If $HARNESS_DIR/eval/eval.yaml does not exist, run scaffold:

epic eval --init

Read the config:

cat $HARNESS_DIR/eval/eval.yaml

Step 0.5: Scaffold benchmarks (when no benchmark infrastructure exists)

If eval.yaml has benchmarks: [] and no benchmark files are found in the project:

1. Generate stub files using the CLI:

   epic eval --scaffold
   

   Supported stacks (auto-detected from project markers):

   Stack	Detected by	Generated file	Output format
   Rust	Cargo.toml	benches/eval_harness.rs	criterion (exit code)
   Python	pyproject.toml / setup.py	benchmarks/eval_runner.py	JSON composite
   TypeScript	tsconfig.json	benchmarks/eval.ts	JSON composite
   Node.js	package.json	benchmarks/eval.mjs	JSON composite
   Go	go.mod	benchmarks/eval_test.go	JSON composite
   Java	pom.xml / build.gradle	benchmarks/EvalBenchmark.java	exit code
   Kotlin	build.gradle.kts	benchmarks/EvalBenchmark.kt	exit code
   Ruby	Gemfile	benchmarks/eval_benchmark.rb	JSON composite
   PHP	composer.json	benchmarks/eval_benchmark.php	JSON composite
   C#	*.csproj / *.sln	Benchmarks/EvalBenchmark.cs	JSON composite
   Swift	Package.swift	benchmarks/EvalBenchmark.swift	JSON composite
   Elixir	mix.exs	benchmarks/eval_benchmark.exs	JSON composite
   C++	CMakeLists.txt	benchmarks/eval_benchmark.cpp	exit code