prompt-evaluate

Prompt Evaluate

Score prompt content for claude-code-modes against 10 quality criteria grounded in Anthropic's emotion research and prompt engineering best practices.

Before starting

Read these references:

1. references/emotion-research.md — emotion vector findings
2. references/prompt-quality.md — 10 quality criteria with thresholds
3. references/scoring-rubric.md — measurement methods and commands

Phase 1: Identify what to evaluate

Determine the target. Options:

A base directory — e.g., prompts/chill/ or a custom base path. Read all fragment files and the manifest.

A modifier file — e.g., prompts/modifiers/debug.md. Read the single file.

An assembled prompt — run bun run src/build-prompt.ts <preset> [--base <base>] --print to get the fully assembled output. This is the most comprehensive evaluation since it includes axis fragments and modifiers.

A file path provided by the user — read it directly.

If the user doesn't specify, ask what to evaluate.

Phase 2: Run measurements

For each of the 10 criteria, collect data using the methods in references/scoring-rubric.md.

Automated measurements

Run these commands against the target files:

# 1. Negative instruction count
grep -ciE '\b(don.t|do not|never|avoid|must not|should not|cannot)\b' <files>

# 2. ALL-CAPS emphasis count
grep -cE '\b(IMPORTANT|CRITICAL|MUST|NEVER|DO NOT)\b' <files>

# 4. Worked examples count
grep -c '<example>' <files>

# 5. Priority hierarchy presence
grep -ciE '(when.*conflict|priority|comes first|takes precedence)' <files>

# 7. Character count (token efficiency)
wc -c <files>

Manual assessments

For criteria that require reading and judgment (3, 6, 8, 9, 10), read the content and assess against the rubric thresholds.

Phase 3: Produce scorecard

Present results as a table. For each criterion, show: rating (Good/Concerning/Poor), measured value, and a specific finding.