operational-value-designer

Operational Value Grader

Design one deterministic operational-value grader that reports absolute operational attainment for each workflow run. Keep exactly one authoritative primary value, and declare any normalized diagnostic metrics separately.

Operational value is the degree to which the workflow's intended repository outcome is attained for the opportunity assigned to a run, demonstrated by accepted repository evidence under a frozen contract. It is not execution quality, output volume, safe-output creation, or an agent's assessment.

Output

Create one executable evaluator at:

.github/graders/WORKFLOW-NAME-operational-value.sh

Configure the workflow:

graders:
  operational-value:
    run: .github/graders/WORKFLOW-NAME-operational-value.sh

The grader's primary operational value (value) is absolute attainment in [0,1]. A comparable frozen baseline may be reported separately as baselineValue; gh-aw derives deltaFromBaseline. Never define the primary operational value as a difference from baseline.

Design Procedure

1. Validate OWNER/REPO and resolve .github/workflows/WORKFLOW-NAME.md. Do not infer inputs from the workspace or remotes.
2. Recover adoption-time intent from the workflow's first commit and first parent. Use only adoption-time workflow content and pre-adoption evidence to choose opportunities, accepted evidence, formulas, targets, or a baseline.
3. Define how every workflow run binds to one operational case:
   • produce a stable opportunityKey;
   • prevent overlapping ownership where possible;
   • preserve repeated keys when duplicate runs target the same opportunity so downstream analysis can cluster or deduplicate them;
   • treat reruns with the same GitHub run ID as the same subject.
4. Freeze accepted evidence, evidence repositories, matching rules, zero-versus-missing behavior, and maturesAt computation.

• Declare only the workflow permission scopes required to collect that evidence. The evaluator receives GH_TOKEN with the agent job's declared permissions; gh-aw does not add evidence permissions automatically.

1. Choose exactly one direct primary metric in [0,1]. Higher must always mean greater attainment. Optional diagnostic metrics may provide normalized outcome context, but must remain separate rather than being combined into the primary value. Keep trace graders and activity counts separate.
2. If comparable pre-adoption evidence exists, score it with the same metric and freeze it under baseline. Otherwise use attainment-only with a null baseline value.
3. Implement the evaluator interface below and run: