optimize

Optimize

Metric-driven optimization with explicit guards, rollback criteria, experiment log.

Input Schema

{
  "goal": "required measurable improvement objective",
  "mode": "single|campaign",
  "metric_cmd": "required command that emits or validates the target metric",
  "metric_direction": "higher|lower",
  "guard_cmd": "required command that must continue to pass",
  "max_iterations": "optional integer, default 1",
  "min_delta": "optional practical significance threshold",
  "scope_files": [
    "paths the optimization may edit"
  ],
  "done_when": "metric improves without guard regression"
}

Workflow

01: Create run directory

Run create_run.py --skill optimize per ../../shared/helper-cli-contract.md.

02: Validate metric and guard commands

Require:

• Repeatable metric_cmd producing comparable value or pass/fail.
• Known metric_direction.
• guard_cmd fails on unacceptable regressions.
• Bounded scope_files.
• Explicit, bounded max_iterations for campaign.
• Protect files/scripts used by metric_cmd/guard_cmd unless user explicitly scopes them and accepts measurement-integrity risk.

Dry-run both before edit:

Execute configured metric_cmd and guard_cmd separately with host-native command runner. Write complete combined output to <run-directory>/metric-baseline.txt and <run-directory>/guard-baseline.txt; retain both exit codes and stop before editing if either command cannot run.

03: Record baseline and hypothesis

Write <run-directory>/hypothesis.md:

• metric to improve
• expected mechanism
• files allowed to change
• guard risk
• rollback condition

For campaign, noisy metrics, GPU/ML performance, or correctness-sensitive code, read and apply ../../shared/specialist-orchestration.md; otherwise do not load it. Write <run-directory>/specialist-optimization-plan.md with narrow context packs for:

• squeezer: profiling mechanism, bottleneck hypothesis, measurement plan.
• qa-specialist: guard coverage and regression risk.
• data-steward: data pipeline or reproducibility impact.
• scientist: metric validity, ablation design, statistical noise.
• challenger: overfitting to metric or weakening guard checks.