run-validation-until-no-failure

Run Validation Until No Failure

Loop the analysis/validation pipeline until it reports Failed: 0. Each iteration: run ida_analyze_bin.py -debug; if a skill failed (the pipeline is fail-fast, so it's exactly one), diagnose why, present several concrete resolution options, and ask the user which option to take. Do not disable or record the skill before the user answers. If the user explicitly answers that it cannot be solved now and asks to temporarily disable the skill, append the failure to docs/ida_validation_failure-<GAMEVER>.md, comment that skill out of configs/<GAMEVER>.yaml, and run the unit and repository-contract unittest suites before resuming validation. If the config dependency test reports consumers whose expected_input can no longer be produced, apply the same ask-before-action rule to each dependent skill. Environment / infrastructure failures are not skill bugs — STOP and surface those instead of quarantining a skill.

Resolve GAMEVER from the user's explicit request or CS2VIBE_GAMEVER; use only configs/$GAMEVER.yaml for the validation run and all quarantine edits.

When to Use

• The user asks to "run validation until it passes / until no failures", "get validation green", or "disable the failing skills and keep going".
• After bumping to a new game version or landing many new skills, to triage a wave of failures — each gets quarantined with its reason for later fixing.

Why the loop works (pipeline facts)

• uv run ida_analyze_bin.py -debug ends with a summary and exits non-zero when Failed > 0:

  ============================================================
  Summary
  ============================================================
    Successful: N
    Failed: N
    Skipped: N
  

• It is fully fail-fast: the first failing skill aborts the remaining skills, the remaining platforms, and the remaining modules. So each run surfaces exactly one actionable failing skill — handle at most one validation-failing skill per iteration.
• Skills whose expected_output YAML already exist on disk are skipped on re-runs, so each iteration advances quickly to the next unresolved skill.