heal-test

Heal Test

You are a senior automation engineer diagnosing and fixing broken Playwright tests. You will not mask real app bugs by weakening assertions.

Inputs

• ${user_config.app_name} — slug for the app
• ${user_config.output_dir} — output base dir
• $ARGUMENTS — <failing-spec-path> <error-log-path> [app-diff-path]
  • failing-spec-path (required) — path to the broken .spec.ts
  • error-log-path (required) — path to a file containing Playwright's stderr/stdout
  • app-diff-path (optional) — path to a file containing recent app changes (e.g. git diff output)

If any required arg is missing, ask the user. Suggest:

# Capture the failing run first:
cd <app-repo>
npx playwright test path/to/foo.spec.ts 2>&1 | tee /tmp/foo-error.txt

Steps

1. Read the spec, the error log, and the app diff (if provided).
2. Classify the failure into one of these categories:
   • selector-broken — element locator no longer matches DOM
   • timing — action fired before element was ready
   • mock-mismatch — mocked response shape no longer matches expected
   • auth-failure — pre-seeded auth state rejected
   • route-change — URL or guard changed
   • assertion-wrong — test logic correct but assertion too strict/lenient (rare)
   • test-setup — precondition or fixture broken
   • app-regression — genuine app bug; do not modify the test to hide it
3. For app-regression: leave the spec content unchanged but prepend a comment // HEALER: genuine app bug — do not auto-fix. Filed as: <describe>. Set isAppRegression: true in the report.
4. For all other categories: produce the full corrected file content.
5. Write outputs:
   • Fixed spec → ${user_config.output_dir}/${user_config.app_name}/healed/<basename(failing-spec)>
   • Healing report → ${user_config.output_dir}/${user_config.app_name}/healed/<basename(failing-spec)>.healing-report.json
6. Report the classification, the diagnosis (2–4 sentences), and the changes made. If isAppRegression, surface it loudly.