regression-testing

Forbidden invocations

NEVER invoke /sdd-* skills from this workflow. SDD is an optional user-installed ceremony; this skill ships self-contained and does not chain SDD under any condition. If you need to refactor KATA, fixtures, cli/, scripts/, or api/schemas/ pipeline, exit this skill first and invoke /framework-development — which itself runs Plan → Code → Verify → Archive natively (no SDD required).

This boundary is mechanical, not advisory: scripts/lint-skills.ts rejects any /sdd- mention outside this section. See: .agents/skills/agentic-qa-core/references/skill-composition-strategy.md §4 (governs users who manually install SDD).

Regression Testing — Execute, Analyze, Decide

Orchestrates the full release-readiness pipeline: trigger a CI suite, monitor it to completion, classify failures, score against release criteria, and emit a GO / CAUTION / NO-GO verdict plus a stakeholder report.

Three phases, always in this order: Execute → Analyze → Report. Do not skip analysis and jump to a report. Do not guess classification without reading failure logs.

Compact Rules

• DO: run Execute → Analyze → Report in that order. Never skip analysis and jump to a report, and never classify a failure without reading its logs.
• DO: clear the readiness preflight before triggering anything — gh authenticated, the suite's workflow file present, GitHub Actions secrets set, Allure resolvable, active env confirmed. A long run that 401s mid-way is the expensive failure.
• DO: persist RUN_ID the moment the trigger returns, before anything else. Resume re-attaches to a live run instead of re-triggering CI; a trigger that landed without the id saved costs the whole run again.
• DO NOT: mark a failure REGRESSION without checking its history first — the single most common misclassification. A first-ever failure with no history is NEW TEST, unverified, not a regression.
• DO: classify every failure into exactly one of KNOWN-BLOCKED / KNOWN ISSUE / ENVIRONMENT / NEW TEST / FLAKY / REGRESSION, and assess severity on a separate axis — a FLAKY test on checkout is still CRITICAL. Classification is fail-closed: every class but REGRESSION cites its evidence (the @blocked:{BUG-KEY} marker; a ticket created BEFORE the run; the matching log line plus the other failures on that host; the empty history query; the ids and results of at least 5 runs on an unchanged build). No citation, or a ticket this session filed, = REGRESSION.
• DO: exclude @blocked:{BUG-KEY} tests from the gating pass-rate and report their count with each blocking key. They are parked behind an already-filed bug: never REGRESSION, and never a new bug.
• DO NOT: use ENVIRONMENT as a scapegoat. Many unrelated tests failing on one host is environment; one test failing on an endpoint other tests reach fine is more likely a REGRESSION.
• DO NOT: call a test flaky on fewer than 5 runs of history — mark "insufficient history" and re-evaluate rather than guessing.
• DO NOT: emit GO while any REGRESSION-class failure stands. Hard vetoes regardless of score: any @critical test failing, any HIGH/CRITICAL-severity regression, or a pass rate below 90%.
• DO: file only CONFIRMED product failures — the REGRESSION class, plus a NEW TEST failure once manually confirmed to be a real defect. FLAKY, ENVIRONMENT and KNOWN ISSUE get no issue at all. Triage decides WHETHER to file; the defect-management doctrine decides the type and the fields.
• DO NOT: open a GitHub issue for a quality failure. It is filed in the issue tracker, parented to the QA Defect Management process epic and linked to the source Story — never to a product or dev epic.
• DO: create the RTR (Regression Test Results, a Test Execution: RTR: {scope-id}: Regression Testing, parent QA Test Artifacts, Test Environment set, assignee self, testPlan → RTP) BEFORE triggering CI, persist its key beside RUN_ID, and pass it as the execution_key dispatch input. One RTR per verdict. The STR is created or completed ONLY when the run is the sprint close (then it links to both the STP and the RTP).
• DO NOT: import a regular regression run into the sprint STR, and never let smoke or sanity write into a regression execution: they import only when an execution key is passed explicitly.
• DO: close the RTR (or the sprint-close STR) via complete only AFTER the verdict comment is posted on it, and leave the RTP at its ready status: a suite run never completes the plan it ran from.
• DO NOT: invent a sprint number. The RTR needs none (its scope-id is {env}-{YYYY-MM-DD} or a release tag). N matters only for the sprint-close STR: take it from the user or from the STP's own scope-id, and ask before creating anything at sprint altitude.
• DO NOT: skip the artifact download on a red build (evidence vanishes after the retention window), and never merge smoke and regression results into one pass-rate — their SLOs differ.
• DO: inspect traces and screenshots with /playwright-cli ([AUTOMATION_TOOL]); it is the only browser path, no MCP backs it.