visual-regression

Visual Regression Testing

Overview

Two-mode skill for catching visual regressions. During development, create screenshot walkthroughs of affected flows. At release time, compare screenshots between two git refs and use a sub-agent to classify changes as expected or unexpected.

Core principle: Screenshots are cheap and regenerable. Never commit them to git. The value is in the comparison and classification, not the screenshots themselves.

Dependency: Playwright must be installed in the project.

When to Use

• After making any user-facing change (component, layout, CSS, copy)
• Before a release to catch accumulated visual regressions
• When QA is a bottleneck and you need to prioritize what humans review

When NOT to Use

• API-only or backend-only changes
• As a CI gate on PRs (too slow — run at release time only)

Common Mistakes

Mistake	Why it's wrong
Committing screenshots to git	They're regenerable. Bloats repo for no value.
Running visual comparison in CI on every PR	Too slow. Rob: "You cannot do this in PR merge."
Using hard-coded pixel thresholds alone	The key insight is AI-powered analysis of why something changed, not just that it changed.
Over-engineering with versioned baselines and symlinks	Keep it simple: checkout old tag, screenshot, checkout new tag, screenshot, diff.
Skipping the textual walkthrough	Screenshots without descriptions are useless for review. Always include what the user sees and what flow this belongs to.

Mode 1: document (During Development)

When you make a user-facing change, do this:

Step 1: Identify affected flows

Look at what you changed and list the user flows it touches (e.g., "login flow", "checkout step 2").

Step 2: Write or update Playwright screenshot scripts

Create scripts under e2e/visual/ that walk through each affected flow and capture screenshots at each meaningful step.

// e2e/visual/login-flow.spec.ts
import { test } from '@playwright/test';

test('login flow walkthrough', async ({ page }) => {
  // Step 1: Landing page
  await page.goto('/');
  await page.waitForLoadState('networkidle');
  await page.screenshot({ path: '.screenshots/flows/login-flow/01-landing-page.png', fullPage: true });

  // Step 2: Click login, see form
  await page.click('[data-testid="login-button"]');
  await page.waitForSelector('[data-testid="login-form"]');
  await page.screenshot({ path: '.screenshots/flows/login-flow/02-login-form.png', fullPage: true });

  // Step 3: Fill and submit
  await page.fill('[name="email"]', '[email protected]');
  await page.fill('[name="password"]', 'password123');
  await page.click('[data-testid="submit"]');
  await page.waitForSelector('[data-testid="dashboard"]');
  await page.screenshot({ path: '.screenshots/flows/login-flow/03-dashboard.png', fullPage: true });
});