migration-comparison

Migration Comparison

Compare two different migration attempts of the same codebase (e.g., AI agent A vs ground truth, or agent A vs agent B). Produces a self-contained HTML comparison report with categorized changes, semantic analysis, and side-by-side diffs.

---

Phase 1: Setup & Validation

1. Collect Inputs

Ask the user for:

• Reference A: git URL with branch (e.g., https://github.com/org/repo@branch) or a local directory path. Ask for a label (e.g., "Ground Truth", "Agent A").
• Reference B: same format. Ask for a label.
• File filters (optional): glob patterns to restrict which files are compared (e.g., *.tsx, src/**/*.ts).
• Migration target (optional): identifier for target-specific pattern scoring (e.g., patternfly). When specified, the scoring step uses target-specific pattern detectors for more precise quality scoring.

2. Create Workspace

Create a temporary workspace directory outside either project:

WORK_DIR=$(mktemp -d -t migration-comparison-XXXXXXXX)

All artifacts go inside $WORK_DIR.

3. Validate Inputs

For each reference (A and B):

If local path:

• Verify the directory exists and contains files.
• If the path is invalid or empty, tell the user what is wrong and ask them to provide a corrected path.

If git URL:

• Attempt: git clone --branch <branch> --depth 1 <url> $WORK_DIR/<label>
• If the clone fails (auth error, invalid URL, branch not found, network issue), show the user the error and ask:
  1. Provide a corrected URL/branch
  2. Provide a local path instead
  3. Abort
• If the branch is not found, run git ls-remote --heads <url> to list available branches and show them to the user.

4. Check Tool Availability

Run:

python3 scripts/run_diffs.py --check-gumtree

This returns JSON: {"available": true/false, "method": "native|podman|docker|none", "version": "..."}.

If GumTree is not available, inform the user and ask:

1. Continue with text-only diffing (still produces a useful report, just without AST-level semantic categorization)
2. Install GumTree first (podman pull gumtreediff/gumtree, docker pull gumtreediff/gumtree, or download from GitHub releases)
3. Abort