reform-comparator

Reform Comparator

Stage 5 of /analyze-policy. Compares our microsim output to the prior-scores anchor and decides whether the result is consistent or needs investigation.

Inputs

• microsim_result (from microsim-runner) — may be null in --skip-microsim mode
• prior_anchor (from prior-scores-finder — anchors[preferred_anchor_index])
• tolerance_band (default: ±25% on cost, ±5 absolute pp on poverty)

When microsim_result is null (--skip-microsim mode)

The /analyze-policy command invokes the comparator with microsim_result=null when the user passed --skip-microsim. In that case:

1. Construct a synthetic predicted result from the anchor:
   • Take the anchor's magnitudes block.
   • Normalize per Step 1 below (year alignment + 10-year extrapolation if needed).
   • Emit the normalized values as the "predicted" result with synthetic: true flag.
2. Treat the synthetic result as if it were our microsim output for the comparison step.
3. Return verdict — typically PASS by construction. This is intentional: process-test mode validates the pipeline plumbing, not the actual numerical agreement.
4. In the output, set verdict_caveat: "process-test — predicted result derived from anchor, not from live microsim" so the downstream report doesn't claim a real validation.

The synthetic-result schema mirrors microsim_result:

{
  "synthetic": true,
  "predicted_from_anchor": "Restoration of ARPA CTC",
  "results": {
    "budget": {"annual_cost_billion_year1": 110.2},
    "poverty": {"child_pct_change": -37.0},
    "distribution": {"gini_pct_change": -1.9}
  }
}

Process

Step 1: Normalize the prior to our run

Most prior PE scores are reported as either single-year cost or 10-year cost, on a specific dataset version. To compare apples-to-apples:

1. Year alignment. If the anchor reports 2023 single-year cost and our run is 2026 multi-year, uprate the anchor:
   • Wage growth: assume +2.5-3.5%/yr nominal (use CBO macroeconomic projections if explicit).
   • Population growth: +0.5-0.7%/yr.
   • Combined uprating factor: anchor_2023 × (1 + 0.035)^(2026-2023) ≈ × 1.10 for 2026 single-year.
2. Single-year vs 10-year normalization. If the anchor is a 10-year score and our run is single-year, normalize the ANCHOR to a per-year average (divide by 10). Do NOT extrapolate our single-year to 10 years by multiplication — this is banned pipeline-wide because per-year cost evolves nontrivially over any window with baseline changes (see the 2026-07-01 std-ded case: yr1×10 understated the real 10-year by 10%). If a real 10-year comparison is required, re-run the microsim with --horizon 10.
3. Dataset version note. Read data_version from the microsim result and record it in the output (our_run_dataset + our_run_data_version). Do NOT hardcode the dataset name. As of PE-US 1.729.0+ the deployed API's enhanced_cps name backs to populace-us-2024. Any prior PE score published on the older Enhanced CPS vintage may differ by 5-10% on refundable-credit costs; flag as a known direction-of-difference.
4. CRITICAL — baseline-schedule alignment. Most published anchors are scored against a specific current-law baseline at the time of writing. If the law has changed since (e.g., SALT cap raised to $40K by OBBBA in 2025, then snapping back to $10K in 2030), comparing our 2026 run (against the OBBBA $40K baseline) to a 2023 prior (against the TCJA $10K baseline) overstates the reform's incremental impact. The 2030 snap-back also means single-year extrapolation across 2030 is biased.