benchmark

Benchmark

Benchmarking without a baseline is noise. A benchmark that can't be reproduced is a guess. Every benchmark in this skill produces a number, a method, and a comparison — not an impression.

Performance is a feature. Regressions ship silently. This skill makes regressions visible before they reach users.

---

Benchmark Principles

• Measure before you optimize. An optimization without a before measurement is an assumption. Always baseline first.
• Reproduce exactly. A benchmark is only valid if another engineer can reproduce it with the same setup. Document the environment, the command, and the conditions.
• Medians lie. Use percentiles. P50 hides the users who are suffering. Always report P75, P95, and P99 alongside the median.
• Synthetic vs real-user data. Lighthouse in a lab tells you what's possible. RUM (Real User Monitoring) tells you what's happening. Use both.
• One change at a time. If two things change between benchmarks, you can't attribute the delta to either.
• A regression is a regression. A 10% slowdown on a fast page is still a regression. Set thresholds and enforce them.

---

Step 0: Establish the Baseline

Before any optimization or PR review, establish the baseline. Without it, "after" numbers are meaningless.

# Run Lighthouse baseline (3 runs, take median)
npx lighthouse https://your-app.com/key-page \
  --output json \
  --output-path ./baseline.json \
  --chrome-flags="--headless" \
  --throttling-method=simulate \
  --preset=desktop

# Run 3 times and average — single runs are noisy
for i in 1 2 3; do
  npx lighthouse https://your-app.com/key-page \
    --output json \
    --output-path "./baseline-run-$i.json" \
    --chrome-flags="--headless"
done

Record in benchmark-baseline.md:

## Baseline — [Page/Endpoint] — [Date] — [Commit SHA]

| Metric | Run 1 | Run 2 | Run 3 | Median |
|--------|-------|-------|-------|--------|
| LCP    |       |       |       |        |
| INP    |       |       |       |        |
| CLS    |       |       |       |        |
| FCP    |       |       |       |        |
| TTFB   |       |       |       |        |
| Total JS bundle | | | | |
| Total CSS bundle | | | | |
| Network requests | | | | |

**Environment:** macOS M2 / 6-core CPU throttle / Fast 3G simulation
**Tool:** Lighthouse 12.x / Chrome 124
**URL:** https://your-app.com/key-page