squid-perf

Squid performance comparison

Compare sync-time performance across one or more Squid SDK deployments. Produces a self-contained HTML report and a Markdown summary in the current working directory.

Skill dir: Resolve the directory containing this SKILL.md once and use its absolute path as <skill-dir>. Do not assume a Claude-specific installation path. Output dir: ./squid-perf-output/ (CWD-relative; created if missing)

Locked-in design (do not re-litigate)

These are settled — don't ask the user again unless they change something.

• Comparison unit: per-service (e.g., compare settlement-arbitrum in deployment A vs B independently of other services). Services present in every compared deployment go in comparison tables; services present in only some go in a "solo metrics" section with a warning.
• Sync time metric: report wall-clock elapsed (headline), active processing time (excludes gaps > 120s — configurable via --downtime-threshold), and downtime (wall − active). All three per breakpoint. If a restart occurs, measure the final uninterrupted processor segment and report the full restart count separately.
• Block alignment: assume all compared indexers cover the same block ranges (user's stated assumption). Use relative-from-first-log per deployment. If detected ranges diverge noticeably across deployments for a given service, emit a loud warning in the summary but still render.
• Tier of metrics extracted:
  • Tier 1 (always): sqd:processor / sqd:batch-processor progress lines → (ts, current_block, target_block, rate, mapping_rate, items_per_sec, eta).
  • Tier 2 (always if present): sqd:multicall latency lines, restarts (detected via current_block going backward), and warning/error/critical totals. Detailed warning/error samples are capped at 1000/service.
  • Tier 3 (auto-discovered): any logger namespace appearing ≥10 times in ALL compared deployments for the SAME service; extract numeric fields; render as a small stats table (count, mean, median, p95).
• Breakpoint selection: percentage-based — 10 evenly-spaced breakpoints at 10%, 20%, ..., 100% of each service's effective range, where effective range = (catchupBlock - firstBlock). catchupBlock = first progress row where current >= target - 10 (indexer reached chain tip). Anything past this is steady-state, not sync, and is excluded from the metric.
  • Rationale: fraction-based clips (e.g., "99.9% of observed range") fail when the idle tail has many progress rows but few blocks — a deployment synced in 4 min and idled 10 days ends up with 99.9% of its blocks still inside the sync phase but 100% of the time inside the tail.
  • If a service has no catchup point in the logs (stillSyncing), falls back to actual lastBlock with a "never caught up" warning.
  • If a service was already caught up at the first progress row (wasAlreadyCaughtUp), the entire captured window is the idle tail — emit a warning and fall back to lastBlock (metrics reflect steady-state latency, not sync).
  • For multi-deployment comparison: shared effective range = min(catchupBlock across deployments) - firstBlock.
  • Output surfaces both the percentage and the absolute block count (e.g., "10% (500K blocks)"). Override via --breakpoints 500K,1M,5M,10M,20M (absolute block offsets from firstBlock; catchup logic does NOT apply to overrides).
• Catchup gap threshold: CATCHUP_GAP_BLOCKS = 10 (constant in report.mjs). Matches the indexer's own steady-state lag behind chain head; tuneable if a chain's head noise is higher.
• Output: self-contained HTML (Chart.js inlined — no CDN), plus Markdown with tables only (no charts in MD, link to HTML at the top).
• HTML template: the HTML report MUST be rendered from <skill-dir>/templates/report.html. report.mjs resolves this path relative to itself and substitutes placeholders rather than building markup via string concatenation. Edit the template to change layout/styling; do not inline HTML in the script.
• Output layout:

  ./squid-perf-output/
  ├── cache/
  │   └── <ref-slug>__<since>.log (+ .capture-start metadata and .done sentinel)
  ├── <ISO-timestamp>/
  │   ├── compare-syncs.json
  │   ├── parsed/<ref-slug>.json
  │   ├── report.html
  │   ├── report.md
  │   └── run.log
  └── latest -> <ISO-timestamp>/
  

• Caching: (ref, since) keyed; .done sentinel written atomically after successful fetch. --force-refresh ignores cache. Interrupted runs (no sentinel) are treated as absent.
• Fetching: parallel Bash (not subagents), one per deployment, wraps sqd logs in expect to handle the "type \"it\" to fetch more logs" pagination prompt. Retries 3× with 10s backoff. Partial success is OK (continue with ≥1 fetched deployment); if all fail, abort.
• Script language: Node (ESM, zero deps). Uses stdlib only. Shell scripts use bash + expect.
• Still-syncing detection: if last log timestamp ≤ 60s of fetch-start, flag as "live/partial" in summary.