benchmark-analyzer

Benchmark Analyzer

Establish baselines, set targets, and track progress — turn raw metrics into benchmarks you can measure against.

When to Use

• "What's our baseline conversion rate?"
• "Set a target for checkout completion — and track us against it"
• "How does this quarter compare to Q2?"
• "Are we on track to hit our retention target?"
• "What's a good rage-click rate? Are we above or below average?"
• "Benchmark our error rate against industry standards"

Mental Model

A metric without context is just a number. A benchmark gives it context:

• Internal baseline: What's normal for us? (12-month average, last quarter's average)
• Target: Where do we want to be? (goal set by the team, industry standard)
• Comparison: How do we compare? (to ourselves over time, to peers if data available)

Workflow

Step 1: Define what to benchmark

Ask the user:

• "What metrics matter most?" (conversion, retention, error rate, etc.)
• "What time period should be the baseline?" (last quarter, last 12 months, since launch)
• "Do you have specific targets, or do you want me to suggest them?"

Step 2: Compute the baseline

For each metric, compute over the baseline period:

fullstory:build_metric(query="checkout conversion rate", output_type="single_number")
fullstory:compute_metric(metric_id, time_range="last_12_months")
→ Baseline: 19.8% (12-month average)

Also compute the standard deviation or range:

fullstory:build_metric(query="checkout conversion rate", output_type="trend")
fullstory:compute_metric(metric_id, time_range="last_12_months")
→ Range: 16% - 24%, typical weekly variance: ±3pp

Step 3: Compare current to baseline

Current (last 30 days): 21.2%
Baseline (12-month avg): 19.8%
→ +1.4pp above baseline (+7%)
→ Within normal range (baseline range: 16-24%) ✅

Step 4: Set targets

Based on the baseline, suggest targets:

• Conservative (achievable): 21-22% (slightly above baseline)
• Ambitious (stretch): 23-24% (top of historical range)
• Moonshot (breakthrough): 25%+ (above historical max)