ab-testing

A/B Testing for Marketing

When to Activate

• Testing landing page variations to improve conversion rate
• Optimizing email subject lines, copy, or design
• Comparing ad creatives, headlines, or CTAs
• Evaluating pricing page layouts or offers
• Running feature experiments for product-led growth
• Resolving internal debates about what works with data instead of opinions
• Building a systematic testing program

First Questions

1. What are you testing? (Landing page, email, ad, pricing, feature)
2. What is your primary conversion metric? (Sign-up, purchase, click, engagement)
3. What is your current conversion rate for this metric?
4. How much traffic or volume do you have? (Monthly visitors, email list size, daily ad impressions)
5. What is the minimum improvement that would be meaningful to the business?
6. What testing tool are you using? (Google Optimize successor, VWO, Optimizely, LaunchDarkly, in-house)
7. Have you run tests before? What has your win rate been?

Hypothesis Formation

The If/Then/Because Format

Every test must start with a hypothesis. No hypothesis = no learning regardless of outcome.

Template:

If we [change this specific thing], then [this metric] will [increase/decrease] by [estimated amount], because [reason based on data, research, or user insight].

Examples:

• "If we shorten the sign-up form from 6 fields to 3, then sign-up completion rate will increase by 15%, because our funnel analysis shows 40% drop-off at the form step and user research indicates friction from too many fields."
• "If we add social proof (customer count) above the fold on pricing, then trial sign-ups will increase by 10%, because our exit surveys show 'trust' as the #2 concern for prospects."

What Makes a Good Hypothesis

• Based on data or user insight, not a hunch.
• Specific about what changes and what metric is affected.
• Includes a directional estimate (forces you to think about magnitude).
• Falsifiable — you can clearly prove it wrong.