A/B test calculator

Decide a test’s size before it starts. The calculator tells you how many visitors you need and reads the result of a finished test.

An example with the default values. With JavaScript on, the result updates as you type.

Plan a test

You need 53,211 visitors per variant, 106,422 in all.

The test detects a change from 3 % to 3.3 %.

At current traffic that takes about 43 days. Run whole weeks: 7 weeks, or 49 days.

Whole weeks even out the weekday pattern, and the shortest sensible test is two weeks.

Read a result

Not enough evidence yet: the difference may be chance.

The threshold is p < 0.05 in a two-sided test.

Conversion A: 3.00 %
Conversion B: 3.45 %
Difference: +0.45 pp
Relative difference: +15.0 %
p-value: 0.072
95 % interval for the difference: −0.04 to +0.94 pp

Decide the size in advance and read the result only once it is reached. If you check daily and stop at the first significant moment, you will see false winners far more often than the confidence level promises.

Formula and assumptions

The sample size is the two-sided two-proportion z-test without continuity correction: n = (z₁₋α/₂ · √(2p̄(1 − p̄)) + z₁₋β · √(p₁(1 − p₁) + p₂(1 − p₂)))² / (p₂ − p₁)², where p̄ is the mean of the two rates.

The result uses the pooled two-proportion z-test with a two-sided p-value. The interval is the 95 % normal approximation for the difference.

Assumptions: traffic is split evenly, each visitor counts once, and nothing else changes during the test.

Need help with search visibility or testing?

Contact