Why do surveys so often need 385 responses?

At 95% confidence (z ≈ 1.96), a ±5% margin of error and the most conservative proportion of 50%, n = 1.96² × 0.5 × 0.5 / 0.05² ≈ 384.15, which rounds up to 385. That barely changes whether the population is ten thousand or a hundred million; only for small populations does the finite population correction help noticeably (about 278 for a population of 1,000). If you expect a proportion far from 50%, you need fewer responses.

What do 80% power and α = 0.05 mean?

α is the type I error rate — the chance of declaring a difference when there is none, usually 5%. Power (1 − β) is the chance of detecting the difference when it really is as large as you specified, typically 80% or 90%. More observations mean more power, and smaller effects need many more: the sample size grows roughly with 1/effect², so halving the effect size quadruples the sample.

What should I enter as the minimum detectable effect for an A/B test?

The MDE is the smallest lift you want the test to reliably detect. Enter it as an absolute change (20% to 25% conversion is 5 percentage points) or a relative one (a 25% lift is the same 20% → 25%). Smaller MDEs need much more traffic: with a 20% baseline, α = 5% two-sided and 80% power, detecting 5 points takes about 1,030 users per variant, while 2 points takes over 6,000. Set it to the smallest lift that would actually be worth shipping.

Why do the t-test and z approximation differ by a few people?

The z approximation assumes the population standard deviation is known and uses the normal distribution. A real t-test estimates the SD from the data, so the critical value comes from the t distribution and power from the noncentral t, which needs slightly more observations in small samples. For d = 0.5, α = 0.05 two-sided and 80% power, z gives 63 per group and the exact t-test 64. The exact t is the default and matches G*Power; for large samples the two agree.

👥

👥Sample Size Calculator - Surveys, A/B Tests & Power

Before running a survey, an experiment or an A/B test, work out how many observations you need. Survey proportion: from the confidence level, expected proportion and margin of error, n₀ = z²p(1 − p)/E², with the finite population correction for small populations — 95% confidence, ±5% and p = 50% gives the classic 385. Estimating a mean works the same way with n₀ = (zσ/E)². Hypothesis tests: two proportions use the pooled Fleiss formula with an optional continuity correction; two independent means and paired designs can use the normal approximation or the exact noncentral t distribution, checking the power for each n so results match G*Power (64 per group for d = 0.5, α = 0.05 two-sided, 80% power). A/B tests: enter the baseline conversion rate and the minimum detectable effect, absolute or relative, to get the per-variant sample size from Evan Miller’s formula, then the number of days the test needs at your daily traffic. Every mode also works in reverse: from a sample size to the margin of error, the power or the smallest detectable lift. Each result shows the formula with your numbers plugged in and a power (or margin) curve against sample size, so you can weigh cost against precision.

Free · No sign-up · Runs in your browserLast updated: 2026-10-26

Loading tool...

How to use Sample Size & Power Calculator

1Choose the scenario: survey proportion, mean, two proportions, two means, paired or A/B test
2Enter the confidence level or α, power, and the expected proportion, SD or effect size
3Read the required sample size (or the margin or power in reverse mode), the worked formula and the power curve

FAQ

Why do surveys so often need 385 responses?: At 95% confidence (z ≈ 1.96), a ±5% margin of error and the most conservative proportion of 50%, n = 1.96² × 0.5 × 0.5 / 0.05² ≈ 384.15, which rounds up to 385. That barely changes whether the population is ten thousand or a hundred million; only for small populations does the finite population correction help noticeably (about 278 for a population of 1,000). If you expect a proportion far from 50%, you need fewer responses.
What do 80% power and α = 0.05 mean?: α is the type I error rate — the chance of declaring a difference when there is none, usually 5%. Power (1 − β) is the chance of detecting the difference when it really is as large as you specified, typically 80% or 90%. More observations mean more power, and smaller effects need many more: the sample size grows roughly with 1/effect², so halving the effect size quadruples the sample.
What should I enter as the minimum detectable effect for an A/B test?: The MDE is the smallest lift you want the test to reliably detect. Enter it as an absolute change (20% to 25% conversion is 5 percentage points) or a relative one (a 25% lift is the same 20% → 25%). Smaller MDEs need much more traffic: with a 20% baseline, α = 5% two-sided and 80% power, detecting 5 points takes about 1,030 users per variant, while 2 points takes over 6,000. Set it to the smallest lift that would actually be worth shipping.
Why do the t-test and z approximation differ by a few people?: The z approximation assumes the population standard deviation is known and uses the normal distribution. A real t-test estimates the SD from the data, so the critical value comes from the t distribution and power from the noncentral t, which needs slightly more observations in small samples. For d = 0.5, α = 0.05 two-sided and 80% power, z gives 63 per group and the exact t-test 64. The exact t is the default and matches G*Power; for large samples the two agree.