ABC Tools

Power and Sample Size for t-Tests, Explained

After reading this you can find the sample size a t-test needs to reach a target power, or check the power of a design you already have, and you can explain why both numbers depend on the effect size you expect.

What power means and why it matters

Power is the probability that your test rejects the null hypothesis when the null is actually false. If your true effect is real but your sample is small, you will often fail to reach significance. That failure rate is the Type II error rate, written \beta. Power is 1 - \beta.

Here is the hook. Suppose two teaching methods really differ by half a standard deviation, a Cohen's d of 0.5. You run a two-sample t-test with 20 people per group at \alpha = 0.05. The power of that design is only about 0.34. You would miss a genuine effect roughly two times out of three. To reach the common target of 0.80, you need about 64 people per group. That jump from 20 to 64 is the whole reason to run these calculations before collecting data, not after.

When to use this calculator, and when not

Use it for the ordinary t-test comparisons of means:

  • Two-sample: two independent groups, such as treatment versus control, with equal variance assumed.
  • One-sample or paired: one group against a fixed value, or the differences within matched pairs.

The results assume a two-sided test and roughly normal data with equal group sizes. If your groups are very unequal, or your outcome is a proportion rather than a mean, this is the wrong tool. For sizing a study that estimates a mean or proportion to within a margin of error rather than testing a hypothesis, use the Sample Size Calculator instead.

Do not compute power after the fact using the effect you observed. That "observed power" adds no information beyond the p-value you already have. Power planning is a decision you make before data collection, using an effect size you expect or care about.

The four quantities that trade off

Four numbers are locked together. Fix any three and the fourth is determined:

Effect size (d)
The true difference in means divided by the standard deviation. Larger effects are easier to detect.
Significance level (α)
The Type I error rate, usually 0.05. Lowering it makes rejection harder, which lowers power.
Power (1 − β)
The chance of detecting the effect. Common targets are 0.80 and 0.90.
Sample size (n)
People per group for a two-sample test, or total pairs for a paired test.

The calculator solves for either n or power. To raise power you can raise n, raise the effect size you are targeting, or accept a larger \alpha. In practice n is the lever you control.

The formula and the intuition behind it

A t-test statistic follows a central t distribution when the null is true. When the null is false, it follows a noncentral t distribution, shifted away from zero by a quantity called the noncentrality parameter \delta. For a two-sample test with n per group,

\delta = d \sqrt{\dfrac{n}{2}}

Here d is Cohen's d and n is the per-group size. For a one-sample or paired test the factor changes because there is one sample, not two:

\delta = d \sqrt{n}

Power is the probability that the noncentral t statistic lands beyond the critical value from the central distribution. With degrees of freedom \nu and a two-sided test,

1 - \beta = P\left(T_{\nu,\delta} \gt t_{1-\alpha/2,\,\nu}\right) + P\left(T_{\nu,\delta} \lt -t_{1-\alpha/2,\,\nu}\right)

where T_{\nu,\delta} is the noncentral t distribution and t_{1-\alpha/2,\nu} is the two-sided critical value. For the two-sample test \nu = 2n - 2; for the one-sample test \nu = n - 1. The second term is tiny for a positive effect and is included for accuracy.

The intuition: \delta is how far the alternative distribution sits from zero, measured in standard errors. Both a bigger d and a bigger n push \delta up, which slides more of the alternative distribution past the critical value.

Notice that \delta grows with \sqrt{n}, not n. To halve the standard error you need four times the sample. That square-root law is why the last bit of power is expensive.

Worked example: reproducing the demo

Sample size for d = 0.5, two-sample

Use the defaults: a two-sample test, effect size d = 0.5, \alpha = 0.05, and target power 0.80. Solve for n.

  1. Guess n = 64 per group. Degrees of freedom \nu = 2(64) - 2 = 126.
  2. Noncentrality: \delta = 0.5 \sqrt{64/2} = 0.5 \times 5.657 = 2.828.
  3. The two-sided critical value at \nu = 126 is about 1.979.
  4. Power is the chance a noncentral t with \delta = 2.828 exceeds 1.979, which comes to about 0.802.

So 64 per group clears the 0.80 target with a hair to spare, and the calculator returns n = 64. For a quick sanity check, the classic approximation ignoring the t correction gives

n \approx \dfrac{2\,(z_{1-\alpha/2} + z_{1-\beta})^2}{d^2} = \dfrac{2\,(1.96 + 0.8416)^2}{0.25} \approx 62.8

which rounds up to 63. The exact noncentral t answer is 64 because the t distribution has slightly heavier tails than the normal.

Power rises steeply through the middle, then flattens. The vertical marker at 64 shows where power crosses 0.80.

An interactive view of the tradeoff

With a two-sample test at α = 0.05, the sample size per group needed for 0.80 power is about 393 for d = 0.2, 64 for d = 0.5, and 26 for d = 0.8. Smaller effects cost far more data: halving the effect size roughly quadruples the required n.

Reading and interpreting the results

When you solve for n, the number returned is the smallest per-group size that meets or exceeds your target power. Round up, never down, and plan for dropout by inflating the count. If you expect 15 percent attrition, divide the target by 0.85: 64 becomes 76.

When you solve for power, read the result as the detection probability for the specific effect size you entered. A design with power 0.34 is not "34 percent right," it is a study that will miss a true effect of that size two times in three. Power below 0.50 is usually a signal to redesign.

The single biggest input is d. The table shows how the required per-group n for 0.80 power collapses as the effect grows.

Per-group n for 0.80 power, two-sample test, α = 0.05, two-sided
Cohen's dn per group
0.2393
0.3176
0.564
0.826
1.017

Common mistakes

A few errors show up again and again.

  • Inventing an optimistic d. If you pick d = 0.8 because it makes the study cheap, you are planning for a large effect that may not exist. Base d on prior studies or the smallest effect worth detecting.
  • Confusing total n with per-group n. For a two-sample test the calculator reports the count in each group. A result of 64 means 128 people total.
  • Using a two-sample plan for paired data. A paired design uses the standard deviation of the differences, which is often much smaller than the raw standard deviation. Enter d as the effect size of those differences and select the one-sample option.
  • Forgetting the tails. These results assume a two-sided test. A one-sided test at the same \alpha has slightly higher power, so treating a one-sided design as two-sided overestimates the sample you need.

Related tools

Effect size is the input that drives everything here, so get it right first. The Cohen's d Effect Size Calculator turns group means and standard deviations into a d you can paste in. If your prior evidence is reported as a correlation or an odds ratio, the Effect Size Converter puts it on the d scale. Once your study is done and you want an interval around the mean rather than a yes-or-no test, the Confidence Interval Calculator takes over.

Frequently asked questions

What effect size should I use if I have no prior data?

Pick the smallest effect that would matter in practice, not the effect you hope to see. Cohen's rough benchmarks are 0.2 small, 0.5 medium, and 0.8 large, but a benchmark is not a substitute for a value that means something in your field.

Why is the required n higher than the normal approximation gives?

The t distribution has heavier tails than the normal, so the critical value is larger, especially at small samples. That pushes the required n up by one or two per group compared to the z-based formula.

Is 0.80 power a rule I have to follow?

No. It is a convention that balances cost against missed effects. For high-stakes work, 0.90 is common. Going from 0.80 to 0.90 at d = 0.5 raises the per-group n from 64 to about 86.

How does a paired design change the sample size?

A paired test uses one sample of differences, so \delta = d\sqrt{n} instead of d\sqrt{n/2}. If the pairing reduces variability, the effect size of the differences is larger, and you need fewer subjects than an independent two-group design.

Can power be exactly 1?

Not for any finite sample. Power approaches 1 as n grows, but there is always a small chance the noncentral t statistic falls short of the critical value. At d = 0.5 and 200 per group, power is about 0.999, close but never exactly 1.

Try it yourself in the tool

Related tools

Confidence Interval Calculator

CIs for a mean (z or t) and for a proportion (Wald & Wilson).

Sample Size Calculator

Required n to estimate a mean or proportion within a margin of error.

Cohen's d Effect Size Calculator

Effect size from group means and SDs, or from a t statistic.

Effect Size Converter

Convert between Cohen's d, r, η² and odds ratios.

Binomial Distribution Calculator

Probabilities of k successes in n independent trials.

Chi-Square Test of Independence

Test association between two categorical variables from a contingency table.