ABC Tools

Chi-Square Goodness-of-Fit Test, Explained

After reading this you will be able to test whether a set of observed category counts matches an expected distribution, compute the chi-square statistic by hand, and read the p-value without fooling yourself.

What the test does

The chi-square goodness-of-fit test compares counts you observed in several categories against counts you expected. It answers one question: are the differences between observed and expected large enough to be surprising, or could they be ordinary sampling noise?

Suppose you roll a die 100 times and get 50 ones, 30 twos and 20 threes (ignore the other faces for the moment). If the die were fair across three outcomes you would expect about 33.3 of each. You saw 50, 30 and 20 instead. Is that a loaded die, or just luck? The test turns that vague worry into a single number and a probability.

The variable being tested must be categorical, and each observation must fall into exactly one category. You test counts, never percentages or averages.

When to use it, and when not to

Use this test when you have one categorical variable, a set of observed frequencies, and a claimed distribution to check against. Common cases: is a die fair, do customers pick the four package tiers in the ratio the sales team assumed, do defects spread evenly across five machines, does a genetics cross follow a 9:3:3:1 ratio.

Do not use it in these situations:

  • You have two categorical variables and want to know if they are related. That is a test of independence, not goodness of fit.
  • Your data are proportions or means rather than counts. For a single proportion against a target, use the one-proportion z-test.
  • Expected counts are very small. When any expected count drops below about 5, the chi-square approximation gets shaky.

Feed the test raw counts, not percentages. If you enter 50, 30, 20 as percentages that happen to sum to 100, the test cannot tell the difference from 100 actual observations, and with a real sample of 1000 your statistic would be ten times larger. The counts carry the sample size, and the sample size drives the whole result.

The formula and why it works

Let O_i be the observed count in category i and E_i the expected count. The test statistic is:

\chi^2 = \sum_{i=1}^{k} \frac{(O_i - E_i)^2}{E_i}

Here k is the number of categories and the sum runs over all of them. Each term takes the gap between observed and expected, squares it so positive and negative gaps both count, then divides by the expected count so that a gap of 10 matters more when only 5 were expected than when 500 were.

The statistic is zero when every observed count matches its expected count exactly, and it grows as the fit gets worse. To turn it into a probability you compare it to a chi-square distribution with degrees of freedom:

df = k - 1 - m

Here m is the number of parameters you estimated from the data to build the expected counts. In the simplest case, where the expected distribution is fully specified in advance, m = 0 and df = k - 1. You lose one degree of freedom because the counts must sum to the fixed total: once k-1 of them are known, the last is determined.

Worked example with the demo data

Three categories, 50 / 30 / 20

Enter 50, 30, 20 as the observed counts and leave the expected field blank. A blank expected field means all categories are equally likely, so each expected count is the total divided by the number of categories.

  1. Total observations: 50 + 30 + 20 = 100.
  2. Expected under a uniform distribution: 100 / 3 = 33.33 in each category.
  3. Category 1: (50 - 33.33)^2 / 33.33 = 16.67^2 / 33.33 = 8.333.
  4. Category 2: (30 - 33.33)^2 / 33.33 = 3.33^2 / 33.33 = 0.333.
  5. Category 3: (20 - 33.33)^2 / 33.33 = 13.33^2 / 33.33 = 5.333.
  6. Sum: \chi^2 = 8.333 + 0.333 + 5.333 = 14.0.
  7. Degrees of freedom: 3 - 1 = 2.

With \chi^2 = 14.0 and df = 2, the p-value is about 0.0009. That is far below 0.05, so you reject the hypothesis that the three categories are equally likely. The counts are not consistent with a uniform split.

Contribution of each category to the demo statistic
CategoryObservedExpected(O − E)²/E
15033.338.333
23033.330.333
32033.335.333
Total10010014.00
Categories 1 and 3 sit far from the flat expected line of 33.33; category 2 is close. Those gaps produce the 8.33 and 5.33 contributions that dominate the statistic.

Reading and interpreting the result

The calculator returns the chi-square statistic, the degrees of freedom, and the p-value. The p-value is the probability of seeing a fit at least this bad if the expected distribution were true. Small p-value, poor fit, reject the null. Large p-value, no evidence of misfit.

Compare the p-value to your chosen significance level, usually 0.05. In the demo, 0.0009 < 0.05, so you reject. If instead you had rolled 40, 33 and 27, the statistic would be (6.67^2 + 0.33^2 + 6.33^2)/33.33 = 2.54, giving a p-value near 0.28. That is well above 0.05, so you would not reject uniformity even though the raw counts look uneven.

A large statistic tells you the fit is poor but not why. Scan the per-category contributions. In the demo, category 1 contributes 8.33 and category 3 contributes 5.33, so those two categories drive the result. Reporting where the misfit lives is more useful than reporting the total alone.

With the demo counts 50, 30, 20 the statistic is 14.0 on 2 degrees of freedom, p ≈ 0.0009. If you move the counts toward 33, 33, 34 the statistic falls toward 0 and the p-value rises toward 1; if you push them toward 70, 20, 10 the statistic climbs past 40 and the p-value drops below 0.0001.

Using custom expected proportions

Leaving the expected field blank tests a uniform distribution. To test any other claim, enter the expected counts or proportions. The calculator rescales them to match your observed total, so you can enter 9, 3, 3, 1 for a genetics ratio or 0.5, 0.3, 0.2 for a market-share claim without doing the arithmetic yourself.

As an example, suppose you expect a 3:1 split and observe 90, 30 across two categories. The total is 120, so expected counts are 90 and 30, an exact match, and \chi^2 = 0. Now observe 75, 45: expected stays 90 and 30, so \chi^2 = (15^2/90) + (15^2/30) = 2.5 + 7.5 = 10.0 on 1 degree of freedom, p ≈ 0.0016. The same absolute gap of 15 hurts three times as much in the smaller expected category.

Common mistakes

Tiny expected counts
When an expected count falls below 5, the p-value from the chi-square approximation can be off. Combine sparse categories, collect more data, or for a 2×2 layout use Fisher's exact test.
Testing proportions instead of counts
Entering percentages throws away the sample size. Always use the actual frequencies.
Forgetting estimated parameters
If you fit a distribution to the data first (for example estimating a Poisson rate from the same counts), subtract one degree of freedom per estimated parameter. Skipping that step understates the statistic's tail and gives a p-value that is too small.
Reading direction into the result
The test is one-sided on the statistic but has no direction on the categories. It flags misfit, not which category is high or low. Read the contributions to see that.

Related tools

If your question involves two categorical variables rather than one, use the chi-square test of independence. To compare two proportions directly, the A/B test calculator runs a two-proportion z-test. For matched before-and-after binary data, use McNemar's test. When you already have a chi-square value and only need the tail probability, the p-value calculator converts it for any degrees of freedom. For numeric outcomes rather than categories, see the t-test and one-way ANOVA calculators.

Frequently asked questions

What is a good chi-square value?

There is no single threshold, because the cutoff depends on degrees of freedom. With df = 2, a statistic above 5.99 gives p below 0.05. With df = 5 you need 11.07 for the same p-value. Compare the p-value, not the raw statistic, unless you know the degrees of freedom.

How many observations do I need?

A practical rule is that every expected count should be at least 5. For five categories under a uniform distribution that means at least 25 observations, since 25 / 5 = 5. More categories and more skewed expected proportions need larger samples.

Can I use it with only two categories?

Yes. With two categories and a specified proportion, the goodness-of-fit test on 1 degree of freedom is equivalent to a two-sided one-proportion z-test: the chi-square statistic equals the z statistic squared. Both give the same p-value.

Why did I get a large statistic but small per-category differences?

Sample size. The gap (O_i - E_i) is squared and scaled by E_i, which grows with the total. Doubling every count keeps the observed proportions identical but doubles the chi-square statistic, so a big sample can flag a difference that looks small in percentage terms.

Does the order of the categories matter?

No. The test sums independent contributions, so shuffling the categories leaves the statistic unchanged. If order carries meaning (for example an ordinal scale), the goodness-of-fit test ignores it, and a test built for ordered data may have more power.

Try it yourself in the tool

Related tools

Chi-Square Test of Independence

Test association between two categorical variables from a contingency table.

One-Way ANOVA Calculator

Compare the means of three or more groups with an F-test.

P-value Calculator

Convert z, t, χ² or F statistics into p-values.

t-Test Calculator

One-sample, two-sample (pooled/Welch) and paired t-tests from raw data.

A/B Test Significance Calculator

Two-proportion z-test with p-values, uplift and CI of the difference.

Fisher's Exact Test

Exact test for association in a 2×2 table — reliable for small counts.