ABC Tools

Chi-Square Test of Independence, Explained

After reading this you can take a table of counts, decide whether two categorical variables are associated, and report a p-value and an effect size that a reader will trust.

What the test does

The chi-square test of independence asks a single question: are two categorical variables related, or do they vary on their own? You start with a contingency table of counts. The rows are the categories of one variable, the columns the categories of the other.

Here is a hook. Suppose you email two versions of a landing page to 100 people. Version A goes to 50 people and 30 buy. Version B goes to 50 people and 15 buy. The purchase rate is 60% for A and 30% for B. The chi-square test tells you whether a gap that large is plausible under pure chance, or whether the choice of page and the choice to buy move together.

The variables here are page version (A or B) and outcome (buy or not). The test never tells you why they are linked. It only measures whether the counts stray further from independence than random sampling would usually allow.

When to use it, and when not to

Reach for this test when both variables are categorical and each observation falls into exactly one cell. Survey responses cross-tabulated by region, treatment group against recovery status, and click versus no-click by ad design all fit.

Do not use it in these situations:

  • Paired or matched data. If the same subject is counted before and after, use McNemar's test instead.
  • Small expected counts. When any expected count drops below about 5, the chi-square approximation gets unreliable. For a 2×2 table with small counts, prefer Fisher's exact test.
  • Numeric outcomes. If one variable is a number you want to compare across groups, use a t-test or one-way ANOVA.

The counts in each cell must be independent observations, not percentages, rates or averages. If you feed the calculator the numbers 60 and 30 as percentages instead of the raw counts 30 and 15, the test statistic is wrong and the p-value is meaningless.

The formula and the intuition behind it

The test compares the counts you observed against the counts you would expect if the two variables were independent. For a table with r rows and c columns, the expected count in a cell is its row total times its column total divided by the grand total.

E_{ij} = \frac{R_i \cdot C_j}{N}

Here R_i is the total of row i, C_j is the total of column j, and N is the grand total of all cells. That formula is just the definition of independence written in counts: if the row and column are unrelated, the share in a column should be the same across rows.

The statistic sums the squared gap between observed and expected, scaled by the expected count.

\chi^2 = \sum_{i,j} \frac{(O_{ij} - E_{ij})^2}{E_{ij}}

Each O_{ij} is an observed count and each E_{ij} is the matching expected count. Dividing by E_{ij} means a gap of 5 matters more where you only expected 10 than where you expected 200. When observed equals expected everywhere, \chi^2 = 0. Larger values mean more departure from independence.

You compare the statistic to a chi-square distribution with degrees of freedom set by the table shape.

df = (r - 1)(c - 1)

A 2×2 table has df = 1. A 3×4 table has df = 6.

A worked example with the demo data

Testing the 2×2 demo table

The demo button loads this table:

Observed counts
Column 1Column 2Row total
Row 1302050
Row 2153550
Column total4555100
  1. Compute expected counts. Top-left is 50 \times 45 / 100 = 22.5. The others follow: top-right 27.5, bottom-left 22.5, bottom-right 27.5.
  2. Find each term. Top-left contributes (30 - 22.5)^2 / 22.5 = 56.25 / 22.5 = 2.5. By symmetry every cell contributes 2.5.
  3. Sum them: \chi^2 = 2.5 \times 4 = 10.0.
  4. Degrees of freedom: (2-1)(2-1) = 1.
  5. The p-value for \chi^2 = 10.0 at 1 degree of freedom is about 0.00157.

That p-value is well below 0.05, so you reject independence. Row and column are associated: in row 1, 60% of the counts sit in column 1, but in row 2 only 30% do.

Each observed bar sits 7.5 counts away from its expected bar. That constant gap is why every cell contributes the same 2.5 to the statistic.

Reading the results

The calculator returns four things worth reading together.

Chi-square statistic
The total departure from independence. On its own it is hard to judge, because its scale depends on table size and sample size.
Degrees of freedom
Set by the table shape, not the data. It tells you which chi-square curve to compare against.
p-value
The probability of seeing a statistic at least this large if the variables were truly independent. Small means the data are hard to explain by chance alone.
Cramér's V
An effect size between 0 and 1 that does not grow with sample size. It answers "how strong is the association", which the p-value does not.

Cramér's V is computed from the statistic:

V = \sqrt{\frac{\chi^2}{N \cdot (k - 1)}}

Here N is the grand total and k is the smaller of the row and column counts. For the demo, V = \sqrt{10 / (100 \times 1)} = 0.316, a moderate association. Rough guides for df = 1: 0.1 is small, 0.3 is medium, 0.5 is large.

Always report Cramér's V alongside the p-value. With 100,000 observations, a trivial 0.5% difference in rates can produce a tiny p-value while V stays near 0.01. Significance is not the same as importance.

How the p-value moves with the counts

Use the widget to see how the same rate difference becomes convincing as the sample grows. A 60% versus 30% split with 10 people per row is not significant. The same split with 50 people per row is.

With a fixed 60% vs 30% column-1 rate, the chi-square statistic grows in proportion to the sample size. At 10 per row the statistic is about 1.9 (p about 0.17), at 30 per row about 5.6 (p about 0.018), and at 50 per row about 9.4 (p about 0.0022). Larger samples turn the same effect into a smaller p-value.

Common mistakes

  • Feeding percentages or rates. The math assumes raw counts. Convert back before you test.
  • Ignoring small expected counts. Check the expected table. If any cell falls below 5, treat the p-value with suspicion and switch to Fisher's exact test for a 2×2 layout.
  • Reading direction into it. The test does not say row 1 causes column 1. It says the two variables are not independent.
  • Testing many tables and reporting only the significant one. Run 20 independent tests on random data and about one crosses 0.05 by chance. Decide your tests in advance.
  • Confusing goodness-of-fit with independence. If you are comparing one set of counts to a fixed expected distribution, you want the goodness-of-fit test, not this one.

Related tools

For a 2×2 table comparing two proportions with a confidence interval on the difference, the A/B test calculator gives the same conclusion in proportion language, and the one-proportion z-test handles a single rate against a target. If you already have a chi-square statistic and just need the tail area, the p-value calculator converts it. For ranked or ordinal data across groups, look at the Kruskal-Wallis test or the Mann-Whitney U test.

Frequently asked questions

What is a good p-value for a chi-square test?

There is no universal cutoff. Many fields use 0.05, so a p-value below that leads you to reject independence. The demo p-value of 0.00157 is far below 0.05, which is strong evidence of association. Fix your threshold before you see the data.

Do I need Yates' continuity correction?

Some textbooks apply it to 2×2 tables to reduce the chi-square value slightly. It makes the test more conservative but often too conservative. For small 2×2 tables the cleaner choice is Fisher's exact test rather than a corrected approximation.

What if a cell count is zero?

A zero observed count is fine as long as the expected count for that cell stays at 5 or above. If the expected count is small, the approximation breaks down and you should use Fisher's exact test.

Can I use it for a table bigger than 2×2?

Yes. A 3×3 or 4×5 table works the same way. The degrees of freedom grow as (r-1)(c-1), and Cramér's V uses k-1 where k is the smaller dimension, so it stays comparable across table shapes.

Does a significant result prove causation?

No. It shows the two variables move together in your data. A lurking third variable could drive both. Only a controlled design supports a causal claim.

Try it yourself in the tool

Related tools

One-Way ANOVA Calculator

Compare the means of three or more groups with an F-test.

P-value Calculator

Convert z, t, χ² or F statistics into p-values.

t-Test Calculator

One-sample, two-sample (pooled/Welch) and paired t-tests from raw data.

A/B Test Significance Calculator

Two-proportion z-test with p-values, uplift and CI of the difference.

Chi-Square Goodness-of-Fit Test

Test whether observed counts match an expected distribution.

Fisher's Exact Test

Exact test for association in a 2×2 table — reliable for small counts.