p-Hacking Simulator
This simulator shows how testing many hypotheses manufactures false positives. Every experiment draws two groups from the same distribution — so the true effect is exactly zero — and computes a p-value. Run one test and it is probably not significant; run twenty and at least one crosses p < 0.05 most of the time. A histogram shows the p-values are uniform under the null, the significant ones are flagged red, and a counter tracks how often pure noise hands you a publishable-looking result.
Runs 100% in your browser — simulations are computed locally on your device.
Read the full guide to this tool
Notes
- Under the null hypothesis p-values are uniform on [0, 1], so each test has a 5% chance of falsely clearing p < 0.05.
- Running k independent tests, the chance of at least one false positive is 1 − 0.95^k — about 64% at k = 20.
- Corrections like Bonferroni (test at 0.05/k) or controlling the false discovery rate rein this in.
- Runs 100% in your browser — simulations are computed locally on your device.