p-Hacking Simulator illustration

p-Hacking Simulator

This simulator shows how testing many hypotheses manufactures false positives. Every experiment draws two groups from the same distribution — so the true effect is exactly zero — and computes a p-value. Run one test and it is probably not significant; run twenty and at least one crosses p < 0.05 most of the time. A histogram shows the p-values are uniform under the null, the significant ones are flagged red, and a counter tracks how often pure noise hands you a publishable-looking result.

Runs 100% in your browser — simulations are computed locally on your device.

Notes

  • Under the null hypothesis p-values are uniform on [0, 1], so each test has a 5% chance of falsely clearing p < 0.05.
  • Running k independent tests, the chance of at least one false positive is 1 − 0.95^k — about 64% at k = 20.
  • Corrections like Bonferroni (test at 0.05/k) or controlling the false discovery rate rein this in.
  • Runs 100% in your browser — simulations are computed locally on your device.