ABC Tools

Bayesian Updating for a Coin's Bias, Explained

After reading this you can take any Beta prior for a coin's bias, fold in a batch of flips, and write down the exact posterior, its mean, its MAP and its 95% credible interval by hand.

What Bayesian updating does here

You have a coin with an unknown bias p, the probability it lands heads on a single flip. You want to learn p from flips. Bayesian updating starts with a distribution over p that encodes your belief before any data, then multiplies that belief by the evidence to get an updated belief.

The hook: start with a flat prior that says every bias from 0 to 1 is equally plausible. Flip once and see heads. Your belief immediately shifts to favor higher biases, with the density rising linearly from 0 at p=0 to a peak at p=1. Flip 100 times, see 63 heads, and the belief becomes a narrow bump centered near 0.63 with almost no weight below 0.5. Nothing about the coin changed. Your description of what you know sharpened.

The reason this coin problem is worth studying is that it has an exact closed-form answer. No sampling, no approximation. A Beta prior times a Bernoulli likelihood gives a Beta posterior, and the update is pure counting.

When this exact method applies, and when it does not

The clean arithmetic below works when three things hold. Each flip is either a success or a failure (Bernoulli). Flips are independent given the bias. And your prior is a Beta distribution. When those hold, you have a conjugate pair: the posterior stays in the same Beta family as the prior, so updating never leaves closed form.

This is a teaching model, not a production pipeline. Real coins are close to fair by physics, so a coin demo is a stand-in for any binary rate: click-through, conversion, defect rate, a diagnostic test's true-positive rate. The moment your outcome has more than two categories, or observations depend on each other, or the quantity you want is a function of several unknowns, the tidy Beta arithmetic stops applying and you reach for numerical methods.

Conjugacy is a mathematical convenience, not a modeling requirement. Bayesian inference is defined for any prior and any likelihood. The Beta-Bernoulli case just happens to make the answer a formula you can check on paper, which is exactly why it is the right place to build intuition.

The formula and why it is just counting

A Beta prior with parameters \alpha and \beta has density proportional to p^{\alpha-1}(1-p)^{\beta-1}. After observing h heads and t tails, the Bernoulli likelihood is proportional to p^{h}(1-p)^{t}. Multiply them:

p^{\alpha-1}(1-p)^{\beta-1} \cdot p^{h}(1-p)^{t} = p^{(\alpha+h)-1}(1-p)^{(\beta+t)-1}

The exponents just add. So the posterior is another Beta:

\text{Beta}(\alpha + h,\ \beta + t)

Here \alpha and \beta act as pseudo-counts: \alpha plays the role of heads you imagined before flipping, \beta the role of tails. A uniform prior is \text{Beta}(1,1), worth one imagined head and one imagined tail. The Jeffreys prior is \text{Beta}(0.5, 0.5).

Two summaries follow directly. The posterior mean is

\text{mean} = \frac{\alpha + h}{\alpha + \beta + h + t}

which is a weighted average of the prior mean \alpha/(\alpha+\beta) and the sample frequency h/(h+t). The MAP (the peak of the density, its most probable value) is

\text{MAP} = \frac{\alpha + h - 1}{\alpha + \beta + h + t - 2}

valid when both posterior parameters exceed 1. The mean and MAP differ most when data are scarce and agree closely once counts are large.

A worked example that reproduces the demo

Take the demo defaults: a uniform prior \text{Beta}(1,1) and a true bias of 0.6. Suppose 100 flips produce 63 heads and 37 tails, a plausible batch for that bias.

Uniform prior, 63 heads in 100 flips

  1. Start: \alpha=1, \beta=1. Prior mean 1/2 = 0.5.
  2. Add counts: posterior is \text{Beta}(1+63,\ 1+37) = \text{Beta}(64, 38).
  3. Posterior mean: 64/(64+38) = 64/102 = 0.6275.
  4. MAP: (64-1)/(102-2) = 63/100 = 0.63, which equals the raw sample frequency.
  5. Posterior standard deviation: \sqrt{\frac{64 \cdot 38}{102^2 \cdot 103}} = 0.0478.
  6. The 95% credible interval, taken from the 2.5% and 97.5% quantiles of \text{Beta}(64,38), runs about 0.533 to 0.718. A normal approximation, mean \pm\ 1.96 standard deviations, gives 0.534 to 0.721, close because n is large.

Notice the mean 0.6275 sits between the prior mean 0.5 and the sample frequency 0.63, but almost on top of the data. With 100 flips against a prior worth 2 pseudo-flips, the data outweigh the prior by 50 to 1. The chart below shows the flat prior turning into that tight bump.

The shaded band is the 95% credible interval, 0.533 to 0.718. The vertical marker is the posterior mean 0.6275. The density is near zero below 0.45 and above 0.78.

Watch the interval tighten with n

The single most instructive move is to hold the sample frequency fixed and grow the number of flips. The mean barely moves; the interval collapses. Width shrinks roughly like 1/\sqrt{n}, so getting the interval half as wide takes about four times the data.

With a uniform prior and 60% heads, Beta(7,5) after 10 flips has a 95% interval of about 0.35 to 0.85 (width 0.50). At 50 flips, Beta(31,21) narrows to about 0.47 to 0.73 (width 0.26). At 200 flips, Beta(121,81) narrows to about 0.53 to 0.66 (width 0.13). Each fourfold increase in flips roughly halves the width.

Reading the readouts

Posterior mean
The expected bias, (\alpha+h)/(\alpha+\beta+h+t). Use it as your single best point estimate when squared error is the cost.
MAP
The most probable single value, the density peak. With a uniform prior the MAP equals the raw heads fraction, so 63/100 = 0.63 above.
95% credible interval
A range that holds 95% of the posterior probability. Unlike a frequentist confidence interval, you may say directly that 0.533 to 0.718 has 95% posterior probability of containing the true bias, given the prior and data.

Two priors that both put nonzero density on the truth converge to nearly the same posterior once the data pile up. A skeptical \text{Beta}(20,20) (prior mean 0.5, worth 40 pseudo-flips) and a uniform prior disagree sharply at 10 flips but agree closely at 1000. The data wash out the prior.

Updating flip by flip and updating once with the whole batch give the identical posterior, because the exponents in the likelihood simply add. Beta(1,1), then +1 head three times, lands on Beta(4,1), exactly what one batch of 3 heads gives. Order does not matter either.

Common mistakes

Reading a credible interval as a confidence interval. They answer different questions. The credible interval is a statement about p given your prior; a confidence interval is a statement about the procedure's long-run coverage. They often look similar numerically with a flat prior and large n, but the interpretation differs.

Choosing a prior far stronger than you realize. \text{Beta}(20,20) is not mildly skeptical. It is worth 40 flips. If you only plan to observe 10 flips, that prior dominates the result. Match the prior's pseudo-count \alpha+\beta to how much you actually know.

Confusing MAP with mean when data are thin. After a single head from \text{Beta}(1,1) you have \text{Beta}(2,1): mean 2/3 \approx 0.667 but MAP = 1. Reporting the MAP of 1 as "the coin always lands heads" after one flip is nonsense. The mean is more honest when counts are small.

Assuming the interval measures the coin's true uncertainty. It measures your uncertainty given the model. If flips are not independent, the interval understates how little you really know.

Width falls fast then flattens. From n=50 to n=200 (a fourfold increase) width drops from about 0.256 to 0.131, roughly a halving, matching the 1/sqrt(n) rule.

Related tools

Once you can turn evidence into a posterior probability, several other ideas connect. To act on a predicted probability you need a threshold; the ROC & Threshold Explorer shows how the confusion matrix and expected cost move as you slide it. To check that predicted probabilities mean what they claim, the Calibration & Reliability Diagram compares stated probabilities against observed frequencies. And the prior-versus-data tension here is the same tradeoff studied in the Bias–Variance Decomposition, where a strong prior lowers variance at the cost of bias.

Frequently asked questions

What prior should I use if I know nothing?

The uniform prior \text{Beta}(1,1) and the Jeffreys prior \text{Beta}(0.5,0.5) are the two standard weak choices. With more than a few dozen flips they give nearly identical posteriors, so the choice rarely matters once you have data.

Why does the posterior mean not equal the heads fraction?

Because the prior contributes pseudo-counts. With \text{Beta}(1,1) and 63 heads in 100, the mean is 64/102 = 0.6275, slightly pulled toward the prior mean 0.5. The MAP equals the raw fraction 0.63 under a uniform prior; the mean does not.

Does the order of flips change the answer?

No. Heads then tails gives \text{Beta}(1,1) \to \text{Beta}(2,1) \to \text{Beta}(2,2), the same as tails then heads. Only the totals h and t enter the posterior.

How many flips until my prior stops mattering?

Roughly when h+t is much larger than \alpha+\beta. A prior worth 2 pseudo-flips is drowned out by 50 real flips; a prior worth 40 pseudo-flips needs several hundred flips before the data dominate.

Is a credible interval the same as a confidence interval?

No. A 95% credible interval has 95% posterior probability of containing p, given the prior and data. A 95% confidence interval is a procedure that covers the truth 95% of the time across repeated experiments. They can match numerically but answer different questions.

Try it yourself in the tool

Related tools

ROC & Threshold Explorer

Drag a decision threshold and watch the confusion matrix, ROC curve and expected cost respond in real time.

Bias–Variance Decomposition

See bias², variance and noise split a model’s error as 50 fits wobble around the truth.

Cross-Validation Visualizer

Animate k-fold splits and see why cross-validation beats a single train/test split.

Calibration & Reliability Diagram

Check whether predicted probabilities mean what they say, then fix them with temperature scaling.

Curse of Dimensionality

Watch distances concentrate and nearest neighbors fail as dimensions pile up.

Feature Scaling Demo

Run k-means and gradient descent on raw versus scaled features and watch scaling decide the outcome.