/ lab / bayes

Bayesian, the noun

Three small problems. Each one ends with a probability distribution over the thing you didn't know. Whatever you want to know after that is just an integral over it. The framework I work on at my day job dresses this up; the dressing is there so the same trick scales up to models that don't fit on a page.

Where are the keys?

Twenty-five rooms. The shading is where you think you left them -- your prior. Click a room to look. If they're not in there, the room's mass collapses to zero and the rest of the grid soaks it up. Same total probability, rearranged. Bayes' rule is the rearrangement.

7% 9% 7% 3% 1% 9% 11% 9% 4% 1% 7% 9% 7% 3% 1% 3% 4% 3% 1% 1% 1% 1%

Searches: 0

This is the rule that found the USS Scorpion in 1968 and Air France 447 in 2011. A prior over the ocean floor, narrowed by every place the search team looked and didn't find anything. Same trick, fewer rooms.

A rate, learned from years

Earthquakes per year in the Bay Area, where I live; not a textbook example for me. Each year is a draw from Poisson(rate); the rate itself is what you actually want, and you don't know it. The dim curve is what you'd guess about it before any data shows up. Click observe a year and watch the bright curve narrow as the years stack up.

0 2 4 6 8 10 12 Rate (events / year) Density mean 5.0

0 years · 0 total events · posterior mean 5.0

Closed-form here: a Gamma(2.0, 0.4) prior times a Poisson likelihood is another Gamma, Gamma(2.0, 0.4). Same shape, sharper. For models without a clean conjugate, the framework does the same thing numerically; the answer is the same kind of object either way.

Asking the posterior a question

You have a posterior; the section above made one. Reducing it to a single number ("the rate is 3.5") throws away most of what's there. The real questions usually look more like: is it above some threshold I care about? The posterior already knows. Move the threshold; read the integral off the curve.

0 2 4 6 8 10 12 Rate (events / year) Density P(rate > 5.0) = 37.8%
5.0

The shaded area is P(rate > threshold | data). Same posterior as the section above; observe more years there and watch the answer here move. Every other question you'd ask about the rate is just a different integral over the same object.