Bayes' Theorem Formula
Interactive Bayes Calculator
See Bayes' theorem in action! Adjust the Prior P(A), Sensitivity P(B|A), and False Positive Rate P(B|¬A) to calculate the Posterior P(A|B). This demonstrates the classic medical testing problem: even with a 95% accurate test, a positive result might not mean what you think if the condition is rare!
The Fundamental Equation
Bayes' theorem relates conditional probabilities. It tells us how to update our belief in A after observing B. The posterior P(A|B) combines our prior belief P(A) with the likelihood P(B|A) of seeing the evidence if A is true, normalized by the total probability of the evidence P(B).
The Logic of Updating
Bayesian reasoning is about learning from evidence. We start with a prior belief (before data). We observe evidence and ask: how likely is this evidence under our hypothesis? Then we update to a posterior belief. Strong evidence shifts beliefs more; weak evidence shifts them less.
Breaking Down the Parts
Each component has a specific role: Prior captures existing knowledge, likelihood measures how well the hypothesis predicts the data, evidence normalizes probabilities, and posterior is the result. The process is iterative - today's posterior becomes tomorrow's prior.
Medical Testing Example
The Classic Medical Test
This classic problem demonstrates base rate neglect. A test with 99% sensitivity and 95% specificity sounds excellent. But the answer depends heavily on disease prevalence. Most people guess 95% or 99% probability of having the disease after testing positive. The actual answer surprises them.
Applying Bayes' Theorem
Despite testing positive with a 99% sensitive test, there's only a 16.7% chance you have the disease! Why? Because the disease is rare (1%), most positive tests come from the 5% false positive rate applied to the 99% healthy population. False positives vastly outnumber true positives.
Tree Diagram View
Visualizing with natural frequencies: Of 1,000 people, 10 have the disease (1%). Of these 10, about 10 test positive (99% sensitivity). Of 990 healthy, about 50 test positive (5% false positive). Total positives: 60. Only 10 are true cases. So 10/60 = 16.7%.
Why This Matters
Base rate neglect leads to real-world errors: unnecessary medical procedures from screening, wrongful convictions, ineffective security theater, and non-replicable research. Understanding Bayes' theorem helps evaluate claims critically and make better decisions under uncertainty.
Choosing Priors
Types of Prior Distributions
Priors encode existing knowledge. Uniform priors assume no prior knowledge (rarely appropriate). Informative priors use previous research or expert knowledge. Weakly informative priors provide mild regularization without strong assumptions - often a good default.
Does Prior Choice Matter?
Prior
Prior
Prior
Prior choice matters most when data is limited. With abundant data, the likelihood dominates and posteriors converge regardless of prior. This is a feature: Bayesian inference is self-correcting. For small samples, check sensitivity to prior choice.
Best Practices for Priors
Good prior selection: use existing knowledge, document choices, test sensitivity to alternatives, avoid flat priors (they're not actually non-informative). Bad practice: using the same data to choose priors and fit models (double-dipping), or using improper priors that don't integrate to 1.
Sequential Updating
Learning Over Time
Bayesian updating is sequential: each posterior becomes the prior for the next observation. Flip a coin: start at 50% fair. See heads, update to favor heads-biased. See more heads, confidence grows. This mirrors how we naturally learn from experience over time.
Coin Fairness Example
| Flip | Result | P(Fair) | Trend |
|---|---|---|---|
| 0 | — | 50.0% | Prior |
| 1 | H | 45.5% | ↓ |
| 2 | H | 41.2% | ↓ |
| 3 | T | 44.8% | ↑ |
| 4 | H | 40.7% | ↓ |
| 5 | H | 36.9% | ↓ |
After each flip, we update our belief about the coin being fair. Four heads and one tail makes us more suspicious of bias. Each observation shifts the posterior slightly. With enough data, we'll become highly confident about the coin's true bias.
Eventually Reaching Truth
regardless of starting prior
A key property of Bayesian updating: with sufficient data, posteriors converge to the truth regardless of the starting prior (as long as the prior assigns nonzero probability to the truth). The data eventually overwhelms any reasonable prior.
Bayesian vs Frequentist
Two Views of Probability
Frequentists see probability as long-run frequency and parameters as fixed unknowns. Bayesians see probability as degree of belief and parameters as uncertain quantities with distributions. Neither is 'right' - they answer different questions and suit different contexts.
Confidence vs Credible Intervals
Confidence intervals describe procedure reliability. Credible intervals give direct probability statements about parameters. Most people interpret CIs as if they were credible intervals - that's technically wrong for CIs but exactly correct for credible intervals.
P-values vs Bayes Factors
P-values only tell us how surprising data is under the null. Bayes factors compare evidence for competing hypotheses directly. BF = 10 means data is 10× more likely under H₁ than H₀. Unlike p-values, Bayes factors can provide evidence FOR the null hypothesis.
Choosing Your Approach
Both approaches are valid for different contexts. Bayesian methods excel with prior knowledge, small samples, and complex models. Frequentist methods are standard in regulated fields and when priors would be contentious. Many modern analyses use both.
Real-World Examples
Apply your knowledge to practical scenarios
Email Spam Classification
A spam filter uses Bayesian classification. Given: 20% of all emails are spam (prior). The word 'FREE' appears in 80% of spam emails but only 5% of legitimate emails. An email contains 'FREE'. What's the probability it's spam?
Sample Data
{
"prior_spam": 0.2,
"p_free_given_spam": 0.8,
"p_free_given_ham": 0.05
}Key Insights
- Bayesian classification updates beliefs with each piece of evidence
- Multiple weak signals can combine to give strong classification
- This is exactly how Naive Bayes spam filters work in practice
The Prosecutor's Fallacy
A DNA test has a false positive rate of 1 in 1 million. A suspect's DNA matches evidence from a crime scene. The prosecutor argues: 'The probability of this match by chance is 1 in 1 million, so the suspect is almost certainly guilty.'
Key Insights
- P(evidence|hypothesis) ≠ P(hypothesis|evidence)
- Base rates matter enormously—even 'rare' events occur in large populations
- This fallacy has led to wrongful convictions in real cases
Breast Cancer Screening Decisions
A 50-year-old woman with no family history gets a positive mammogram. Prevalence in her demographic: 0.8%. Mammogram sensitivity: 90%. Specificity: 93%. What's the probability she actually has breast cancer?
Sample Data
{
"prevalence": 0.008,
"sensitivity": 0.9,
"specificity": 0.93
}Key Insights
- Positive screening tests are often false positives when disease is rare
- Natural frequencies (out of 10,000) are easier to understand than probabilities
- Doctors should communicate results using Bayesian reasoning
Bayes' Theorem Quiz
Test your understanding • Passing score: 70%