← Back to the blog
Epistemics · July 14, 2026

Why Your Bayesian Analysis Disagrees with Mine

Why Your Bayesian Analysis Disagrees with Mine

Six careful people looked at the same evidence about where COVID-19 came from and produced final odds spanning twenty-three orders of magnitude. What does that gap say about the method they all used?

The method was Bayesian reasoning, and the six were not cranks. They were the two debaters, Peter Miller arguing for zoonosis and Saar Wilf for a lab leak; the two judges recruited for the exchange, Will van Treuren, a microbiologist and immunologist with a PhD from Stanford, and Eric Stansifer, an applied mathematician from MIT with a background in mathematical virology; Scott Alexander, who ran his own numbers while following along and later wrote the whole affair up; and Daniel Filan, who tracked the debate on his own and produced a separate estimate. Each had access to the same fifteen hours of argument, the same papers, the same rebuttals. Their final estimates, expressed as odds favoring one origin theory over the other, differed by twenty-three orders of magnitude. Drop the most extreme outlier and seven orders of magnitude still separate the rest. Drop both debaters and a factor of fifty still separates the four neutral analysts who remain.

The tempting reaction is to say the six reasoned badly, and that a better-run analysis would converge. The opposite reading holds up better, though it deserves to be held loosely. Bayes's theorem is not at fault. It is exactly right as a rule for updating a belief once a likelihood ratio and a prior are fixed. The trouble sits upstream, in the human judgment calls that produce the numbers fed into the formula. The machinery does not remove those calls. It launders them into a decimal that looks more objective than it is.

Stansifer, one of the two judges, wrote in his published decision that he doubted the Bayesian calculation was even the right tool for the question, and had attempted the formal math only out of fairness, after criticizing the other side's numbers at length. Van Treuren, the other judge, put the point differently. He tried both an intuitive read of the evidence and a full probabilistic breakdown, and was relieved when the two agreed. Then he added a warning worth sitting with: Bayesian methods, in his experience, maximize legibility rather than objectivity, and bury the intuitive, heuristic part of the reasoning inside the choice of what evidence to admit and what number to attach to it, exactly where it is hardest to see.

This is the crux. A likelihood ratio has to be estimated by a person who decides how surprising a given fact would be under each hypothesis. A prior has to be set by a person who decides how the case should be framed before any evidence arrives. Get the frame wrong and the formula carries the error forward faithfully, with three decimal places of false precision. Philosophers have a name for the deepest version of the difficulty. The reference class problem, traced to Hans Reichenbach and given its sharpest modern statement by Alan Hájek, turns on a simple observation: almost any individual event belongs to more than one reference class, and each class yields a different probability. Probability theory contains no formal rule for picking the right one. The debate ran straight into this. Miller argued that the odds of a pandemic first surfacing at a wet market, supposing the true cause were a lab accident, should be measured against the small fraction of a city's population who work at such a market. Wilf objected that this was the wrong comparison class, and borrowed the logic of a familiar fallacy from forensic testimony. A vanishingly small probability of an innocent match tells you nothing on its own about whether the match is meaningful, since it ignores every other way the number could have arisen. Choosing the comparison class settled half the argument before either side touched a likelihood ratio.

The debate also produced a cleaner illustration of how far the problem runs. Miller put up a slide that multiplied twenty separate low-probability judgments into a single figure of roughly one in fifty trillion trillion, a number so extreme that winning a lottery three times running would be the likelier outcome. He may have meant the slide partly as provocation, and he did not rest his case on it. The pattern it displays is a known failure mode, sometimes called the multiple stage fallacy. Chain enough individually plausible low probabilities without accounting for how they correlate, and the product comes out far more extreme, and far less trustworthy, than any of its parts.

None of this makes the exercise worthless, and the counterargument deserves its strongest form. Making the reasoning explicit, however imperfectly, is what let outside observers see where the six estimates diverged and by how much. A purely intuitive debate would have hidden that. Philip Tetlock's decades of forecasting research point the same way. Ordinary experts reasoning informally about complex and contested questions are often barely better than chance, while forecasters trained to break a question into components, attach explicit numbers, and revise them as evidence arrives do measurably better over time. When the alternative to a flawed formal method is an unexamined informal one, the formal method keeps its claim on our attention.

The claim holds, and it does not rescue the twenty-three orders of magnitude. A method that beats unaided intuition on average, across many questions, can still fail on the one question where the reference class is genuinely contested and the framing carries high stakes. COVID origins was that kind of question. Performing somewhat better than a guess falls well short of the standard needed to report a number to three decimal places and expect a stranger to trust it.

The honest posture is to treat a Bayesian estimate the way a careful reader treats a fluent summary. It is useful for surfacing where the disagreement lives, and insufficient on its own to settle what to believe. When two analysts trained in the same method reach wildly different final numbers, the gap is the finding. It shows that the dispute has not bottomed out in a shared reference class or a shared way of weighing the evidence, whatever the decimals suggest. Report the divergence rather than averaging it away, then go looking for the specific judgment call that drives it. That call, and not the arithmetic around it, is where the real disagreement sits.

Formal calibration keeps its place. In domains tested against real outcomes many times over, statistical models have repeatedly beaten informal expert judgment working from the same inputs. The result goes back to Paul Meehl in 1954 and has held up across decades of follow-up. Calibration disciplines a method inside a chosen frame. It cannot certify the frame itself, and no amount of repeated testing makes a method immune to revision once the reference class it leans on turns out to be wrong. The COVID case offered no such repeated test and no prior calibration. That absence is worth naming plainly, rather than papering it over with a confidence the situation never earned.

The case for this reading looks strong, and it is worth holding loosely, because the literature on when formal methods help and when they mislead is still contested among people well versed in it. One discipline would help anyone who applies Bayesian reasoning to a contested real-world question. Name the reference class you chose, name the ones you rejected, and say why. That single sentence would have done more for the COVID debate than another decimal place ever would.

Where else has a formal method produced false confidence in place of real clarity? The counterexamples would teach more than any assumption that this holds everywhere.


REFERENCES

Alexander, S. (2024). Practically-A-Book Review: Rootclaim $100,000 Lab Leak Debate. Astral Codex Ten. astralcodexten.com/p/practically-a-book-review-rootclaim

Hájek, A. (2007). The Reference Class Problem Is Your Problem Too. Synthese, 156(3), 563-585.

Meehl, P. E. (1954). Clinical Versus Statistical Prediction: A Theoretical Analysis and a Review of the Evidence. University of Minnesota Press.

Reichenbach, H. (1949). The Theory of Probability. University of California Press.

Tetlock, P. E., and Gardner, D. (2015). Superforecasting: The Art and Science of Prediction. Crown.

Tetlock, P. E. (2005). Expert Political Judgment: How Good Is It? How Can We Know? Princeton University Press.