A poor introduction to Bayesian Reasoning

In #religion

I've had my disagreements with Jonathan McLatchie a few times, some of it here, and now there is another interview Bayesian Probability and Intelligent Design: A Beginner's Guide that someone forwarded to me. In this interview he starts off by answering the fairly straightforward question "what is Bayes theorem and how does it relate to our understanding of evidence?"

Bayes theorem is a way of probabilistically modeling our assessment of evidence and how we update our confidence and conclusions in response to new data so the way that a Bayesian would conceptualize evidence is in terms of a likelihood ratio that is to say the probability of the evidence existing given the hypothesis is true on the numerator and then on the denominator we have the probability that that same evidence exists given that the hypothesis is false and to the extent that that likelihood ratio is top heavy that is the extent to which you have evidence for your hypothesis

What I find astounding about this opening response is the massive hole that Jonathan leaves -- the prior. He addresses it later, but if you're specifically asked about Bayes Theorem and you only talk about likelihoods, that is a serious omission. If I were asked the question, I would have answered it as I did in my TEDx talk on the issue:

Bayes theorem states that a belief in a particular explanation should scale with how well that explanation fits the data (called the likelihood), it should also scale with how plausible that explanation was before you saw any data (the prior), and it should be scaled down by all of the alternatives.

So why does this opener strike me as bad? There are two reasons. The first I mentioned already, the lack of mentioning a prior. We can see the result of ignoring the prior by examining the example McLatchie says here, one that he has used before,

imagine that you were hiking in a forest in somewhere and you're far away from roads and civilizations and you come across a cabin in the middle of this forest and you wonder whether it is inhabited and so you decide to inspect and you prize open the door and you find a cup of English breakfast tea that that is still steeping, it is not at room temperature, on the table inside this cabin. On the hypothesis that the cabin is inhabited would you have predicted strongly that you would find a cup of specifically English breakfast tea still seeping on this table? Well, no, it's actually quite a low probability on the hypothesis that it's inhabited that you would make that observation nonetheless it is far more probable on the hypothesis that the cabin is inhabited that you would make that observation than it would otherwise be if the cabin is not inhabited -- then it's astronomically improbable that you would make that observation. So again you have this, even though the numerator is not particularly high, you nonetheless in this case have a very strongly top-heavy likelihood ratio such that the evidence overwhelmingly confirms the hypothesis.

In my earlier response to this (where McLatchie used Earl Grey tea in his examle), I introduced the hypothesis that Jean Luc Picard had visited and everyone knows that he likes his "tea, Earl Gray, hot". Because of this, the likelihood of the evidence on this hypothesis is much higher than either of the alternatives that McLatchie gives -- does this mean he strongly believes in Jean Luc Picard? What's the problem? McLatchie consistently ignores priors. I do wonder if he changed the type of tea because of my response.

The second problem with the opener has to do with alternatives. Through rest of the interview he deals only with ratios, and he naively mentions independence,

if the initial ratio were 2 to one then if you had 10 pieces of such evidence and they were all independent then the cumulative power of the evidence would be more than 1,000 to one and so you can see that if you have lots of different pieces of evidence each by themselves not being a particularly great weight they can amount to cumulatively a very powerful cumulative case for the hypothesis

What he says is mathematically correct if 1) you have only one alternative and 2) if the data is indeed independent. The first comes from a naive application of Bayes theorem, and ignoring the fact that some non-monotonic effects can occur when you have more than two hypothesis. The second is often stated and, except for some obvious simple cases, is no often true. Even if there is a small uncertainty in whether you can establish data points as independent you can come to radically different answers.

The net effect of ignoring, or downplaying, priors and only working in two-hypothesis problems is to sweep these sorts of complexities under the rug. Applying Bayes in these cases also distracts one from the fact that the hypotheses McLatchie is proposing makes no unique predictions. Go through everything McLatchie has said, and note how many times he has proposed an experiment to test directly the design hypothesis -- it's zero as far as I can tell, not just for McLatchie but for all of the ID proponents. They will try to point out that a particular evolutionary mechanism can't explain a phenomenon, but they never put forward the positive case. It is the classic God-of-the-Gaps move.

Since McLatchie himself says he's not doing God of the Gaps and is putting forward a positive case, I feel that this topic requires its own response which I will work on later. Until then, the deficiencies in the way that he describes Bayesian probability and applies Bayes theorem itself are cause to doubt his conclusions.