Push Back Against Bayes

In #articles

In his blog post about Bayes' theorem part 1 and part 2, John Loftus wrestles with the notion of prior probability and the application of Bayes' Theorem. His concern seems to be primarily with the idea that there should be some things that are impossible (e.g. pigs flying) and that one shouldn't even consider these claims. He then feels that people using Bayes' theorem in religious arguments are using it both to give the veneer of credibility but also so that they can attempt to justify impossible things - clearly an abuse.

Although I share Loftus' concern about abusing probability, I don't think he is viewing the problem properly. For example, he introduces one form of Bayes' theorem: P(A∣B)=P(B∣A)P(A)P(B) P(A|B) = \frac{P(B|A)P(A)}{P(B)} but then also states

Notice that the prior probability of event "B" cannot be zero."

He thinks this restricts the use of the theorem, continuing with

Anything that is given a zero prior probability is not the subject for Bayes' Theorem. Got it? To use it in cases where there is a zero probability is to use it incorrectly. That's the point, not that every claim has a nonzero prior probability to it.

Clearly P(B)≠0P(B)\neq 0 - the theorem refers to the probability of AA given that BB is true. It wouldn't then make any sense to have P(B)=0P(B)=0. Anyway, we are't talking about the prior probability of BB at all so this seems like a misdirected criticism.

As for claims with zero prior probability, these occur only for logically impossible claims, not simply outlandish claims.

Loftus continues with

"Let's say you know one student in a class of twenty has the flu. Then the prior probability that a student in that class named Sally has the flu, is 1/20. That is your prior probability." Notice you have some factual information, that is, one student in a class of twenty has the flu. This is significant. First comes data, then comes prior probabilities. Bayes is dealing with factual data from the beginning. Without it there is nothing to compute. Compute?

The term prior probability refers to prior to the data. But what is the data? For one problem - as with the flu case - it might be the flu test on an individual, in which case the prior to that data may include the data on general infection numbers. In the case of a coin flip, the prior probability of P(H)=1/2P(H)=1/2 comes from the symmetry of the coin leading to the fact that relabeling the sides results in an identical state of knowledge - and thus an identical probability assignment. Sometimes there is data before the prior, sometimes not.

Finally, Loftus states his biggest concern:

Are the following claims logically impossible?

  • Can I ride Disney's elephant Dumbo?
  • When watching a Batman movie can the Joker leap out and attack me?

others he adds to this list include:

Can I run a one second mile today, unaided by any technology and discounting the fact that I'm running on a moving earth traveling many miles per second around the sun?

Can I build a time machine before sundown today and travel back in time to prevent my grandparents from ever meeting, then come back prior to the time I built the time machine?

Can I build a spaceship and travel to the moon and back before sundown today? Hell, I don't have the money to do this, nor the know-how, nor access to the fuel needed without being arrested. I don't think even NASA could do this if the head received a directive to do this completely from scratch, even with the knowledge and materials. NASA wouldn't even attempt it due to safety concerns for the astronaut(s) inside.

Can pigs fly? Can they propel themselves through the sky without the aid of any technology? And for naysayers, a pig is a pig is a pig. They do not have the means to fly. And no, flying doesn't mean being thrown off a cliff, or riding in a plane.

Surely we would say that we know these are impossible, in the same sense of "know" that Stephen Gould stated:

In science, “fact” can only mean “confirmed to such a degree that it would be perverse to withhold provisional assent.” I suppose that apples might start to rise tomorrow, but the possibility does not merit equal time in physics classrooms.

But how did we come to this? There are cases where people have claimed that pigs can fly, and I would bet that may children might entertain this as a possibility. It is through updating our state of knowledge to such a degree that we can demonstrate that any of these claims being true would undermine many other things we hold as true, and thus are unlikely in themselves. John Loftus is bringing a lot of background knowledge to the table, which he feels should bypass any probabilistic argument. Further, he asserts that the probabilistic arguments add nothing to it but tend to confuse the topic. Fair enough.

But, what if people don't agree on a conclusion? How does one adjudicate this without some formal structure to the argument? What structures do we have access to other than probability theory?

The problem with using Bayes as a tool to evaluate magical mythical miraculous claims is that there are no hard data to go by. If one wants to calculate how many Firestone tires will go bad on Ford trucks, there are data to work with. We can calculate how many tires go bad generally speaking, and how many tires go bad on trucks, etc. Then we can state the probabilities. There's data to make a probability calculation. Magic doesn't provide any data from which to calculate.

Here, Loftus is in fact wrong - there is data. It turns out that the data is always of the worst kind - anecdotal. For example, when pressed, Christians will often cite Craig Keener's tome of Miracle claims. It comes in over 1100 pages! This is actual data, but science has demonstrated that it is the worst kind of data and that our standards are higher now. What is the probability that miracles are actually occurring, and yet we have zero instances of it happening in a controlled setting or in a setting where we have careful pre- and post-miracle evidence? This is an application of Bayes' theorem, whether we couch it in numerical terms or not.

Loftus may be correct that in many cases Bayes' theorem is used to hide this fact, but in those cases I think it then becomes the best tool to demonstrate the fact. For those people that use Bayes to hide their poor reasoning, it is more likely that we can counter using the same tool.