Bayesian Analysis and its Misuses

In Paulogia's response video, he defines Bayesian analysis as:

"Bayesian analysis is a quantification and mathematical description of presuppositions and natural decision-making processes"

which Than and others took umbrage. They suggested these definitions:

Titelbaum's definition: "There is no single view that is Bayesian Epistemology; instead, there are a number of Bayesian epistemologies. Every view I would call a Bayesian epistemology endorses the following two principles: 1. Agents have doxastic attitudes that can usefully be represented by assigning real numbers to claims. 2. Rational requirements on those doxastic attitudes can be represented by mathematical constraints on real numbers closely related to the probability calculus."

Streven's definition: "There are three basic elements to bct. First, it is assumed that the scientist assigns what we will call credences or subjective probabilities to different competing hypotheses...Second, the credences are assumed to behave mathematically like probabilities... Third, scientists are assumed to learn from the evidence by what is called the Bayesian conditionalization rule...The conditionalization rule directs you to update your credences in the light of new evidence in a quantitatively exact way..."

SEP Entry: "Bayesian epistemologists study norms governing degrees of beliefs, including how one's degrees of belief ought to change in response to a varying body of evidence."

Each of these is fine, albeit a bit wordy. I prefer an E. T. Jaynes style definition like:

Bayesian analysis is the application of probability theory as extended logic to quantify and update degrees of belief based on evidence.

His full set of disiderata or axioms, from which Jaynes derives Bayes theorem are,

1. Degrees of plausibility are represented by real numbers. This is a choice of practicality. Perhaps there is some value in probability represented with complex numbers, but so far no useful generalization has been found to my knowledge.
2. Qualitative consistency with common sense. Specifically, the system must be consistent with standard Boolean logic - the logic of syllogisms and deductive logic. This is not a generic call to "common sense" to base the system on, but rather the short-hand informal speech that Jaynes likes to use.
3. Internal consistency. If a conclusion can be reasoned out in more than one way then every possible way must yield the same result.
4. Propriety. We must take into account all of the information provided that is relevant to the question.
5. Equivalent states consistency. Equivalent states of knowledge must be represented by equivalent plausibility assignments.

In the definition above and the disiderata to be so clear as to be hard to argue against, and it doesn't use words like "doxastic" to get the point across.

Problems with Paulogia's definition

"Bayesian analysis is a quantification and mathematical description of presuppositions and natural decision-making processes"

The first thing that jumps out at me is the inclusion of decision-making processes, which isn't strictly in what is commonly referred to as Bayesian -- which restricts itself to inference. However, one can extend the analysis to include decision (unimaginatively called decision theory) by including a cost or utility function for the different actions, and maximizing something like expected utility. I explore this concept comparing faith and trust in an older blog post as well as in my book, Measure of Faith.

Also, while it is common to include ones presuppositions in any probabilistic analysis, this isn't a good definition of Bayesian analysis and allows opponents to discount other things one says. I'd prefer to separate the definition of the process with its common usage, calling out those that misuse the method for what they are doing. I've been speaking out against such misuse for a while, but it is useful to outline the main way Bayesian analysis is put forward in the apologetics space and where it goes awry.

In broad strokes the approach is correct

One of the reasons that the apologist-wielding Bayes can be compelling is that some of its features are useful/correct:

1. Bayes rule is the objective way to update hypotheses with new evidence.
2. The approach allows the apologist to split the problem up into parts, some of which are easier to handle than others, which aids both in communication but also analysis

So where does it typically go wrong?

There are a few common places where the approach goes wrong, and as Paulogia notes, that the apologist injects their presuppositions in a veil of seeming objectivity.

Ignoring or discounting the prior

The most obvious way that apologists misuse Bayes is by ignoring or discounting the prior. In a very long discussion on one of the McGrew's papers using Bayesian analysis to examine Resurrection, one can see how they ignore the prior for their own model while critiquing the prior of competing models. If you ignore the prior, then all sorts of nonsense from goblins to fairies are fair game explanations. Saying that the Resurrection explains the data best is not useful unless you also include the fact that Resurrections are supremely unlikely a-priori.

No attempt at reasonably estimating values

In the rare case where they actually supply values for probabilities, apologists generally make no serious attempt to estimate those values. While it can be difficult, a little thought can get a ball-park estimate.

For example, in this article by Than Christopoulos, as an example prior he proposes a prior of one in a million for the Resurrection. To him this must have been "small", but it is ludicrously too large. A very simple estimate, taking the entire known population ever to have lived on the planet (around 100 billion) with at most 1 resurrection puts it below one in 100 billion. Add to that our understanding of biology, of the psychology of extraordinary claims, etc.. will put this well below that.

The McGrews estimate a Bayes factor for the Resurrection of $10^{44}$ -- a probability against the resurrection less than $p=10^{-43}$ -- a staggering number. However they never even try to check if this is reasonable. What other claims (e.g. things are made of atoms, gravity follows an inverse square law) have this level of confidence? In a really funny interaction between apologists, the McGrews were discussing this with Richard Swinburne who was presenting a Bayes factor of around $10^3$. The consensus of these apologists in the discussion -- at least they are both large, so the Resurrection is likely. Instead, they should have wondering how experts in the field, using the same data and the same approach, could differ by a factor of more than $10^{40}$! This speaks to an abuse of the approach more clearly than almost anything I can say.

Misuse of "independent"

In both the McGrews article and the article by Than Christopoulos they quote an oft-used result that there is some level of independent testimony that can overcome any prior. While true mathematically, it only has a practical effect if you,

1. estimate the priors properly (see earlier section)
2. establish that the testimony is independent
3. actually have enough independent testimony

Apologists fail at each step here. Historians will talk about independent sources, and in the early New Testament books (Gospels, Acts, Letters of Paul) one can argue for about 6 independent sources (Mark, Q, M, L, John, and Paul) -- but this is hotly debated. That said, literary independence does not mean statistical independence -- where if you have two two independent events, knowing one of them tells you nothing about the other. This is so obviously does not hold with the Gospel accounts that to assume statistical independence of these claims here is ludicrous. The McGrews assume each and every disciple contributes statistically independent accounts!

Even in this case where one believes the accounts to be statistically independent, any small uncertainty in this assessment destroys its ability to offset a low prior.

Lack of imagination in alternative models

Apologists universally use the odds form or Bayes factor form of Bayesian analysis where they compare the "Resurrection hypothesis" to the "not Resurrection hypothesis". The problem with the approach is

1. it limits one to two models, when more are at play
2. is bundles many unrelated models into two big blocks

This is discussed in detail in an episode of Bad Apologetics specifically about Bayes.

Conclusion

The net effect of these mistakes is to give the appearance of objectivity to an approach that is anything but objective. One of the advantages of the approach, however, is that it provides a framework for structuring the arguments so it is straightforward to point out specifically where it goes wrong or is being misused.

What problems are you interested in? How can I help?

• @bblais
• bblais@bryant.edu
• (401) 232-6153
• Bryant University
  1150 Douglas Pike
  Smithfield, RI
  02917 US