Notice that the posterior distribution looks a little different depending on what prior you use. The most common criticism lodged against Bayesian methods is that the choice of prior adds an unsavory subjective element to analysis. To a certain extent, they're right about the added subjective element, but their allegation that it is unsavory is way off the mark.

To see why, check out Figure 7.6, which shows both posterior distributions (from priors #1 and #2) in the same plot. Notice how priors #1 and #2—two very different priors—given the evidence, produce posteriors that look more similar to each other than the priors did.

Figure 7.6: The posterior distributions from prior #1 and #2

Now direct your attention to Figure 7.7, which shows the posterior of both priors if the evidence included 80 out of 120 correct trials.

Figure 7.7: The posterior distributions from prior #1 and #2 with more evidence

Note that the evidence still contains 67% correct trials, but there is now more evidence. The posterior distributions are now far more similar. Notice that now both of the posteriors' credible intervals do not contain theta = 0.5; with 80 out of 120 trials correctly predicted, even the most obstinate skeptic has to concede that something is going on (though they will probably disagree that the power comes from the socks!).

Take notice also of the fact that the credible intervals, in both posteriors, are now substantially narrowing, illustrating more confidence in our estimate.

Finally, imagine the case where I correctly predicted 67% of the trials, but out of 450 total trials. The posteriors derived from this evidence are shown in Figure 7.8:

Figure 7.8: The posterior distributions from prior #1 and #2 with even more evidence

The posterior distributions are looking very similar—indeed, they are becoming identical. Given enough trials—given enough evidence—these posterior distributions will be exactly the same. When there is enough evidence available such that the posterior is dominated by it compared to the prior, it is called overwhelming the prior.

As long as the prior is reasonable (that is, it doesn't assign a probability of 0 to theoretically plausible parameter values), given enough evidence, everybody's posterior belief will look very similar.

There is nothing unsavory or misleading about an analysis that uses a subjective prior; the analyst just has to disclose what her prior is. You can't just pick a prior willy-nilly; it has to be justifiable to your audience. In most situations, a prior may be informed by prior evidence like scientific studies and can be something that most people can agree on. A more skeptical audience may disagree with the chosen prior, in which case the analysis can be re-run using their prior, just like we did in the magic socks example. It is sometimes okay for people to have different prior beliefs, and it is okay for some people to require a little more evidence in order to be convinced of something.

The belief that frequentist hypothesis testing is more objective, and therefore more correct, is mistaken insofar as it causes all parties to have a hold on the same potentially bad assumptions. The assumptions in Bayesian analysis, on the other hand, are stated clearly from the start, made public, and are auditable.

To recap, there are three situations you can come across. In all of these, it makes sense to use Bayesian methods, if that's your thing: