Suppose we are interested in two events, A and B. In this case, event A might represent the event that a patient has appendicitis and event B might represent a patient having a high white blood cell count. The conditional probability of event A given event B is essentially the probability that event A will occur when we know that event B has already happened.

Formally, we define the conditional probability of event A given event B as the joint probability of both events occurring divided by the probability of event B occurring:

Note that this is consistent with the way in which we define statistical independence. Statistical independence occurs when the joint probability of two events occurring is just the product of the individual probabilities of the two events. If we substitute this in our previous equation, we have:

This makes sense intuitively because if we know that two events are independent of each other, knowing that event B has occurred does not change the probability of event A occurring. Now, we can rearrange our equation for conditional probability as follows, and note that we can switch over events A and B to get an alternative form:

This last step allows us to state Bayes' Theorem in its simplest form:

In the previous equation, P(A) is referred to as the prior probability of event A, as it represents the probability of event A occurring prior to any new information. P(A|B), which is the conditional probability of event A given that event B has occurred, is often also referred to as the posterior probability of A. It is the probability of event A occurring after receiving some new information; in this case, the fact that event B has occurred.

All of this might seem like algebraic trickery, but if we revisit our example of event A representing a patient having appendicitis and event B representing a patient having a high white blood cell count, the usefulness of Bayes' Theorem will be revealed. Knowing P(A|B), the conditional probability of having appendicitis, given that we observe that a patient has a high white blood cell count (and similarly for other symptoms), is knowledge that would be very useful to doctors. This would allow them to make a diagnosis about something that isn't easily observable (appendicitis) using something that is (high white blood cell count).

Unfortunately, this is something that is very hard to estimate because a high white blood cell count might occur as a symptom of a host of other diseases or pathologies. The reverse probability, P(B|A), however (namely, the conditional probability of having a high white blood cell count given that a patient already has appendicitis), is much easier to estimate. One simply needs to examine records of past cases with appendicitis and inspect the blood tests of those cases. Bayes' Theorem is a fundamental boon to predictive modeling because it allows us to estimate cause by observing effect.