3.3 ROC Analysis

Interestingly, according to (2.8), (2.11), and (3.5), no matter what the type of detector is, its detector statistics always turns out to be the likelihood ratio test (LRT). In other words, all the Bayes, minimax, or Neyman–Pearson detectors can be shown to have the same functional form determined by LRT. Nevertheless, NP detectors are the most practical detectors in real applications since they do not require prior knowledge of the cost function and probabilities of hypotheses that are generally unknown or difficult to obtain in practice. Instead, the performance is evaluated based on the four decisions described in Section 3.1 but with general descriptive terms given in the following:

1. When “H0” is true, “H0” is also true.
In this case, a correct decision is made and the decision is called “true negative” (TN). However, it should be noted that there is no detection term corresponding to this decision since nothing is detected other than noise.
2. When “H0” is true, “H1” is true.
In this case, an incorrect decision is made and the decision is called “false alarm” (FA) or “false positive” (FP) in medical diagnosis.
3. When “H1” is true, “H0” is also true.
In this case, an incorrect decision is made and the decision is called “miss” or “false negative” (FN).
4. When “H1” is true, “H1” is also true.
In this case, a correct decision is made and the decision is called “detection probability, rate or power” or “true positive” (TP).

As per the above decisions, we have two correct decisions 1 and 4, that is, TN and detection rate or power or true positive (TP), and two incorrect decisions 2 and 3, that is, FA (FP) and miss (FN). Consequently, a good detector must be the one that maximizes the probabilities yielded by the correct decisions, TN and detection rate/TP, and in the mean time it also minimizes the probabilities resulting from the two incorrect decisions, FA/FP and miss/FN. However, it is generally true that the two incorrect decisions contradict each other, so do the correct decisions. In other words, minimizing miss/FN is also to increase FA/FP and vice versa. Since minimizing miss/FN is equivalent to maximizing detection power/TP, a common practice is to maximize the detection rate/TP while imposing a constraint on FA/FP specified by (3.4). Using (3.6) and (3.7) we can further derive

(3.8) equation

(3.9) equation

To evaluate the detection performance of δNP(r), the ROC analysis is commonly used as an evaluation tool to assess the effectiveness of a detector based on an ROC curve plotted as a function of PD versus PF as shown in Figure 3.2. As an alternative to the use of ROC curves, the area under curve (AUC), Az, which has been widely used in medical diagnosis (Metz, 1978; Swets and Pickett, 1982), is also calculated by the area under an ROC curve.

Figure 3.2 An example of AUC, Az, calculated by an ROC curve.

img

The use of Az has advantages over the ROC curves. For example, on some occasions two detectors δ1 and δ2 may generate two different 2D ROC curves but have the same area, Az, as shown in Figure 3.3. In this case, both δ1 and δ2 yield the same detection performance even when they have different ROC curves.

Figure 3.3 An example of ROC curves generated by two detectors with the same Az.

img
..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
3.128.78.30