A word about ROC curves

roc1.gif

Sometime in the 1950’s, diet
the term Receiver Operating Characteristic curve was coined. The original purpose for this mathematical device was to determine how effective a receiver was in identifying symbols in a transmission stream, however, in recent decades the popularity of this device has grown beyond the radio communication field. In any kind of situation where there is data analysis to be performed and a binary result to be computed, the ROC curve can be used to determine both the efficiency of the detection algorithm as well as parameter optimization. Looking at the graph above, we can replace sensitivity with true positive detection rate and 1 – specificity as the false positive detection rate. It is clear that the green line represents a random guess detector, where the probability of picking a true positive and a false positive is the same. The blue line is then the curve of some detection algorithm, since the area under the green curve is greater than ½, we can say that the algorithm generally performs better than a random guess algorithm. To generate such a curve with variable detector parameters, we vary the parameters and re-run the detection on some gold-standard data set to get each point on the curve, where each point on the curve is the statistical information generated from a certain set of detector parameters. With this information, we can not only determine the general performance of the algorithm, but also find the point on the curve that gives us the highest sensitivity and the highest specificity, that is, the point that is geometrically closest to the top left corner of the curve. With this point in hand, we can extract the parameters used to generate this point and use them as the optimal detector settings.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>