Three-class ROC analysis--toward a general decision theoretic solution.

Abstract

Multiclass receiver operating characteristic (ROC) analysis has remained an open theoretical problem since the introduction of binary ROC analysis in the 1950s. Previously, we have developed a paradigm for three-class ROC analysis that extends and unifies decision theoretic, linear discriminant analysis, and probabilistic foundations of binary ROC analysis in a three-class paradigm. One critical element in this paradigm is the equal error utility (EEU) assumption. This assumption allows us to reduce the intrinsic space of the three-class ROC analysis (5-D hypersurface in 6-D hyperspace) to a 2-D surface in the 3-D space of true positive fractions (sensitivity space). In this work, we show that this 2-D ROC surface fully and uniquely provides a complete descriptor for the optimal performance of a system for a three-class classification task, i.e., the triplet of likelihood ratio distributions, assuming such a triplet exists. To be specific, we consider two classifiers that utilize likelihood ratios, and we assumed each classifier has a continuous and differentiable 2-D sensitivity-space ROC surface. Under these conditions, we proved that the classifiers have the same triplet of likelihood ratio distributions if and only if they have the same 2-D sensitivity-space ROC surfaces. As a result, the 2-D sensitivity surface contains complete information on the optimal three-class task performance for the corresponding likelihood ratio classifier.