Free-response methodology: alternate analysis and a new observer-performance experiment.

Abstract

The authors review the recently introduced method for analyzing free-response receiver operating characteristic (FROC) observer-performance experiments. These experiments allow multiple abnormalities and observer responses per image and require correct localization for true-positive events. A fundamental plot, the alternative FROC (AFROC), is emphasized, the area under which is a natural index of performance for measuring FROC observer performance. The authors show that it is possible to analyze free-response data in more general ways. The first method, AFROC analysis, involves scoring FROC images in a different way. The second method involves a new observer-performance experiment, the free-response forced error (FFE) experiment. FROC and AFROC analysis was applied to several FROC data sets, and simultaneous FROC-AFROC-FFE experiments were performed with several radiologist observers to test the three methods. The methods were found to be in good agreement with one another.