Exploration of Analysis Methods for Diagnostic Imaging Tests: Problems with ROC AUC and Confidence Scores in CT Colonography

摘要: Background: Different methods of evaluating diagnostic performance when comparing tests may lead to different results. We compared two such approaches, sensitivity and specificity with area under the Receiver Operating Characteristic Curve (ROC AUC) for evaluation CT colonography detection polyps, either or without computer assisted detection. Methods: In a multireader multicase study 10 readers 107 cases we specificity, using radiological reporting presence absence ROC AUC calculated from confidence scores concerning polyps. Both were assessed against reference standard. Here focus on five readers, selected illustrate issues in design analysis. measures within showing that differences results are due statistical methods. Results: Reader varied widely depending whether was used. There problems scores; assigning all cases; use zero no polyps identified; bimodal non-normal distribution fitting curves extrapolation beyond data; undue influence few false positive Variation exceeded between test AUC. Conclusions: The recorded our violated many assumptions methods, rendering these inappropriate. identified will apply other studies scores. found more reliable clinically appropriate method compare tests.