Reader variance in ROC studies-Generalizability to reader population at high and low performance levels

摘要: Rationale and Objectives To investigate the variability between discriminative performances of readers as a function average performance levels during receiver operating characteristic (ROC) studies. Materials Methods Four subsets cases from previously ascertained ROC rating data by 12 observers when detecting interstitial disease pneumothorax on posteroanterior chest films were selected for each abnormality reanalyzed to assess changes in “reader” variance component. The based prestudy subjective assessment subtleness depicted (positive cases) difficulty determining its absence (negative cases). Reader component was estimated using bootstrap approach subset results used general relationship level. Results reader decreased substantially (from 0.007704 0.000426), expected, areas under curves (AUC) pneumothoraces increased 84% 97%. On other hand, 0.000890 0.005181) AUC 59% 87%. large magnitude resulted consistent nonmonotone related components included additon Conclusion Among several factors affecting generalizability population readers, depended nonmonotonically diagnostic is lowest at both very high low performance.