Issues and Advances in Anomaly Detection Evaluation for Joint Human-Automated Systems

摘要: As human-managed systems become more complex, automated anomaly detection can provide assistance—but only if it is effective. Rigorous evaluation of vital for determining its effectiveness before implementation into systems. We identified recurring issues in practices limiting the conclusions that be applied from published studies to broader application. In this paper, we demonstrate implications these and illustrate solutions. show how receiver operating characteristic curves reveal performance tradeoffs masked by reporting single metric results using multiple simulation data examples prevent biases result training testing examples. also methods incorporating latency tradeoff analyses. Application will help researchers, engineers, decision makers with a objective basis evaluation, resulting greater utility, better performance, cost savings engineering.