Measuring intrusion detection capability

摘要: A fundamental problem in intrusion detection is what metric(s) can be used to objectively evaluate an system (IDS) terms of its ability correctly classify events as normal or intrusive. Traditional metrics (e.g., true positive rate and false rate) measure different aspects, but no single metric seems sufficient the capability systems. The lack a unified makes it difficult fine-tune IDS. In this paper, we provide in-depth analysis existing metrics. Specifically, analyze typical cost-based scheme [6], demonstrate that approach very confusing ineffective when cost factor not carefully selected. addition, novel information-theoretic IDS propose new highly complements analysis. When examining process from point view, intuitively, should have less uncertainty about input (event data) given output (alarm data). Thus, our metric, CI D (Intrusion Detection Capability), defined ratio mutual information between entropy input. has desired property that: (1) It takes into account all important aspects naturally, i.e., rate, predictive value, negative base rate; (2) provides intrinsic capability; (3) sensitive operation parameters such which effect subtle changes We appropriate performance maximize fine-tuning obtained best achieved by data. use numerical examples well experiments actual IDSs on various data sets show using D, choose (optimal) operating for compare IDSs.