摘要: In this paper we analyze the most popular evaluation metrics for separate-and-conquer rule learning algorithms. Our results show that all commonly used heuristics, including accuracy, weighted relative entropy, Gini index and information gain, are equivalent to one of two fundamental prototypes: precision, which tries optimize area under ROC curve unknown costs, a cost-weighted difference between covered positive negative examples, find optimal point known or assumed costs. We also straightforward generalization m-estimate trades off these prototypes.
tu-darmstadt.de参考文章(14)
Johannes Fürnkranz, Peter A. Flach, An Analysis of Rule Learning Heuristics ,(2003)
Bojan Cestnik, Estimating probabilities: a crucial task in machine learning european conference on artificial intelligence. pp. 147- 149 ,(1990)
José Hernández-Orallo, Peter A. Flach, César Ferri, Learning Decision Trees Using the Area Under the ROC Curve international conference on machine learning. pp. 139- 146 ,(2002)
Ricardo Vilalta, Daniel Oblinger, A Quantification of Distance Bias Between Evaluation Metrics In Classification international conference on machine learning. pp. 1087- 1094 ,(2000)
William W. Cohen, Fast Effective Rule Induction Machine Learning Proceedings 1995. pp. 115- 123 ,(1995) , 10.1016/B978-1-55860-377-6.50023-2
Johannes Fürnkranz, Separate-and-Conquer Rule Learning Artificial Intelligence Review. ,vol. 13, pp. 3- 54 ,(1999) , 10.1023/A:1006524209794
D. Gamberger, N. Lavrac, Expert-guided subgroup discovery: methodology and application Journal of Artificial Intelligence Research. ,vol. 17, pp. 501- 527 ,(2002) , 10.1613/JAIR.1089
J.R. Quinlan, Learning Logical Definitions from Relations Machine Learning. ,vol. 5, pp. 239- 266 ,(1990) , 10.1023/A:1022699322624
Andrew P. Bradley, ROC curves and the X2 test Pattern Recognition Letters. ,vol. 17, pp. 287- 294 ,(1996) , 10.1016/0167-8655(95)00121-2
Foster Provost, Tom Fawcett, Robust Classification for Imprecise Environments Machine Learning. ,vol. 42, pp. 203- 231 ,(2001) , 10.1023/A:1007601015854