An experimental bias-variance analysis of SVM ensembles based on resampling techniques

作者: G. Valentini

DOI: 10.1109/TSMCB.2005.850183

关键词:

摘要: Recently, bias-variance decomposition of error has been used as a tool to study the behavior learning algorithms and develop new ensemble methods well suited characteristics base learners. We propose procedures, based on Domingo's unified theory, evaluate quantitatively measure in ensembles machines. apply these compare single support vector machines (SVMs) SVMs resampling techniques, their relationships with cardinality training samples. In particular, we present an experimental analysis bagged random aggregated order verify theoretical variance reduction properties. The characterizes between bagging aggregating, explains reasons why built small subsamples data work large databases. Our also suggests directions for research improve classical bagging.

参考文章(42)
Theodoros Evgeniou, Tomaso Poggio, Massimiliano Pontil, Luis Perez-Breva, Bounds on the Generalization Performance of Kernel Machine Ensembles international conference on machine learning. pp. 271- 278 ,(2000)
Pedro Domingos, A Unified Bias-Variance Decomposition for Zero-One and Squared Loss national conference on artificial intelligence. pp. 564- 569 ,(2000)
Robert E. Banfield, Lawrence O. Hall, Kevin W. Bowyer, W. Philip Kegelmeyer, A New Ensemble Diversity Measure Applied to Thinning Ensembles Multiple Classifier Systems. pp. 306- 316 ,(2003) , 10.1007/3-540-44938-8_31
Eun Bae Kong, Thomas G. Dietterich, Error-Correcting Output Coding Corrects Bias and Variance Machine Learning Proceedings 1995. pp. 313- 321 ,(1995) , 10.1016/B978-1-55860-377-6.50046-3
Ron Kohavi, David Wolpert, Bias plus variance decomposition for zero-one loss functions international conference on machine learning. pp. 275- 283 ,(1996)
Giorgio Valentini, Thomas G. Dietterich, Bias-Variance Analysis and Ensembles of SVM multiple classifier systems. pp. 222- 231 ,(2002) , 10.1007/3-540-45428-4_22
Pedro Domingos, A Unifeid Bias-Variance Decomposition and its Applications international conference on machine learning. pp. 231- 238 ,(2000)
G. Valentini, M. Muselli, F. Ruffino, Bagged ensembles of Support Vector Machines for gene expression data analysis international joint conference on neural network. ,vol. 3, pp. 1844- 1849 ,(2003) , 10.1109/IJCNN.2003.1223688
Thomas G. Dietterich, Ensemble Methods in Machine Learning Multiple Classifier Systems. pp. 1- 15 ,(2000) , 10.1007/3-540-45014-9_1
Thomas G. Dietterich, Giorgio Valentini, Bias-Variance Analysis of Support Vector Machines for the Development of SVM-Based Ensemble Methods Journal of Machine Learning Research. ,vol. 5, pp. 725- 775 ,(2004) , 10.5555/1005332.1016783