Variable Selection with Optimal Cell Damage

作者: Tautvydas Cibas , Françroise Fogelman Soulié , Patrick Gallinari , Sarunas Raudys

DOI: 10.1007/978-1-4471-2097-1_171

关键词: Classifier (UML)RegressionFeature selectionArtificial neural networkAlgorithmHeuristicsComputer scienceRegularization (mathematics)

摘要: Neural Networks -NN- have been used in a large variety of real-world applications. In those, one could measure potentially number N variables Xi; probably not all Xi are equally informative: if select n« “best” Xi, then reduce the amount data to gather and process; hence costs. Variable selection is thus an important issue Pattern Recognition Regression. It also complex problem; needs criterion value subset that will course depend on predictor or classifier further used. Conventional variable techniques based upon statistical heuristics tools [Fukunaga, 90]: major difficulty comes from intrinsic combinatorics problem. this paper we show how use NNs for with evaluation usefulness. Various methods proposed assess weight (e.g. saliency [Le Cun et al. 90] Optimal Brain-Damage -OBD- procedure): along similar ideas, derive method, called Cell Damage -OCD-, which evaluates usefulness input Multi-Layer Network prunes least useful. achieved during training classifier, ensuring selected set matches complexity. viewed here as extension pruning. One can regularization approach selection, discuss elsewhere [Cibas al., 94]. We illustrate our method two relatively small problems: prediction synthetic time series classification waveforms [Breiman 84], representative hard problems.

参考文章(7)
Richard A Olshen, Charles J Stone, Leo Breiman, Jerome H Friedman, Classification and regression trees ,(1983)
Yann LeCun, John Denker, Sara Solla, None, Optimal Brain Damage neural information processing systems. ,vol. 2, pp. 598- 605 ,(1989)
Chris Bishop, Exact calculation of the Hessian matrix for the multilayer perceptron Neural Computation. ,vol. 4, pp. 494- 501 ,(1992) , 10.1162/NECO.1992.4.4.494
M. de Bollivier, P. Gallinari, S. Thiria, Cooperation of neural nets and task decomposition IJCNN-91-Seattle International Joint Conference on Neural Networks. pp. 573- 576 ,(1991) , 10.1109/IJCNN.1991.155397
David E. Rumelhart, Andreas S. Weigend, Bernardo A. Huberman, Generalization by Weight-Elimination with Application to Forecasting neural information processing systems. ,vol. 3, pp. 875- 882 ,(1990)
Yves Chauvin, Dynamic behavior of constrained back propagation networks neural information processing systems. pp. 642- 649 ,(1990)
Keinosuke Fukunaga, Statistical pattern recognition Handbook of pattern recognition & computer vision. pp. 33- 60 ,(1993)