Optimization of the Architecture of Feed-forward Neural Networks with Hidden Layers by Unit Elimination
DOI:
关键词: Feed forward neural 、 Discrete mathematics 、 Physics 、 Algorithm 、 Unit (ring theory) 、 Hidden layer
摘要: A method for red ucing t he numb er of uni ts in hidden layers a feed-for war d neur al network is pr esent ed . Starting wit h net hat oversize, un dan unit s layer are elimi na by introducing an ad dit ional cost fun ct ion on set uxilia ry linear resp onse T ex ra fu nct enables uxiliary units o fuse ogether the redun dant origin , and aux iliary serve only as int erm edi at e const ruct vanishes whe n met hod converges. Nume rica l tests P arity Symmetry problem illu strat usefu lness his actice.
complex-systems.com参考文章(11)
Ben S. Wittner, John J. Hopfield, Sara A. Solla, Lawrence D. Jackel, John S. Denker, Daniel B. Schwartz, Richard E. Howard, Large Automatic Learning, Rule Extraction, and Generalization. Complex Systems. ,vol. 1, ,(1987)
Peter M. Todd, Shailesh U. Hegde, Geoffrey F. Miller, Designing neural networks using genetic algorithms international conference on genetic algorithms. pp. 379- 384 ,(1989)
Gerald Tesauro, Bob Janssens, Scaling relationships in back-propagation learning Complex Systems. ,vol. 2, pp. 39- 44 ,(1988)
Yves Chauvin, Generalization Performance of Overtrained Back-Propagation Networks Lecture Notes in Computer Science. ,vol. 412, pp. 46- 55 ,(1990) , 10.1007/3-540-52255-7_26
M Marchand, M Golea, P Ruján, A Convergence Theorem for Sequential Learning in Two-Layer Perceptrons EPL. ,vol. 11, pp. 487- 492 ,(1990) , 10.1209/0295-5075/11/6/001
Jean-Pierre Nadal, STUDY OF A GROWTH ALGORITHM FOR A FEEDFORWARD NETWORK International Journal of Neural Systems. ,vol. 01, pp. 55- 59 ,(1989) , 10.1142/S0129065789000463
M Mezard, Jean-P Nadal, Learning in feedforward layered networks: the tiling algorithm Journal of Physics A. ,vol. 22, pp. 2191- 2203 ,(1989) , 10.1088/0305-4470/22/12/019
Anselm Blumer, Andrzej Ehrenfeucht, David Haussler, Manfred K. Warmuth, Occam's razor Information Processing Letters. ,vol. 24, pp. 377- 380 ,(1987) , 10.1016/0020-0190(87)90114-1
E.D. Karnin, A simple procedure for pruning back-propagation trained neural networks IEEE Transactions on Neural Networks. ,vol. 1, pp. 239- 242 ,(1990) , 10.1109/72.80236
Eric B. Baum, David Haussler, What Size Net Gives Valid Generalization neural information processing systems. ,vol. 1, pp. 81- 90 ,(1988) , 10.1162/NECO.1989.1.1.151