Improvements to the Conventional Layer-by-Layer BP Algorithm
作者： Xu-Qin Li , Fei Han , Tat-Ming Lok , Michael R. Lyu , Guang-Bin Huang
DOI: 10.1007/11538356_20
关键词:
摘要: This paper points out some drawbacks and proposes modifications to the conventional layer-by-layer BP algorithm. In particular, we present a new perspective learning rate, which is use heuristic rule define rate so as update weights. Meanwhile, pull algorithm of saturation area prevent it from converging local minimum, momentum term introduced former And finally effectiveness efficiency proposed method are demonstrated by two benchmark examples.
springer.com
researchgate.net 
sci-hub.st 参考文章(17)
Neural Networks: Tricks of the Trade Springer. ,(2012) , 10.1007/978-3-642-35289-8
David E. Rumelhart, James L. McClelland, , Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations Computational Models of Cognition and Perception. ,(1986) , 10.7551/MITPRESS/5236.001.0001
Howard B. Demuth, Martin T. Hagan, Mark Beale, Neural network design ,(1995)
RGM Morris, Parallel & distributed processing ACM SIGGRAPH 2005 Courses on - SIGGRAPH '05. pp. 11- ,(2005) , 10.1145/1198555.1198750
De-Shuang Huang, The local minima-free condition of feedforward neural networks for outer-supervised learning systems man and cybernetics. ,vol. 28, pp. 477- 480 ,(1998) , 10.1109/3477.678658
Deshuang Huang, The “bottleneck” behaviours in linear feedforward neural network classifiers and their breakthrough Journal of Computer Science and Technology. ,vol. 14, pp. 34- 43 ,(1999) , 10.1007/BF02952485
A. Van Ooyen, B. Nienhuis, Improving the convergence of the back-propagation algorithm Neural Networks. ,vol. 5, pp. 465- 471 ,(1992) , 10.1016/0893-6080(92)90008-7
DE-SHUANG HUANG, RADIAL BASIS PROBABILISTIC NEURAL NETWORKS: MODEL AND APPLICATION International Journal of Pattern Recognition and Artificial Intelligence. ,vol. 13, pp. 1083- 1101 ,(1999) , 10.1142/S0218001499000604
Sang-Hoon Oh, Improving the error backpropagation algorithm with a modified error function IEEE Transactions on Neural Networks. ,vol. 8, pp. 799- 803 ,(1997) , 10.1109/72.572117
S. Ergezinger, E. Thomsen, An accelerated learning algorithm for multilayer perceptrons: optimization layer by layer IEEE Transactions on Neural Networks. ,vol. 6, pp. 31- 42 ,(1995) , 10.1109/72.363452