Improving the Quickprop algorithm
作者： Chi-Chung Cheung , Sin-Chun Ng , Andrew K Lui
DOI: 10.1109/IJCNN.2012.6252546
关键词:
摘要: Backpropagation (BP) algorithm is the most popular supervised learning that extensively applied in training feed-forward neural networks. Many BP modifications have been proposed to increase convergence rate of standard algorithm, and Quickprop one fast algorithms. The very fast; however, it easily trapped into a local minimum thus cannot converge global minimum. This paper proposes new modified from Quickprop. By addressing drawbacks has systematic approach improve capability Our performance investigation shows always converges with faster compared improvement especially large. In problem (application), increased 4% 100%.
sci-hub.se 参考文章(20)
Scott Kirkpatrick, Optimization by Simulated Annealing: Quantitative Studies Journal of Statistical Physics. ,vol. 34, pp. 975- 986 ,(1984) , 10.1007/BF01009452
X-H Yu, None, Acceleration of backpropagation learning using optimised learning rate and momentum Electronics Letters. ,vol. 29, pp. 1288- 1290 ,(1993) , 10.1049/EL:19930860
M. Gori, A. Tesi, On the problem of local minima in backpropagation IEEE Transactions on Pattern Analysis and Machine Intelligence. ,vol. 14, pp. 76- 86 ,(1992) , 10.1109/34.107014
S. Kirkpatrick, C. D. Gelatt, M. P. Vecchi, Optimization by Simulated Annealing Science. ,vol. 220, pp. 671- 680 ,(1983) , 10.1126/SCIENCE.220.4598.671
Javier E. Vitela, Jaques Reifman, Premature saturation in backpropagation networks: mechanism and necessary conditions Neural Networks. ,vol. 10, pp. 721- 735 ,(1997) , 10.1016/S0893-6080(96)00117-7
Edward K. Blum, Leong Kwan Li, Approximation theory and feedforward networks Neural Networks. ,vol. 4, pp. 511- 515 ,(1991) , 10.1016/0893-6080(91)90047-9
Youngjik Lee, Sang-Hoon Oh, Myung Won Kim, Original Contribution: An analysis of premature saturation in back propagation learning Neural Networks. ,vol. 6, pp. 719- 728 ,(1993) , 10.1016/S0893-6080(05)80116-9
Fritz Stäger, Mukul Agarwal, Three methods to speed up the training of feedforward and feedback perceptrons Neural Networks. ,vol. 10, pp. 1435- 1443 ,(1997) , 10.1016/S0893-6080(97)00053-1
Christian Igel, Michael Hüsken, Empirical evaluation of the improved Rprop learning algorithms Neurocomputing. ,vol. 50, pp. 105- 123 ,(2003) , 10.1016/S0925-2312(01)00700-7
A. Van Ooyen, B. Nienhuis, Improving the convergence of the back-propagation algorithm Neural Networks. ,vol. 5, pp. 465- 471 ,(1992) , 10.1016/0893-6080(92)90008-7