SNIWD: Simultaneous Weight Noise Injection with Weight Decay for MLP Training
作者： John Sum , Kevin Ho
DOI: 10.1007/978-3-642-10677-4_56
关键词: Fault tolerance 、 Weight decay 、 Artificial intelligence 、 Algorithm 、 Training (meteorology) 、 Radial basis function network 、 Noise 、 Artificial neural network 、 Machine learning 、 Multilayer perceptron 、 Computer science
摘要: Despite noise injecting during training has been demonstrated with success in enhancing the fault tolerance of neural network, theoretical analysis on dynamic this injection-based online learning algorithm far from complete. In particular, convergence proofs for those algorithms have not shown. regards, paper presents an empirical study non-convergence properties weight noises a multilayer perceptron, and called SNIWD (simultaneous injection decay) to overcome such problem. Simulation results show that is able improve enforce small magnitude network parameters (input weights, input biases output weights). Moreover, make similar ability as using pure approach.
springer.com
sci-hub.st 参考文章(18)
Nait Charif Hammadi, Hideo Ito, A learning algorithm for fault tolerant feedforward neural networks IEICE Transactions on Information and Systems. ,vol. 80, pp. 21- 27 ,(1997)
Jose L. Bernier, J. Ortega, I. Rojas, E. Ros, A. Prieto, Obtaining Fault Tolerant Multilayer Perceptrons Using an Explicit Regularization Neural Processing Letters. ,vol. 12, pp. 107- 113 ,(2000) , 10.1023/A:1009698206772
Ching-Tai Chiu, K. Mehrotra, C.K. Mohan, S. Ranka, Modifying training algorithms for improved fault tolerance world congress on computational intelligence. ,vol. 1, pp. 333- 338 ,(1994) , 10.1109/ICNN.1994.374185
Kevin Ho, Chi-sing Leung, John Sum, On Weight-Noise-Injection Training international conference on neural information processing. pp. 919- 926 ,(2009) , 10.1007/978-3-642-03040-6_112
Naotake Kamiura, Teijiro Isokawa, Kazuharu Yamato, Yutaka Hata, Nobuyuki Matsui, On a Weight Limit Approach for Enhancing Fault Tolerance of Feedforward Neural Networks IEICE Transactions on Information and Systems. ,vol. 83, pp. 1931- 1939 ,(2000)
Advances in Neuro-Information Processing Lecture Notes in Computer Science. ,vol. 5507, ,(2009) , 10.1007/978-3-642-03040-6
Kam-Chuen Jim, C.L. Giles, B.G. Horne, An analysis of noise in recurrent neural networks: convergence and generalization IEEE Transactions on Neural Networks. ,vol. 7, pp. 1424- 1438 ,(1996) , 10.1109/72.548170
Salvatore Cavalieri, Orazio Mirabella, A novel learning algorithm which improves the partial fault tolerance of multilayer neural networks Neural Networks. ,vol. 12, pp. 91- 106 ,(1999) , 10.1016/S0893-6080(98)00094-X
Gleb Basalyga, Emilio Salinas, When response variability increases neural network robustness to synaptic noise Neural Computation. ,vol. 18, pp. 1349- 1379 ,(2006) , 10.1162/NECO.2006.18.6.1349
C. Neti, M.H. Schneider, E.D. Young, Maximally fault tolerant neural networks IEEE Transactions on Neural Networks. ,vol. 3, pp. 14- 23 ,(1992) , 10.1109/72.105414