Modeling the Influence of Data Structure on Learning in Neural Networks: The Hidden Manifold Model
作者： Sebastian Goldt , Marc Mézard , Florent Krzakala , Lenka Zdeborová
DOI: 10.1103/PHYSREVX.10.041044
关键词: Artificial intelligence 、 Data structure 、 Manifold (fluid mechanics) 、 Deep neural networks 、 Artificial neural network 、 Computer science
摘要: Understanding the reasons for the success of deep neural networks trained using stochastic gradient-based methods is a key open problem for the nascent theory of deep learning. The …
harvard.edu
sci-hub.st 参考文章(38)
A. Engel, C. Van den Broeck, Statistical Mechanics of Learning The handbook of brain theory and neural networks. pp. 925- 929 ,(2001) , 10.1017/CBO9781139164542
A. C. C. Coolen, D. Saad, Dynamics of learning with restricted training sets Physical Review E. ,vol. 62, pp. 5444- 5487 ,(2000) , 10.1103/PHYSREVE.62.5444
Michael Biehl, Peter Riegler, Christian Wöhler, Transient dynamics of on-line learning in two-layered neural networks Journal of Physics A. ,vol. 29, pp. 4769- 4780 ,(1996) , 10.1088/0305-4470/29/16/005
A. C. C Coolen, D Saad, Yuan-Sheng Xiong, On-line learning from restricted training sets in multilayer neural networks EPL. ,vol. 51, pp. 691- 697 ,(2000) , 10.1209/EPL/I2000-00394-5
Baktash Babadi, Haim Sompolinsky, Sparseness and Expansion in Sensory Representations Neuron. ,vol. 83, pp. 1213- 1226 ,(2014) , 10.1016/J.NEURON.2014.07.035
David Saad, Sara A. Solla, On-line learning in soft committee machines Physical Review E. ,vol. 52, pp. 4225- 4243 ,(1995) , 10.1103/PHYSREVE.52.4225
D. E. Muller, Application of Boolean algebra to switching circuit design and to error detection Transactions of the I.R.E. Professional Group on Electronic Computers. ,vol. 3, pp. 6- 12 ,(1954) , 10.1109/IREPGELC.1954.6499441
W Kinzel, Pal Rujan, Improving a Network Generalization Ability by Selecting Examples EPL. ,vol. 13, pp. 473- 477 ,(1990) , 10.1209/0295-5075/13/5/016
Peter Grassberger, Itamar Procaccia, Measuring the strangeness of strange attractors Physica D: Nonlinear Phenomena. ,vol. 9, pp. 189- 208 ,(1983) , 10.1016/0167-2789(83)90298-1
M Biehl, H Schwarze, Learning by on-line gradient descent Journal of Physics A. ,vol. 28, pp. 643- 656 ,(1995) , 10.1088/0305-4470/28/3/018