On the Number of Linear Regions of Deep Neural Networks
作者： Yoshua Bengio , Razvan Pascanu , Guido F Montufar , Kyunghyun Cho
DOI:
关键词: Artificial neural network 、 Structure (category theory) 、 Algorithm 、 Mathematical optimization 、 Rectifier (neural networks) 、 Layer (object-oriented design) 、 Deep learning 、 Mathematics 、 Computation 、 Feedforward neural network 、 Piecewise linear function 、 Artificial intelligence
摘要: … by deep feedforward neural networks with piecewise linear activations in terms of the symmetries and the number of linear regions that they have. Deep networks … deep neural networks …
arxiv.org
nips.cc
mpg.de
acm.org 参考文章(21)
Peter Bartlett, Martin M. Anthony, Learning in Neural Networks: Theoretical Foundations Cambridge University Press. ,(1999)
Martin Anthony, Peter L Bartlett, Peter L Bartlett, Neural Network Learning: Theoretical Foundations ,(1999)
Geoffrey E. Hinton, Vinod Nair, Rectified Linear Units Improve Restricted Boltzmann Machines international conference on machine learning. pp. 807- 814 ,(2010)
Yoshua Bengio, Razvan Pascanu, Guido Montufar, On the number of response regions of deep feed forward networks with piece-wise linear activations arXiv: Learning. ,(2013)
Yoshua Bengio, Razvan Pascanu, Guido Montufar, On the number of inference regions of deep feed forward networks with piece-wise linear activations international conference on learning representations. ,(2014)
Thomas Zaslavsky, Facing Up to Arrangements: Face-Count Formulas for Partitions of Space by Hyperplanes ,(1975)
Guido F. Montúfar, Universal approximation depth and errors of narrow belief networks with discrete units Neural Computation. ,vol. 26, pp. 1386- 1407 ,(2014) , 10.1162/NECO_A_00601
Guido Montufar, Nihat Ay, Refinements of universal approximation results for deep belief networks and restricted boltzmann machines Neural Computation. ,vol. 23, pp. 1306- 1319 ,(2011) , 10.1162/NECO_A_00113
G. Cybenko, Approximation by superpositions of a sigmoidal function Mathematics of Control, Signals, and Systems. ,vol. 2, pp. 303- 314 ,(1989) , 10.1007/BF02551274
Dan Cireşan, Ueli Meier, Jonathan Masci, Jürgen Schmidhuber, 2012 Special Issue: Multi-column deep neural network for traffic sign classification Neural Networks. ,vol. 32, pp. 333- 338 ,(2012) , 10.1016/J.NEUNET.2012.02.023