Regularization for Deep Learning: A Taxonomy
作者： Vladimir Golkov , Daniel Cremers , Jan Kukačka
DOI:
关键词:
摘要: Regularization is one of the crucial ingredients of deep learning, yet the term regularization has various definitions, and regularization methods are often studied separately from each other. In our work we present a systematic, unifying taxonomy to categorize existing methods. We distinguish methods that affect data, network architectures, error terms, regularization terms, and optimization procedures. We do not provide all details about the listed methods; instead, we present an overview of how the methods can be sorted into …
harvard.edu
arxiv.org参考文章(36)
Kunihiko Fukushima, Sei Miyake, Neocognitron: A Self-Organizing Neural Network Model for a Mechanism of Visual Pattern Recognition Springer, Berlin, Heidelberg. pp. 267- 285 ,(1982) , 10.1007/978-3-642-46466-9_18
Shin-ichi Maeda, A Bayesian encourages dropout arXiv: Learning. ,(2014)
Pierre Sermanet, Yann LeCun, David Eigen, Rob Fergus, Michael Mathieu, Xiang Zhang, OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks arXiv: Computer Vision and Pattern Recognition. ,(2013)
Xavier Bouthillier, Roland Memisevic, Kishore Konda, Pascal Vincent, Dropout as data augmentation arXiv: Machine Learning. ,(2015)
Christopher M. Bishop, Neural networks for pattern recognition ,(1995)
Kotagiri Ramamohanarao, Christopher Leckie, James Bailey, Sergey Demyanov, Invariant backpropagation: how to train a transformation-invariant neural network arXiv: Machine Learning. ,(2015)
Geoffrey Hinton, Oriol Vinyals, Jeff Dean, Distilling the Knowledge in a Neural Network arXiv: Machine Learning. ,(2015)
Ilya Sutskever, Geoffrey E. Hinton, Alex Krizhevsky, Ruslan R. Salakhutdinov, Nitish Srivastava, Improving neural networks by preventing co-adaptation of feature detectors arXiv: Neural and Evolutionary Computing. ,(2012)
Matthew D. Zeiler, Rob Fergus, Stochastic Pooling for Regularization of Deep Convolutional Neural Networks arXiv: Learning. ,(2013)
Tianqi Chen, Naiyan Wang, Mu Li, Bing Xu, Empirical Evaluation of Rectified Activations in Convolutional Network. arXiv: Learning. ,(2015)