Neural Optimizer Search with Reinforcement Learning
作者： Vijay Vasudevan , Quoc V. Le , Irwan Bello , Barret Zoph
DOI:
关键词:
摘要: We present an approach to automate the process of discovering optimization methods, with a focus on deep learning architectures. We train a Recurrent Neural Network controller to …
arxiv.org参考文章(34)
Matthew D. Zeiler, ADADELTA: An Adaptive Learning Rate Method arXiv: Learning. ,(2012)
Tomas Mikolov, Martin Karafiát, Sanjeev Khudanpur, Jan Cernocký, Lukás Burget, Recurrent neural network based language model conference of the international speech communication association. pp. 1045- 1048 ,(2010)
S. Bengio, Y. Bengio, J. Cloutier, Use of genetic programming for the search of a new learning rule for neural networks world congress on computational intelligence. pp. 324- 327 ,(1994) , 10.1109/ICEC.1994.349932
Mitch Marcus, Beatrice Santorini, Mary Ann Marcinkiewicz, None, Building a large annotated corpus of English: the penn treebank Computational Linguistics. ,vol. 19, pp. 313- 330 ,(1993) , 10.21236/ADA273556
Yoshua Bengio, Razvan Pascanu, Revisiting Natural Gradient for Deep Networks arXiv: Learning. ,(2013)
T.P. Runarsson, M.T. Jonsson, Evolution and design of distributed learning rules 2000 IEEE Symposium on Combinations of Evolutionary Computation and Neural Networks. Proceedings of the First IEEE Symposium on Combinations of Evolutionary Computation and Neural Networks (Cat. No.00. pp. 59- 63 ,(2000) , 10.1109/ECNN.2000.886220
Dong C. Liu, Jorge Nocedal, On the limited memory BFGS method for large scale optimization Mathematical Programming. ,vol. 45, pp. 503- 528 ,(1989) , 10.1007/BF01589116
Sepp Hochreiter, Jürgen Schmidhuber, Long short-term memory Neural Computation. ,vol. 9, pp. 1735- 1780 ,(1997) , 10.1162/NECO.1997.9.8.1735
Andrew Y. Ng, Jiquan Ngiam, Adam Coates, Quoc V. Le, Ahbik Lahiri, Bobby Prochnow, On optimization methods for deep learning international conference on machine learning. pp. 265- 272 ,(2011)
Ronald J. Williams, Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning Machine Learning. ,vol. 8, pp. 229- 256 ,(1992) , 10.1007/BF00992696