A Stochastic Quasi-Newton Method for Large-Scale Optimization
作者： R. H. Byrd , S. L. Hansen , Jorge Nocedal , Y. Singer
DOI: 10.1137/140954362
关键词: Quasi-Newton method 、 Mathematics 、 Robustness (computer science) 、 Stochastic optimization 、 Algorithm 、 Sub-sampling 、 Curvature 、 Broyden–Fletcher–Goldfarb–Shanno algorithm 、 Stochastic approximation 、 Pointwise
摘要: … The goal of this paper is to propose a quasi-Newton method that operates in the stochastic approximation regime. We employ the well-known limited memory BFGS updating formula, …
arxiv.org
harvard.edu
arxiv-vanity.com
optimization-online.org 参考文章(22)
Søren Asmussen, Peter W. Glynn, Stochastic Simulation: Algorithms and Analysis ,(2008)
Nicol N. Schraudolph, Simon Günter, Jin Yu, A stochastic quasi-Newton method for online convex optimization international conference on artificial intelligence and statistics. pp. 436- 443 ,(2007)
Nicolas L. Roux, Andrew W. Fitzgibbon, A fast natural Newton method international conference on machine learning. pp. 623- 630 ,(2010)
Shun-ichi Amari, Natural gradient works efficiently in learning Neural Computation. ,vol. 10, pp. 177- 202 ,(1998) , 10.1162/089976698300017746
A. Nemirovski, A. Juditsky, G. Lan, A. Shapiro, Robust Stochastic Approximation Approach to Stochastic Programming SIAM Journal on Optimization. ,vol. 19, pp. 1574- 1609 ,(2009) , 10.1137/070704277
Herbert Robbins, Sutton Monro, A Stochastic Approximation Method Annals of Mathematical Statistics. ,vol. 22, pp. 400- 407 ,(1951) , 10.1214/AOMS/1177729586
H Park, S.-I Amari, K Fukumizu, Adaptive natural gradient learning algorithms for various stochastic models Neural Networks. ,vol. 13, pp. 755- 764 ,(2000) , 10.1016/S0893-6080(00)00051-4
Paul Tseng, Sangwoon Yun, A coordinate gradient descent method for nonsmooth separable minimization Mathematical Programming. ,vol. 117, pp. 387- 423 ,(2008) , 10.1007/S10107-007-0170-0
Dong C. Liu, Jorge Nocedal, On the limited memory BFGS method for large scale optimization Mathematical Programming. ,vol. 45, pp. 503- 528 ,(1989) , 10.1007/BF01589116
Richard H. Byrd, Gillian M. Chin, Jorge Nocedal, Yuchen Wu, Sample size selection in optimization methods for machine learning Mathematical Programming. ,vol. 134, pp. 127- 155 ,(2012) , 10.1007/S10107-012-0572-5