Tracking the gradients using the Hessian: A new look at variance reducing stochastic methods

摘要: Our goal is to improve variance reducing stochastic methods through better control variates. We first propose a modification of SVRG which uses the Hessian track gradients over time, rather than recondition, increasing correlation variates and leading faster theoretical convergence close optimum. then accurate computationally efficient approximations Hessian, both using diagonal low-rank matrix. Finally, we demonstrate effectiveness our method on wide range problems.