Finito: A faster, permutable incremental gradient method for big data problems

摘要: Recent advances in optimization theory have shown that smooth strongly convex finite sums can be minimized faster than by treating them as a black box "batch" problem. In this work we introduce new method class with theoretical convergence rate four times existing methods, for sufficiently many terms. This is also amendable to sampling without replacement scheme practice gives further speed-ups. We give empirical results showing state of the art performance.