Distributed Fixed Point Methods with Compressed Iterates
作者： Martin Takác , Peter Richtárik , Adil Salim , Dmitry Kovalev , Ahmed Khaled
DOI:
关键词:
摘要: We propose basic and natural assumptions under which iterative optimization methods with compressed iterates can be analyzed. This problem is motivated by the practice of federated learning, where a large model stored in cloud before it sent to mobile device, then proceeds training based on local data. develop standard variance reduced methods, establish communication complexity bounds. Our algorithms are first distributed iterates, fixed point iterates.
arxiv.org参考文章(33)
John N Tsitsiklis, Zhi-Quan Luo, Communication complexity and convex optimization Journal of Complexity. ,vol. 3, pp. 231- 243 ,(1987) , 10.1016/0885-064X(87)90013-6
Patrick L. Combettes, Heinz H. Bauschke, Convex Analysis and Monotone Operator Theory in Hilbert Spaces ,(2011)
Gersende Fort, Eric Moulines, Yves F. Atchade, On stochastic proximal gradient algorithms arXiv: Statistics Theory. ,(2014)
Laurent Condat, A primal-dual splitting method for convex optimization involving Lipschitzian, proximable and linear composite terms Journal of Optimization Theory and Applications. ,vol. 158, pp. 460- 479 ,(2013) , 10.1007/S10957-012-0245-9
Antonin Chambolle, Thomas Pock, A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging Journal of Mathematical Imaging and Vision. ,vol. 40, pp. 120- 145 ,(2011) , 10.1007/S10851-010-0251-1
Angelia Nedic, Alex Olshevsky, Asuman Ozdaglar, John N. Tsitsiklis, Distributed subgradient methods and quantization effects conference on decision and control. pp. 4177- 4184 ,(2008) , 10.1109/CDC.2008.4738860
Bằng Công Vũ, A splitting algorithm for dual monotone inclusions involving cocoercive operators Advances in Computational Mathematics. ,vol. 38, pp. 667- 681 ,(2013) , 10.1007/S10444-011-9254-8
Olivier Devolder, François Glineur, Yurii Nesterov, First-order methods of smooth convex optimization with inexact oracle Mathematical Programming. ,vol. 146, pp. 37- 75 ,(2014) , 10.1007/S10107-013-0677-5
M.G. Rabbat, R.D. Nowak, Quantized incremental algorithms for distributed optimization IEEE Journal on Selected Areas in Communications. ,vol. 23, pp. 798- 808 ,(2005) , 10.1109/JSAC.2005.843546
Borja Peleato, Stephen Boyd, Neal Parikh, Jonathan Eckstein, Eric Chu, Distributed Optimization and Statistical Learning Via the Alternating Direction Method of Multipliers ,(2011)