Analog Gradient Aggregation for Federated Learning Over Wireless Networks: Customized Design and Convergence Analysis

作者: Huayan Guo , An Liu , Vincent K. N. Lau

DOI: 10.1109/JIOT.2020.3002925

关键词:

摘要: This article investigates the analog gradient aggregation (AGA) solution to overcome communication bottleneck for wireless federated learning applications by exploiting idea of over-the-air transmission. Despite various advantages, this special transmission also brings new challenges both transceiver design and algorithm due nonstationary local gradients time-varying channels in different rounds. To address these issues, we propose a novel AGA solution. In particular, parameters are optimized with consideration nonstationarity based on simple feedback variable. Moreover, rate is proposed stochastic descent algorithm, which adaptive quality estimation. Theoretical analyses provided convergence Finally, effectiveness confirmed two separate experiments linear regression shallow neural network. The simulation results verify that outperforms state-of-the-art baseline schemes much faster speed.

参考文章(28)
N.D. Sidiropoulos, T.N. Davidson, Zhi-Quan Luo, Transmit beamforming for physical-layer multicasting IEEE Transactions on Signal Processing. ,vol. 54, pp. 2239- 2251 ,(2006) , 10.1109/TSP.2006.872578
Le-Nam Tran, Muhammad Fainan Hanif, Markku Juntti, A Conic Quadratic Programming Approach to Physical Layer Multicasting for Large-Scale Antenna Arrays IEEE Signal Processing Letters. ,vol. 21, pp. 114- 117 ,(2014) , 10.1109/LSP.2013.2293840
Jakub Konečný, Daniel Ramage, H. Brendan McMahan, Federated Optimization: Distributed Optimization Beyond the Datacenter arXiv: Learning. ,(2015)
Michael Gastpar, Uncoded transmission is exactly optimal for a simple Gaussian "sensor" network information theory and applications. ,vol. 54, pp. 5247- 5251 ,(2007) , 10.1109/TIT.2008.929967
Jakub Konečný, Daniel Ramage, Peter Richtarik, H. Brendan McMahan, Federated Optimization: Distributed Machine Learning for On-Device Intelligence arXiv: Learning. ,(2016)
Jakub Konečný, Ananda Theertha Suresh, Dave Bacon, Felix X. Yu, Peter Richtarik, H. Brendan McMahan, Federated Learning: Strategies for Improving Communication Efficiency arXiv: Learning. ,(2016)
Ananda Theertha Suresh, Felix X. Yu, Sanjiv Kumar, H. Brendan McMahan, Distributed mean estimation with limited communication international conference on machine learning. pp. 3329- 3337 ,(2017)
Songfu Cai, Vincent K. N. Lau, Modulation-Free M2M Communications for Mission-Critical Applications ieee transactions on signal and information processing over networks. ,vol. 4, pp. 248- 263 ,(2018) , 10.1109/TSIPN.2017.2686002
Li Chen, Xiaowei Qin, Guo Wei, A Uniform-Forcing Transceiver Design for Over-the-Air Function Computation IEEE Wireless Communications Letters. ,vol. 7, pp. 942- 945 ,(2018) , 10.1109/LWC.2018.2840157
Li Chen, Nan Zhao, Yunfei Chen, F. Richard Yu, Guo Wei, Over-the-Air Computation for IoT Networks: Computing Multiple Functions With Antenna Arrays IEEE Internet of Things Journal. ,vol. 5, pp. 5296- 5306 ,(2018) , 10.1109/JIOT.2018.2843321