Deep Reinforcement Learning for Mobile Video Offloading in Heterogeneous Cellular Networks

作者: Nan Zhao , Chao Tian , Menglin Fan , Minghu Wu , Xiao He

DOI: 10.4018/IJMCMC.2018100103

关键词:

摘要: Heterogeneous cellular networks can balance mobile video loads and reduce cell arrangement costs, which is an important technology of future mobile video communication networks. Because of the characteristics of non-convexity of the mobile offloading problem, the design of the optimal strategy is an essential issue. For the sake of ensuring users' quality of service and the long-term overall network utility, this article proposes the distributive optimal method by means of multiple agent reinforcement learning in the downlink heterogeneous cellular networks. In addition, to solve the computational load issue generated by the large action space, deep reinforcement learning is introduced to gain the optimal policy. The learning policy can provide a near-optimal solution efficiently with a fast convergence speed. Simulation results show that the proposed approach is more efficient at improving the performance than the Q-learning method.

参考文章(49)
Caoimhín Graham, David Bell, Zhihui Luo, Multi-Agent Reinforcement Learning – An Exploration Using Q-Learning rdis. pp. 293- 298 ,(2010) , 10.1007/978-1-84882-983-1_21
Shao-Yu Lien, Shao-Chou Hung, Kwang-Cheng Chen, Ying-Chang Liang, Ultra-low-latency ubiquitous connections in heterogeneous cloud radio access networks IEEE Wireless Communications. ,vol. 22, pp. 22- 31 ,(2015) , 10.1109/MWC.2015.7143323
Paul Fergus, Dhiya Al-Jumeily, Abir Jaafar Hussain, Using adaptive neural networks to provide self-healing autonomic software International Journal of Space-Based and Situated Computing. ,vol. 5, pp. 129- 140 ,(2015)
Zhaohui Hu, Hao Chen, Network load balancing strategy based on supervised reinforcement learning with shaping rewards international conference on intelligent control and information processing. pp. 393- 397 ,(2013) , 10.1109/ICICIP.2013.6568104
Ahmed R. Elsherif, Wei-Peng Chen, Akira Ito, Zhi Ding, Resource Allocation and Inter-Cell Interference Management for Dual-Access Small Cells IEEE Journal on Selected Areas in Communications. ,vol. 33, pp. 1082- 1096 ,(2015) , 10.1109/JSAC.2015.2416990
Qiaoyang Ye, Beiyu Rong, Yudong Chen, Mazin Al-Shalash, Constantine Caramanis, Jeffrey G. Andrews, User Association for Load Balancing in Heterogeneous Cellular Networks IEEE Transactions on Wireless Communications. ,vol. 12, pp. 2706- 2716 ,(2013) , 10.1109/TWC.2013.040413.120676
Gavin Taylor, Ronald Parr, Kernelized value function approximation for reinforcement learning Proceedings of the 26th Annual International Conference on Machine Learning - ICML '09. pp. 1017- 1024 ,(2009) , 10.1145/1553374.1553504
Soummya Kar, Jose M. F. Moura, H. Vincent Poor, Distributed reinforcement learning in multi-agent networks ieee international workshop on computational advances in multi sensor adaptive processing. pp. 296- 299 ,(2013) , 10.1109/CAMSAP.2013.6714066
Siavash Bayat, Raymond H. Y. Louie, Zhu Han, Branka Vucetic, Yonghui Li, Distributed User Association and Femtocell Allocation in Heterogeneous Wireless Networks IEEE Transactions on Communications. ,vol. 62, pp. 3027- 3043 ,(2014) , 10.1109/TCOMM.2014.2339313
Kaiming Shen, Wei Yu, Distributed Pricing-Based User Association for Downlink Heterogeneous Cellular Networks IEEE Journal on Selected Areas in Communications. ,vol. 32, pp. 1100- 1113 ,(2014) , 10.1109/JSAC.2014.2328143