Mobile Internet big data platform in China Unicom
作者： Wenliang Huang , Zhen Chen , Wenyu Dong , Hang Li , Bin Cao
关键词: Big data 、 Database 、 Mobile Web 、 China 、 Engineering 、 Table (database) 、 Compound annual growth rate 、 Data processing 、 Data warehouse 、 Volume (computing)
摘要: China Unicom, the largest WCDMA 3G operator in China, meets requirements of historical Mobile Internet Explosion, or surging Traffic from mobile terminals. According to internal statistics user traffic has increased rapidly with a Compound Annual Growth Rate (CAGR) 135%. Currently Unicom monthly stores more than 2 trillion records, data volume is over 525 TB, and highest reached peak 5 PB. Since October 2009, been developing home-brewed big storage analysis platform based on open source Hadoop Distributed File System (HDFS) as it long-term strategy make full use this Big Data. All well served using platform. Currently, writing speed 1 390 000 records per second, record retrieval time table that contains trillions less 100 ms. To take advantage opportunity be Data Operator, developed new functions multiple innovations solve space constraint challenges presented processing. In paper, we will introduce our detail. Based platform, building an industry ecosystem Data, considers telecom centric can formed critical reach prosperity modern communications business.
tsinghuajournals.com参考文章(11)
J. Manyika, Michael Chui, Brad Brown, Jacques Bughin, Richard Dobbs, Charles Roxburgh, Angela Hung Byers, Big data: The next frontier for innovation, competition, and productivity ,(2011)
Luca Deri, Francesco Fusco, MicroCloud-based network traffic monitoring integrated network management. pp. 864- 867 ,(2013)
Vern Paxson, Anja Feldmann, Holger Dreger, Robin Sommer, Stefan Kornexl, Building a time machine for efficient recording and retrieval of high-volume network traffic internet measurement conference. pp. 23- 23 ,(2005) , 10.5555/1251086.1251109
Youngseok Lee, Wonchul Kang, Hyeongu Son, An Internet traffic analysis method with MapReduce network operations and management symposium. pp. 357- 361 ,(2010) , 10.1109/NOMSW.2010.5486551
Feng Qian, Zhaoguang Wang, Yudong Gao, Junxian Huang, Alexandre Gerber, Zhuoqing Mao, Subhabrata Sen, Oliver Spatscheck, None, Periodic transfers in mobile applications: network-wide origin, impact, and optimization the web conference. pp. 51- 60 ,(2012) , 10.1145/2187836.2187844
Yeonhee Lee, Youngseok Lee, Toward scalable internet traffic measurement and analysis with Hadoop acm special interest group on data communication. ,vol. 43, pp. 5- 13 ,(2012) , 10.1145/2427036.2427038
Robin Sommer, Holger Dreger, Anja Feldmann, Vern Paxson, Fabian Schneider, Gregor Maier, Enriching network security analysis with time travel ACM SIGCOMM Computer Communication Review. ,vol. 38, pp. 183- 194 ,(2008) , 10.1145/1402946.1402980
Jun Li, Shuai Ding, Ming Xu, Fuye Han, Xin Guan, Zhen Chen, TIFA: Enabling Real-Time Querying and Storage of Massive Stream Data international conference on networking. pp. 61- 64 ,(2011) , 10.1109/ICNDC.2011.20
K Wu, S Ahern, E W Bethel, J Chen, H Childs, E Cormier-Michel, C Geddes, J Gu, H Hagen, B Hamann, W Koegler, J Lauret, J Meredith, P Messmer, E Otoo, V Perevoztchikov, A Poskanzer, Prabhat, O Rübel, A Shoshani, A Sim, K Stockinger, G Weber, W-M Zhang, FastBit: interactively searching massive data Lawrence Berkeley National Laboratory. ,vol. 180, pp. 012053- ,(2009) , 10.1088/1742-6596/180/1/012053
Zhen Chen, Lingyun Ruan, Junwei Cao, Yifan Yu, Xin Jiang, TIFAflow: enhancing traffic archiving system with flow granularity for forensic analysis in network security Tsinghua Science & Technology. ,vol. 18, pp. 406- 417 ,(2013) , 10.1109/TST.2013.6574679