Evaluating MapReduce for profiling application traffic

作者: MCC Cloud Computing , Thiago Vieira , Stenio Fernandes , Vinicius Cardoso Garcia

DOI: 10.1145/2465839.2465846

关键词: ScalabilityParallel computingBlock sizeData processingData typeProfiling (information science)Computer scienceNetwork packetTraffic analysisDeep packet inspection

摘要: The use of MapReduce for distributed data processing has been growing and achieving benefits with its application different workloads. can be used traffic analysis, although network traces present characteristics which are not similar to the type commonly processed through MapReduce. Motivated by profiling due lack evaluation analysis peculiarity this kind data, paper evaluates performance in packet level DPI, analysing scalability, speed-up, behavior phases. experiments provide evidences predominant phases job, show impact input size, block size number nodes, on completion time scalability.

参考文章(18)
Gregory R. Ganger, Elie Krevat, Raja R. Sambasivan, Michael Stroucken, William Wang, Spencer Whitman, Lianghong Xu, Michael De Rosa, Alice X. Zheng, Diagnosing performance changes by comparing request flows networked systems design and implementation. pp. 43- 56 ,(2011) , 10.5555/1972457.1972463
Karthik Nagaraj, Charles Killian, Jennifer Neville, Structured comparative analysis of systems logs to diagnose performance problems networked systems design and implementation. pp. 26- 26 ,(2012)
Jennifer Rexford, Dave Maltz, Srikanth Kandula, Changhoon Kim, Minlan Yu, Lihua Yuan, Albert Greenberg, Profiling network performance for multi-tier data center applications networked systems design and implementation. pp. 57- 70 ,(2011) , 10.5555/1972457.1972464
Yeonhee Lee, Wonchul Kang, Youngseok Lee, A hadoop-based packet trace processing tool traffic monitoring and analysis. pp. 51- 63 ,(2011) , 10.1007/978-3-642-20305-3_5
Thiago Vieira, Paulo Soares, Marco Machado, Rodrigo Assad, Vinicius Garcia, Measuring Distributed Applications through MapReduce and Traffic Analysis international conference on parallel and distributed systems. pp. 704- 705 ,(2012) , 10.1109/ICPADS.2012.104
Jian Tan, Xiaoqiao Meng, Li Zhang, Coupling scheduler for MapReduce/Hadoop Proceedings of the 21st international symposium on High-Performance Parallel and Distributed Computing - HPDC '12. pp. 129- 130 ,(2012) , 10.1145/2287076.2287097
Kyong-Ha Lee, Yoon-Joon Lee, Hyunsik Choi, Yon Dohn Chung, Bongki Moon, Parallel data processing with MapReduce ACM SIGMOD Record. ,vol. 40, pp. 11- 20 ,(2012) , 10.1145/2094114.2094118
A. Verma, L. Cherkasova, V. S. Kumar, R. H. Campbell, Deadline-based workload management for MapReduce environments: Pieces of the performance puzzle network operations and management symposium. pp. 900- 905 ,(2012) , 10.1109/NOMS.2012.6212006
Matei Zaharia, Dhruba Borthakur, Joydeep Sen Sarma, Khaled Elmeleegy, Scott Shenker, Ion Stoica, Delay scheduling: a simple technique for achieving locality and fairness in cluster scheduling european conference on computer systems. pp. 265- 278 ,(2010) , 10.1145/1755913.1755940
Peng Lu, Young Choon Lee, Chen Wang, Bing Bing Zhou, Junliang Chen, Albert Y. Zomaya, Workload Characteristic Oriented Scheduler for MapReduce international conference on parallel and distributed systems. pp. 156- 163 ,(2012) , 10.1109/ICPADS.2012.31