Toward scalable internet traffic measurement and analysis with Hadoop
作者： Yeonhee Lee , Youngseok Lee
关键词:
摘要: Internet traffic measurement and analysis has long been used to characterize network usage user behaviors, but faces the problem of scalability under explosive growth high-speed access. Scalable is difficult because a large data set requires matching computing storage resources. Hadoop, an open-source platform MapReduce distributed file system, become popular infrastructure for massive analytics it facilitates scalable processing services on system consisting commodity hardware. In this paper, we present Hadoop-based monitoring that performs IP, TCP, HTTP, NetFlow multi-terabytes in manner. From experiments with 200-node testbed, achieved 14 Gbps throughput 5 TB files IP HTTP-layer jobs. We also explain performance issues related
acm.org
sigcomm.org 参考文章(13)
Dave Plonka, FlowScan: A Network Traffic Flow Reporting and Visualization Tool usenix large installation systems administration conference. pp. 305- 318 ,(2000)
Steve Romig, The OSU Flow-tools Package and CISCO NetFlow Logs usenix large installation systems administration conference. pp. 291- 304 ,(2000)
A. Finamore, M. Mellia, M. Meo, M. M. Munafò, D. Rossi, Live traffic monitoring with tstat: capabilities and experiences wired wireless internet communications. ,vol. 6074, pp. 290- 301 ,(2010) , 10.1007/978-3-642-13315-2_24
Tom White, Hadoop: The Definitive Guide ,(2009)
Martin Roesch, Snort - Lightweight Intrusion Detection for Networks usenix large installation systems administration conference. pp. 229- 238 ,(1999)
Matthias Vallentin, Robin Sommer, Jason Lee, Craig Leres, Vern Paxson, Brian Tierney, The NIDS cluster: scalable, stateful network intrusion detection on commodity hardware recent advances in intrusion detection. pp. 107- 126 ,(2007) , 10.1007/978-3-540-74320-0_6
Yeonhee Lee, Wonchul Kang, Youngseok Lee, A hadoop-based packet trace processing tool traffic monitoring and analysis. pp. 51- 63 ,(2011) , 10.1007/978-3-642-20305-3_5
Francesco Fusco, Luca Deri, High speed network traffic analysis with commodity multi-core systems internet measurement conference. pp. 218- 224 ,(2010) , 10.1145/1879141.1879169
Yeonhee Lee, Youngseok Lee, Detecting DDoS attacks with Hadoop Proceedings of The ACM CoNEXT Student Workshop on - CoNEXT '11 Student. pp. 7- ,(2011) , 10.1145/2079327.2079334
Kenjiro Cho, Kensuke Fukuda, Hiroshi Esaki, Akira Kato, Observing slow crustal movement in residential user traffic conference on emerging network experiment and technology. pp. 12- ,(2008) , 10.1145/1544012.1544024