A cloud/edge computing streaming system for network traffic monitoring and threat detection

摘要: The unyielding trend of increasing cyber threats has made security paramount in protecting personal and private intellectual property. To provide a highly secured network environment, threat detection systems must handle real-time big data from varied places enterprise networks. In this paper, we introduce streaming-based system that can rapidly analyse intensive traffic real-time, utilising clustering algorithms to detect abnormal activities. developed integrates the high-performance analysis capabilities Flume, Spark Hadoop into cloud-computing environment monitoring intrusion detection. Our performance evaluation validates cope with significant volume streaming high accuracy good performance. We further extend our for edge computing discuss some key challenges, as well potential solutions, aiming improve scalability system.