Extracting hidden anomalies using sketch and non Gaussian multiresolution statistical detection procedures

作者: Guillaume Dewaele , Kensuke Fukuda , Pierre Borgnat , Patrice Abry , Kenjiro Cho

DOI: 10.1145/1352664.1352675

关键词: Computer scienceAnomaly (natural sciences)Reduction (complexity)Curse of dimensionalityPattern recognitionArtificial intelligenceMarginal distributionIdentification (information)GaussianAnomaly detectionRandom projection

摘要: A new profile-based anomaly detection and characterization procedure is proposed. It aims at performing prompt accurate of both short-lived long-lasting low-intensity anomalies, without the recourse any prior knowledge targetted traffic. Key features algorithm lie in joint use random projection techniques (sketches) a multiresolution non Gaussian marginal distribution modeling. The former enables reduction dimensionality data measurement reference (i.e., normal) traffic behavior, while latter extracts anomalies different aggregation levels. This used to blindly analyze large-scale packet trace database collected on trans-Pacific transit link from 2001 2006. can detect identify large number known unknown attacks, whose intensities are low (down below one percent). Using sketches also makes possible real-time identification source or destination IP addresses associated detected hence their mitigation.

参考文章(25)
Kenjiro Cho, Koushirou Mitsuya, Akira Kato, Traffic data repository at the WIDE project usenix annual technical conference. pp. 51- 51 ,(2000)
Chen-Mou Cheng, H.T. Kung, Koan-Sin Tan, Use of spectral analysis in defense against DoS attacks global communications conference. ,vol. 3, pp. 2143- 2148 ,(2002) , 10.1109/GLOCOM.2002.1189011
Geoffrey M. Voelker, Stefan Savage, David Moore, Inferring internet denial-of-service activity usenix security symposium. pp. 2- 2 ,(2001)
Michèle Basseville, Distance measures for signal processing and pattern recognition Signal Processing. ,vol. 18, pp. 349- 369 ,(1989) , 10.1016/0165-1684(89)90079-0
Kavé Salamatian, Nina Taft, Augustin Soule, Combining filtering and statistical methods for anomaly detection internet measurement conference. pp. 31- 31 ,(2005) , 10.5555/1251086.1251117
Jaeyeon Jung, Balachander Krishnamurthy, Michael Rabinovich, Flash crowds and denial of service attacks: characterization and implications for CDNs and web sites the web conference. pp. 293- 304 ,(2002) , 10.1145/511446.511485
Mikkel Thorup, Yin Zhang, Tabulation based 4-universal hashing with applications to second moment estimation symposium on discrete algorithms. pp. 615- 624 ,(2004) , 10.5555/982792.982884
Patrice Abry, Pierre Borgnat, Guillaume Dewaele, Invited Talk: Sketch Based Anomaly Detection, Identification and Performance Evaluation symposium on applications and the internet. pp. 80- 80 ,(2007) , 10.1109/SAINT-W.2007.55