A fast and scalable method for threat detection in large-scale DNS logs

摘要: This paper presents a fast and scalable method for detecting threats in large-scale DNS logs. In such logs, queries about “abnormal” domain strings are often correlated with malicious behavior. With our method, language model algorithm learns “normal” domain-names from large dataset to rate the extent of domain-name “abnormality” within big data stream organization. Variable-order Markov Models (VMMs) serve as out underlying algorithmic tool since their running time is linear input sequence while memory requirements constantly bounded above, both very appealing characteristics. Our experimental study indicates that proposed can detect names generated by genuine Domain Generation Algorithm, used Advanced Persistent Threat attack scenarios, less than 5% false-negative 1% false-positive rates. detection similar more computationally intensive methods not environments.