Malware detection using bilayer behavior abstraction and improved one-class support vector machines

摘要: Malware detection is one of the most challenging problems in computer security. Recently, methods based on machine learning are very popular unknown and variant malware detection. In order to achieve a successful learning, extracting discriminant stable features important prerequisite. this paper, we propose bilayer behavior abstraction method semantic analysis dynamic API sequences. Operations sensitive system resources complex behaviors abstracted an interpretable way at different layers. At lower layer, raw calls combined abstract low-layer via data dependency analysis. higher further construct more high-layer with good interpretability. The extracted finally embedded into high-dimensional vector space. Hence, can be directly used by many algorithms. Besides, tackle problem that benign programs not adequately sampled or severely imbalanced, improved one-class support (OC-SVM) named OC-SVM-Neg proposed which makes use available negative samples. Experimental results show feature extraction outperforms binary classifiers false alarm rate generalization ability.