Automatic malware classification and new malware detection using machine learning

The explosive growth of malware variants poses a major threat to information security. Traditional anti-virus systems based on signatures fail to classify unknown malware into their corresponding families and to detect new kinds of malware programs. Therefore, we propose a machine learning based malware analysis system, which is composed of three modules: data processing, decision making, and new malware detection. The data processing module deals with gray-scale images, Opcode n-gram, and import functions, which are employed to extract the features of the malware. The decision-making module uses the features to classify the malware and to identify suspicious malware. Finally, the detection module uses the shared nearest neighbor (SNN) clustering algorithm to discover new malware families. Our approach is evaluated on more than 20 000 malware instances, which were collected by Kingsoft, ESET NOD32, and Anubis. The results show that our system can effectively classify the unknown malware with a best accuracy of 98.9%, and successfully detects 86.7% of the new malware.     

A graph-based model for malware detection and classification using system-call groups

In this paper we present a graph-based model that, utilizing relations between groups of System-calls, detects whether an unknown software sample is malicious or benign, and classifies a malicious software to one of a set of known malware families. More precisely, we utilize the System-call Dependency Graphs (or, for short, ScD-graphs), obtained by traces captured through dynamic taint analysis. We design our model to be resistant against strong mutations applying our detection and classification techniques on a weighted directed graph, namely Group Relation Graph, or Gr-graph for short, resulting from ScD-graph after grouping disjoint subsets of its vertices. For the detection process, we propose the \(\Delta \)-similarity metric, and for the process of classification, we propose the SaMe-similarity and NP-similarity metrics consisting the SaMe-NP similarity. Finally, we evaluate our model for malware detection and classification showing its potentials against malicious software measuring its detection rates and classification accuracy.     

Network visualization for financial crime detection

Objective: We present a new software system, VisFAN, for the visual analysis of financial activity networks.MethodsWe combine enhanced graph drawing techniques to devise novel algorithms and interaction functionalities for the visual exploration of networked data sets, together with tools for SNA and for the automatic generation of reports.ResultsAn application example constructed on real data is presented. We also report the results of a study aimed at qualitatively understanding the satisfaction level of the analysts when using VisFAN.ConclusionVisFAN makes a strong use of visual interactive tools, combined with ad-hoc clustering techniques and customizable layout constraints management.ImplicationsAs this system confirms, information visualization can play a crucial role to face the discovery of financial crimes.     

Pushdown model checking for malware detection

The number of malware is growing extraordinarily fast. Therefore, it is important to have efficient malware detectors. Malware writers try to obfuscate their code by different techniques. Many well-known obfuscation techniques rely on operations on the stack such as inserting dead code by adding useless push and pop instructions, or hiding calls to the operating system, etc. Thus, it is important for malware detectors to be able to deal with the program’s stack. In this study, we propose a new model-checking approach for malware detection that takes into account the behavior of the stack. Our approach consists in: (1) Modeling the program using a pushdown system (PDS). (2) Introducing a new logic, called stack computation tree predicate logic (SCTPL), to represent the malicious behavior. SCTPL can be seen as an extension of the branching-time temporal logic CTL with variables, quantifiers, and predicates over the stack. (3) Reducing the malware detection problem to the model-checking problem of PDSs against SCTPL formulas. We show how our new logic can be used to precisely express malicious behaviors that could not be specified by existing specification formalisms. We then consider the model-checking problem of PDSs against SCTPL specifications. We reduce this problem to emptiness checking in Symbolic Alternating Büchi Pushdown Systems, and we provide an algorithm to solve this problem. We implemented our techniques in a tool and applied it to detect several viruses. Our results are encouraging.     

Behavior-based features model for malware detection

The sharing of malicious code libraries and techniques over the Internet has vastly increased the release of new malware variants in an unprecedented rate. Malware variants share similar behaviors yet they have different syntactic structure due to the incorporation of many obfuscation and code change techniques such as polymorphism and metamorphism. The different structure of malware variants poses a serious problem to signature-based detection technique, yet their similar exhibited behaviors and actions can be a remarkable feature to detect them by behavior-based techniques. Malware instances also largely depend on API calls provided by the operating system to achieve their malicious tasks. Therefore, behavior-based detection techniques that utilize API calls are promising for the detection of malware variants. In this paper, we propose a behavior-based features model that describes malicious action exhibited by malware instance. To extract the proposed model, we first perform dynamic analysis on a relatively recent malware dataset inside a controlled virtual environment and capture traces of API calls invoked by malware instances. The traces are then generalized into high-level features we refer to as actions. We assessed the viability of actions by various classification algorithms such as decision tree, random forests, and support vector machine. The experimental results demonstrate that the classifiers attain high accuracy and satisfactory results in the detection of malware variants.     

Graph-based malware detection using dynamic analysis

We introduce a novel malware detection algorithm based on the analysis of graphs constructed from dynamically collected instruction traces of the target executable. These graphs represent Markov chains, where the vertices are the instructions and the transition probabilities are estimated by the data contained in the trace. We use a combination of graph kernels to create a similarity matrix between the instruction trace graphs. The resulting graph kernel measures similarity between graphs on both local and global levels. Finally, the similarity matrix is sent to a support vector machine to perform classification. Our method is particularly appealing because we do not base our classifications on the raw n-gram data, but rather use our data representation to perform classification in graph space. We demonstrate the performance of our algorithm on two classification problems: benign software versus malware, and the Netbull virus with different packers versus other classes of viruses. Our results show a statistically significant improvement over signature-based and other machine learning-based detection methods.     

Visual malware detection using local malicious pattern

Journal of Computer Virology and Hacking Techniques - In recent years, malware authors have had significant developments in offering new generations of malware and have tried to use different...     

基于深度置信网络的Android恶意软件检测

针对采用重打包和代码混淆技术的Android恶意软件检测准确率低的问题,提出了一种基于深度置信网络的Android恶意软件检测算法。通过自动化提取Android应用软件的特征,构建对应的特征向量,训练基于深度置信网络的深度学习模型,实现了一种新的基于深度置信网络的Android恶意软件检测算法。实验结果表明,基于深度置信网络的深度学习模型可以更好地表征Android恶意软件,其检测效果也明显优于传统的机器学习模型。     

Joint network-host based malware detection using information-theoretic tools

In this paper, we propose two joint network-host based anomaly detection techniques that detect self-propagating malware in real-time by observing deviations from a behavioral model derived from a benign data profile. The proposed malware detection techniques employ perturbations in the distribution of keystrokes that are used to initiate network sessions. We show that the keystrokes’ entropy increases and the session-keystroke mutual information decreases when an endpoint is compromised by a self-propagating malware. These two types of perturbations are used for real-time malware detection. The proposed malware detection techniques are further compared with three prominent anomaly detectors, namely the maximum entropy detector, the rate limiting detector and the credit-based threshold random walk detector. We show that the proposed detectors provide considerably higher accuracy with almost 100% detection rates and very low false alarm rates.     

HybriDroid: an empirical analysis on effective malware detection model developed using ensemble methods

The Journal of Supercomputing - Malware detection from the smartphone has become a challenging issue for academicians and researchers. In this research paper, we applied five distinct machine...     

Multimodal information fusion for android malware detection using lazy learning

Multimedia Tools and Applications - Android has a large number of users that are accumulating with each passing day. Security of the Android ecosystem is a major concern for these users with the...     

浅析当前网络犯罪特点及取证对策

本文介绍了当前网络犯罪打击的难点以及提出了一系列的思考与对策,给公安机关执法人员在对待各种新的网络犯罪时,提供了新的启示和策略。     

Support vector machines and malware detection

In this research, we test three advanced malware scoring techniques that have shown promise in previous research, namely, Hidden Markov Models, Simple Substitution Distance, and Opcode Graph based detection. We then perform a careful robustness analysis by employing morphing strategies that cause each score to fail. We show that combining scores using a Support Vector Machine yields results that are significantly more robust than those obtained using any of the individual scores.     

Nonnegative matrix factorization and metamorphic malware detection

Metamorphic malware change their internal code structure by adopting code obfuscation technique while maintaining their malicious functionality during each infection. This causes change of their signature pattern across each infection and makes signature based detection particularly difficult. In this paper, through static analysis, we use similarity score from matrix factorization technique called Nonnegative Matrix Factorization for detecting challenging metamorphic malware. We apply this technique using structural compression ratio and entropy features and compare our results with previous eigenvector-based techniques. Experimental results from three malware datasets show this is a promising technique as the accuracy detection is more than 95%.

     

“云安全”检测技术安全性分析

“云安全”检测已成为病毒查杀领域发展的新趋势,为对其在病毒检测过程中的安全性有进一步了解,研究了“云安全”检测体系结构以及主流“云安全”策略,针对某“云安全”检测软件的文件样本提取方式和网络传输数据的特点,分析了检测流程中存在的安全隐患,基于这些安全隐患设计并实现了“云安全”检测的规避方案,针对规避方案提出了防护建议.实验结果表明,“云安全”检测在实际应用过程中仍可能被恶意程序绕过.     

Subroutine based detection of APT malware

Statistical detection of mass malware has been shown to be highly successful. However, this type of malware is less interesting to cyber security officers of larger organizations, who are more concerned with detecting malware indicative of a targeted attack. Here we investigate the potential of statistically based approaches to detect such malware using a malware family associated with a large number of targeted network intrusions. Our approach is complementary to the bulk of statistical based malware classifiers, which are typically based on measures of overall similarity between executable files. One problem with this approach is that a malicious executable that shares some, but limited, functionality with known malware is likely to be misclassified as benign. Here a new approach to malware classification is introduced that classifies programs based on their similarity with known malware subroutines. It is illustrated that malware and benign programs can share a substantial amount of code, implying that classification should be based on malicious subroutines that occur infrequently, or not at all in benign programs. Various approaches to accomplishing this task are investigated, and a particularly simple approach appears the most effective. This approach simply computes the fraction of subroutines of a program that are similar to malware subroutines whose likes have not been found in a larger benign set. If this fraction exceeds around 1.5 %, the corresponding program can be classified as malicious at a 1 in 1000 false alarm rate. It is further shown that combining a local and overall similarity based approach can lead to considerably better prediction due to the relatively low correlation of their predictions.     

Clustering versus SVM for malware detection

Previous work has shown that cluster analysis can be used to effectively classify malware into meaningful families. In this research, we apply cluster analysis to the challenging problem of classifying previously unknown malware. We perform several experiments involving malware clustering. We compare our clustering results to those obtained when a support vector machine (SVM) is trained on the malware family. Using clustering, we are able to classify malware with an accuracy comparable to that of an SVM. An advantage of the clustering approach is that a new malware family can be classified before a model has been trained specifically for the family.     

Markhor: malware detection using fuzzy similarity of system call dependency sequences

Journal of Computer Virology and Hacking Techniques - Static malware detection approaches are time-consuming and cannot deal with code obfuscation techniques. Dynamic malware detection approaches,...     

Software transformations to improve malware detection

Malware is code designed for a malicious purpose, such as obtaining root privilege on a host. A malware detector identifies malware and thus prevents it from adversely affecting a host. In order to evade detection, malware writers use various obfuscation techniques to transform their malware. There is strong evidence that commercial malware detectors are susceptible to these evasion tactics. In this paper, we describe the design and implementation of a malware transformer that reverses the obfuscations performed by a malware writer. Our experimental evaluation demonstrates that this malware transformer can drastically improve the detection rates of commercial malware detectors.