MACSPMD:基于恶意API调用序列模式挖掘的恶意代码检测

基于动态分析的恶意代码检测方法由于能有效对抗恶意代码的多态和代码混淆技术,而且可以检测新的未知恶意代码等,因此得到了研究者的青睐。在这种情况下,恶意代码的编写者通过在恶意代码中嵌入大量反检测功能来逃避现有恶意代码动态检测方法的检测。针对该问题,提出了基于恶意API调用序列模式挖掘的恶意代码检测方法MACSPMD。首先,使用真机模拟恶意代码的实际运行环境来获取文件的动态API调用序列;其次,引入面向目标关联挖掘的概念,以挖掘出能够代表潜在恶意行为模式的恶意API调用序列模式;最后,将挖掘到的恶意API调用序列模式作为异常行为特征进行恶意代码的检测。基于真实数据集的实验结果表明,MACSPMD对未知和逃避型恶意代码进行检测的准确率分别达到了94.55%和97.73%,比其他基于API调用数据的恶意代码检测方法 的准确率分别提高了2.47%和2.66%,且挖掘过程消耗的时间更少。因此,MACSPMD能有效检测包括逃避型在内的已知和未知恶意代码。     

SBMDS: an interpretable string based malware detection system using SVM ensemble with bagging

Malicious executables are programs designed to infiltrate or damage a computer system without the owner’s consent, which have become a serious threat to the security of computer systems. There is an urgent need for effective techniques to detect polymorphic, metamorphic and previously unseen malicious executables of which detection fails in most of the commercial anti-virus software. In this paper, we develop interpretable string based malware detection system (SBMDS), which is based on interpretable string analysis and uses support vector machine (SVM) ensemble with Bagging to classify the file samples and predict the exact types of the malware. Interpretable strings contain both application programming interface (API) execution calls and important semantic strings reflecting an attacker’s intent and goal. Our SBMDS is carried out with four major steps: (1) first constructing the interpretable strings by developing a feature parser; (2) performing feature selection to select informative strings related to different types of malware; (3) followed by using SVM ensemble with bagging to construct the classifier; (4) and finally conducting the malware detector, which not only can detect whether a program is malicious or not, but also can predict the exact type of the malware. Our case study on the large collection of file samples collected by Kingsoft Anti-virus lab illustrate that: (1) The accuracy and efficiency of our SBMDS outperform several popular anti-virus software; (2) Based on the signatures of interpretable strings, our SBMDS outperforms data mining based detection systems which employ single SVM, Naive Bayes with bagging, Decision Trees with bagging; (3) Compared with the IMDS which utilizes the objective-oriented association (OOA) based classification on API calls, our SBMDS achieves better performance. Our SBMDS system has already been incorporated into the scanning tool of a commercial anti-virus software.     

Determining malicious executable distinguishing attributes and low-complexity detection

Detection of rapidly evolving malware requires classification techniques that can effectively and efficiently detect zero-day attacks. Such detection is based on a robust model of benign behavior and deviations from that model are used to detect malicious behavior. In this paper we propose a low-complexity host-based technique that uses deviations in static file attributes to detect malicious executables. We first develop simple statistical models of static file attributes derived from the empirical data of thousands of benign executables. Deviations among the attribute models of benign and malware executables are then quantified using information-theoretic (Kullback-Leibler-based) divergence measures. This quantification reveals distinguishing attributes that are considerably divergent between benign and malware executables and therefore can be used for detection. We use the benign models of divergent attributes in cross-correlation and log-likelihood frameworks to classify malicious executables. Our results, using over 4,000 malicious file samples, indicate that the proposed detector provides reasonably high detection accuracy, while having significantly lower complexity than existing detectors.     

An intelligent PE-malware detection system based on association mining

The proliferation of malware has presented a serious threat to the security of computer systems. Traditional signature-based anti-virus systems fail to detect polymorphic/metamorphic and new, previously unseen malicious executables. Data mining methods such as Naive Bayes and Decision Tree have been studied on small collections of executables. In this paper, resting on the analysis of Windows APIs called by PE files, we develop the Intelligent Malware Detection System (IMDS) using Objective-Oriented Association (OOA) mining based classification. IMDS is an integrated system consisting of three major modules: PE parser, OOA rule generator, and rule based classifier. An OOA_Fast_FP-Growth algorithm is adapted to efficiently generate OOA rules for classification. A comprehensive experimental study on a large collection of PE files obtained from the anti-virus laboratory of KingSoft Corporation is performed to compare various malware detection approaches. Promising experimental results demonstrate that the accuracy and efficiency of our IMDS system outperform popular anti-virus software such as Norton AntiVirus and McAfee VirusScan, as well as previous data mining based detection systems which employed Naive Bayes, Support Vector Machine (SVM) and Decision Tree techniques. Our system has already been incorporated into the scanning tool of KingSoft’s Anti-Virus software. A short version of the paper is appeared in [33]. The work is partially supported by NSF IIS-0546280 and an IBM Faculty Research Award. The authors would also like to thank the members in the anti-virus laboratory at KingSoft Corporation for their helpful discussions and suggestions.     

DeepAM: a heterogeneous deep learning framework for intelligent malware detection

With computers and the Internet being essential in everyday life, malware poses serious and evolving threats to their security, making the detection of malware of utmost concern. Accordingly, there have been many researches on intelligent malware detection by applying data mining and machine learning techniques. Though great results have been achieved with these methods, most of them are built on shallow learning architectures. Due to its superior ability in feature learning through multilayer deep architecture, deep learning is starting to be leveraged in industrial and academic research for different applications. In this paper, based on the Windows application programming interface calls extracted from the portable executable files, we study how a deep learning architecture can be designed for intelligent malware detection. We propose a heterogeneous deep learning framework composed of an AutoEncoder stacked up with multilayer restricted Boltzmann machines and a layer of associative memory to detect newly unknown malware. The proposed deep learning model performs as a greedy layer-wise training operation for unsupervised feature learning, followed by supervised parameter fine-tuning. Different from the existing works which only made use of the files with class labels (either malicious or benign) during the training phase, we utilize both labeled and unlabeled file samples to pre-train multiple layers in the heterogeneous deep learning framework from bottom to up for feature learning. A comprehensive experimental study on a real and large file collection from Comodo Cloud Security Center is performed to compare various malware detection approaches. Promising experimental results demonstrate that our proposed deep learning framework can further improve the overall performance in malware detection compared with traditional shallow learning methods, deep learning methods with homogeneous framework, and other existing anti-malware scanners. The proposed heterogeneous deep learning framework can also be readily applied to other malware detection tasks.     

一种基于主动学习的恶意代码检测方法

现有恶意代码的检测往往依赖于对足够数量样本的分析.然而新型恶意代码大量涌现,其出现之初,样本数量有限,现有方法无法迅速检测出新型恶意代码及其变种.本文在数据流依赖网络中分析进程访问行为异常度与相似度,引入了恶意代码检测估计风险,并提出一种通过最小化估计风险实现主动学习的恶意代码检测方法.该方法只需要很少比例的训练样本就可实现准确的恶意代码检测,较现有方法更适用于新型恶意代码检测.通过我们对真实的8,340个正常进程以及7,257个恶意代码进程的实验分析,相比于传统基于统计分类器的检测方法,本文方法明显地提升了恶意代码检测效果.即便在训练样本仅为总体样本数量1%的情况下,本文方法可以也可达到5.55%的错误率水平,比传统方法降低了36.5%.     

基于Bagging-SVM的Android恶意软件检测模型

针对Android恶意软件检测中数据不平衡导致检出率低的问题,提出一种基于Bagging-SVM（支持向量机）集成算法的Android恶意软件检测模型。首先,提取AndroidManifest.xml文件中的权限信息、意图信息和组件信息作为特征;然后,提出IG-ReliefF混合筛选算法用于数据集降维,采用bootstrap抽样构造多个平衡数据集;最后,采用平衡数据集训练基于Bagging算法的SVM集成分类器,通过该分类器完成Android恶意软件检测。在分类检测实验中,当良性样本和恶意样本数量平衡时,Bagging-SVM和随机森林算法检出率均高达99.4%;当良性样本和恶意样本的数量比为4：1时,相比随机森林和AdaBoost算法,Bagging-SVM算法在检测精度不降低的条件下,检出率提高了6.6%。实验结果表明所提模型在数据不平衡时仍具有较高的检出率和分类精度,可检测出绝大多数恶意软件。     

A graph mining approach for detecting unknown malwares

Nowadays malware is one of the serious problems in the modern societies. Although the signature based malicious code detection is the standard technique in all commercial antivirus softwares, it can only achieve detection once the virus has already caused damage and it is registered. Therefore, it fails to detect new malwares (unknown malwares). Since most of malwares have similar behavior, a behavior based method can detect unknown malwares. The behavior of a program can be represented by a set of called API's (application programming interface). Therefore, a classifier can be employed to construct a learning model with a set of programs' API calls. Finally, an intelligent malware detection system is developed to detect unknown malwares automatically. On the other hand, we have an appealing representation model to visualize the executable files structure which is control flow graph (CFG). This model represents another semantic aspect of programs. This paper presents a robust semantic based method to detect unknown malwares based on combination of a visualize model (CFG) and called API's. The main contribution of this paper is extracting CFG from programs and combining it with extracted API calls to have more information about executable files. This new representation model is called API-CFG. In addition, to have fast learning and classification process, the control flow graphs are converted to a set of feature vectors by a nice trick. Our approach is capable of classifying unseen benign and malicious code with high accuracy. The results show a statistically significant improvement over n-grams based detection method.     

融合多特征的Android恶意软件检测方法

针对当前基于机器学习的Android恶意软件检测方法特征构建维度单一,难以全方位表征Android恶意软件行为特点的问题,文章提出一种融合软件行为特征、Android Manifest.xml文件结构特征和Android恶意软件分析经验特征的恶意软件检测方法。该方法提取Android应用的Dalvik操作码N-gram语义信息、系统敏感API、系统Intent、系统Category、敏感权限和相关经验特征,多方位表征Android恶意软件的行为并构建特征向量,采用基于XGBoost的集成学习算法构建分类模型,实现对恶意软件的准确分类。在公开数据集DREBIN和AMD上进行实验,实验结果表明,该方法能够达到高于97%的检测准确率,有效提升了Android恶意软件的检测效果。     

HEMD: a highly efficient random forest-based malware detection framework for Android

Mobile phones are rapidly becoming the most widespread and popular form of communication; thus, they are also the most important attack target of malware. The amount of malware in mobile phones is increasing exponentially and poses a serious security threat. Google’s Android is the most popular smart phone platforms in the world and the mechanisms of permission declaration access control cannot identify the malware. In this paper, we proposed an ensemble machine learning system for the detection of malware on Android devices. More specifically, four groups of features including permissions, monitoring system events, sensitive API and permission rate are extracted to characterize each Android application (app). Then an ensemble random forest classifier is learned to detect whether an app is potentially malicious or not. The performance of our proposed method is evaluated on the actual data set using tenfold cross-validation. The experimental results demonstrate that the proposed method can achieve a highly accuracy of 89.91%. For further assessing the performance of our method, we compared it with the state-of-the-art support vector machine classifier. Comparison results demonstrate that the proposed method is extremely promising and could provide a cost-effective alternative for Android malware detection.

     

基于PE静态结构特征的恶意软件检测方法

针对现有检测方法的不足,提出了一种通过挖掘PE文件结构信息来检测恶意软件的方法,并用最新的PE格式恶意软件进行了实验。结果显示,该方法以99.1%的准确率检测已知和未知的恶意软件,评价的重要指标AUC值是0.998,已非常接近最优值1,高于现有的静态检测方法。同时,与其他方法相比,该检测方法的处理时间和系统开销也是较少的,对采用加壳和混淆技术的恶意软件也保持稳定有效,已达到了实时部署使用要求。此外,现有的基于数据挖掘的检测方法在特征选择时存在过度拟合数据的情况,而该方法在这方面具有较强的鲁棒性。     

ELF-Miner: using structural knowledge and data mining methods to detect new (Linux) malicious executables

Linux malware can pose a significant threat—its (Linux) penetration is exponentially increasing—because little is known or understood about Linux OS vulnerabilities. We believe that now is the right time to devise non-signature based zero-day (previously unknown) malware detection strategies before Linux intruders take us by surprise. Therefore, in this paper, we first do a forensic analysis of Linux executable and linkable format (ELF) files. Our forensic analysis provides insight into different features that have the potential to discriminate malicious executables from benign ones. As a result, we can select a features’ set of 383 features that are extracted from an ELF headers. We quantify the classification potential of features using information gain and then remove redundant features by employing preprocessing filters. Finally, we do an extensive evaluation among classical rule-based machine learning classifiers—RIPPER, PART, C4.5 Rules, and decision tree J48—and bio-inspired classifiers—cAnt Miner, UCS, XCS, and GAssist—to select the best classifier for our system. We have evaluated our approach on an available collection of 709 Linux malware samples from vx heavens and offensive computing. Our experiments show that ELF-Miner provides more than 99% detection accuracy with less than 0.1% false alarm rate.     

Rapid pedestrian detection in unseen scenes

In this paper, a rapid adaptive pedestrian detection method based on cascade classifier with ternary pattern is proposed. The proposed method achieves its goal by employing the following three new strategies: (1) A method for adjusting the key parameters of the trained cascade classifier dynamically for detecting pedestrians in unseen scenes using only a small amount of labeled data from the new scenes. (2) An efficient optimization method is proposed, based on the cross entropy method and a priori knowledge of the scenes, to solve the classifier parameter optimization problem. (3) In order to further speed up pedestrian detection in unseen scenes, each strong classifier in the cascade employs a ternary detection pattern. In our experiments, two significantly different datasets, AHHF and NICTA, were used as the training set and testing set, respectively. The experimental results showed that the proposed method can quickly adapt a previously trained detector for pedestrian detection in various scenes compared with other existing methods.     

基于序列模式发现的恶意行为检测方法

为有效预防变形病毒和新出现的恶意软件,提出一种基于序列模式发现的恶意行为静态检测方法。将恶意代码转换为汇编代码,对其进行预处理,采用类Apriori算法完成序列模式发现,并去除正常模式,得到可用于未知恶意代码检测的模式集合。实验结果表明,该方法的正确率较高、漏报率较低。     

SubSeven’s Honey Pot Program

A serious security threat today are malicious executables, especially new, unseen malicious executables often arriving as email attachments. These new malicious executables are created at the rate of thousands every year¹ and pose a serious threat. Current anti-virus systems attempt to detect these new malicious programs with heuristic generated by hand. This approach is costly and often ineffective. In this paper we introduce the Trojan Horse SubSeven, its capabilities and influence over intrusion detection systems. A Honey Pot program is implemented, simulating the SubSeven Server. The Honey Pot Program provides feedback and stores data to and from the SubSeven’s client.     

Dynamic Analysis of Malicious Code

Malware analysis is the process of determining the purpose and functionality of a given malware sample (such as a virus, worm, or Trojan horse). This process is a necessary step to be able to develop effective detection techniques for malicious code. In addition, it is an important prerequisite for the development of removal tools that can thoroughly delete malware from an infected machine. Traditionally, malware analysis has been a manual process that is tedious and time-intensive. Unfortunately, the number of samples that need to be analyzed by security vendors on a daily basis is constantly increasing. This clearly reveals the need for tools that automate and simplify parts of the analysis process. In this paper, we present TTAnalyze, a tool for dynamically analyzing the behavior of Windows executables. To this end, the binary is run in an emulated operating system environment and its (security-relevant) actions are monitored. In particular, we record the Windows native system calls and Windows API functions that the program invokes. One important feature of our system is that it does not modify the program that it executes (e.g., through API call hooking or breakpoints), making it more difficult to detect by malicious code. Also, our tool runs binaries in an unmodified Windows environment, which leads to excellent emulation accuracy. These factors make TTAnalyze an ideal tool for quickly understanding the behavior of an unknown malware.     

基于机器学习算法的Android恶意程序检测系统

对于传统的恶意程序检测方法存在的缺点,针对将数据挖掘和机器学习算法被应用在未知恶意程序的检测方法进行研究。当前使用单一特征的机器学习算法无法充分发挥其数据处理能力,检测效果不佳。文中将语音识别模型与随机森林算法相结合,首次提出了综和APK文件多类特征统一建立N-gram模型,并应用随机森林算法用于未知恶意程序检测。首先,采用多种方式提取可以反映Android恶意程序行为的3类特征,包括敏感权限、DVM函数调用序列以及OpCodes特征;然后,针对每类特征建立N-gram模型,每个模型可以独立评判恶意程序行为;最后,3类特征模型统一加入随机森林算法进行学习,从而对Android程序进行检测。基于该方法实现了Android恶意程序检测系统,并对811个非恶意程序及826个恶意程序进行检测,准确率较高。综合各个评价指标,与其他相关工作对比,实验结果表明该系统在恶意程序检测准确率和有效性上表现更优。     

基于纹理指纹与活动向量空间的Android恶意代码检测

为了进一步提高恶意代码识别的准确率和自动化程度,提出一种基于深度学习的Android恶意代码分析与检测方法。首先,提出恶意代码纹理指纹体现恶意代码二进制文件块内容相似性,选取33类恶意代码活动向量空间来反映恶意代码的潜在动态活动。其次,为确保分类准确率的提高,融合上述特征,训练自编码器（AE）和Softmax分类器。通过对不同数据样本进行测试,利用栈式自编码（SAE）模型对Android恶意代码的分类平均准确率可达94.9%,比支持向量机（SVM）高出1.1个百分点。实验结果表明,所提出的方法能够有效提高恶意代码识别精度。     

改进的基于DNN的恶意软件检测方法

当前基于深度学习的恶意软件检测技术由于模型结构及样本预处理方式不够合理等原因,大多存在泛化性较差的问题,即训练好的恶意软件检测模型对不属于训练样本集的恶意软件或新出现的恶意软件的检出效果较差。提出一种改进的基于深度神经网络（Deep Neural Network,DNN）的恶意软件检测方法,使用多个全连接层构建恶意软件检测模型,并引入定向Dropout正则化方法,在模型训练过程中对神经网络中的权重进行剪枝。在Virusshare和lynx-project样本集上的实验结果表明,与同样基于DNN的恶意软件检测模型DeepMalNet相比,改进方法对恶意PE样本集的平均预测概率提高0.048,对被加壳的正常PE样本集的平均预测概率降低0.64。改进后的方法具有更好的泛化能力,对模型训练样本集外的恶意软件的检测效果更好。     

基于BPSO-NB算法的Android恶意应用检测方法

为了提高Android恶意应用检测效率,将二值粒子群算法(BPSO,Binary Particle Swarm Optimization)用于原始特征全集的优化选择,并结合朴素贝叶斯(NB,Nave Bayesian)分类算法,提出一种基于BPSO-NB的Android恶意应用检测方法。该方法首先对未知应用进行静态分析,提取AndroidManifest.xml文件中的权限信息作为特征。然后,采用BPSO算法优化选择分类特征,并使用NB算法的分类精度作为评价函数。最后采用NB分类算法构建Android恶意应用分类器。实验结果表明,通过二值粒子群优化选择分类特征可以有效提高分类精度,缩短检测时间。 