基于PE静态结构特征的恶意软件检测方法

针对现有检测方法的不足,提出了一种通过挖掘PE文件结构信息来检测恶意软件的方法,并用最新的PE格式恶意软件进行了实验。结果显示,该方法以99.1%的准确率检测已知和未知的恶意软件,评价的重要指标AUC值是0.998,已非常接近最优值1,高于现有的静态检测方法。同时,与其他方法相比,该检测方法的处理时间和系统开销也是较少的,对采用加壳和混淆技术的恶意软件也保持稳定有效,已达到了实时部署使用要求。此外,现有的基于数据挖掘的检测方法在特征选择时存在过度拟合数据的情况,而该方法在这方面具有较强的鲁棒性。     

像素归一化方法在恶意代码可视分析中的应用

恶意代码的编写者通常采用自动化的手段开发恶意代码变种,使得恶意代码的数量呈现迅猛增长的态势。由于自动化的方式会重复利用恶意代码中的核心模块,因此也为病毒研究人员辨识和区分恶意代码族提供了有利依据。借鉴灰度图的思想,利用K-Nearest Neighbor（KNN）分类算法,给出了一种新的研究恶意代码谱系分类的可视化方法。其基本思想是,通过将二进制文件转换成双色通道的位图和像素归一图,从可视化的角度标识恶意样本特性,以此实现恶意代码族的相似度比较及分类。实验结果表明采用了像素归一化的降维映射机制能显著地减小文件可视特征的呈现时间开销,且该方法以自动化操作的方式运用Jaccard距离算法进行快速相似度比较,实现了恶意代码样本的有效分类,提高了分析人员的识别效率。     

Detecting android malware using an improved filter based technique in embedded software

The technological advancements have led to evolution of sophisticated devices called smartphones. By providing extensive capabilities, they are becoming more and more popular. The Android based smartphones are preferred furthermore, due to their open-source nature. This has also led to the development of large number of malwares targeting these smartphones. Thus to protect the devices, some countermeasures are needed. Machine learning methods have gained popularity in detection of malware. This work proposes a malware detection technique in Android devices based on static analysis carried out using the Manifest files extracted from the apk files. The feature selection is performed using the proposed KNN based Relief algorithm and detection of malware is done using the proposed optimized SVM algorithm. The proposed method achieves a True Positive Rate greater than 0.70 and much reduced False Positive Rate values were obtained, with the values of False Positive Rate being very close to zero. The proposed KNN based feature selection is found to select better features in comparison with some popular existing feature selection techniques. The proposed optimized SVM technique achieves a performance that is on par with the performance of Neural Networks.     

Attention-based convolutional neural network deep learning approach for robust malware classification

Recently, transforming windows files into images and its analysis using machine learning and deep learning have been considered as a state-of-the art works for malware detection and classification. This is mainly due to the fact that image-based malware detection and classification is platform independent, and the recent surge of success of deep learning model performance in image classification. Literature survey shows that convolutional neural network (CNN) deep learning methods are successfully employed for image-based windows malware classification. However, the malwares were embedded in a tiny portion in the overall image representation. Identifying and locating these affected tiny portions is important to achieve a good malware classification accuracy. In this work, a multi-headed attention based approach is integrated to a CNN to locate and identify the tiny infected regions in the overall image. A detailed investigation and analysis of the proposed method was done on a malware image dataset. The performance of the proposed multi-headed attention-based CNN approach was compared with various non-attention-CNN-based approaches on various data splits of training and testing malware image benchmark dataset. In all the data-splits, the attention-based CNN method outperformed non-attention-based CNN methods while ensuring computational efficiency. Most importantly, most of the methods show consistent performance on all the data splits of training and testing and that illuminates multi-headed attention with CNN model's generalizability to perform on the diverse datasets. With less number of trainable parameters, the proposed method has achieved an accuracy of 99% to classify the 25 malware families and performed better than the existing non-attention based methods. The proposed method can be applied on any operating system and it has the capability to detect packed malware, metamorphic malware, obfuscated malware, malware family variants, and polymorphic malware. In addition, the proposed method is malware file agnostic and avoids usual methods such as disassembly, de-compiling, de-obfuscation, or execution of the malware binary in a virtual environment in detecting malware and classifying malware into their malware family.     

Comparing files using structural entropy

One of the main trends in the modern anti-virus industry is the development of algorithms that help estimate the similarity of files. Since malware writers tend to use increasingly complex techniques to protect their code such as obfuscation and polymorphism, anti-virus software vendors face problems of the increasing difficulty of file scanning, the considerable growth of anti-virus databases, and file storages overgrowth. For solving such problems, a static analysis of files appears to be of some interest. Its use helps determine those file characteristics that are necessary for their comparison without executing malware samples within a protected environment. The solution provided in this article is based on the assumption that different samples of the same malicious program have a similar order of code and data areas. Each such file area may be characterized not only by its length, but also by its homogeneity. In other words, the file may be characterized by the complexity of its data order. Our approach consists of using wavelet analysis for the segmentation of files into segments of different entropy levels and using edit distance between sequence segments to determine the similarity of the files. The proposed solution has a number of advantages that help detect malicious programs efficiently on personal computers. First, this comparison does not take into account the functionality of analysed files and is based solely on determining the similarity in code and data area positions which makes the algorithm effective against many ways of protecting executable code. On the other hand, such a comparison may result in false alarms. Therefore, our solution is useful as a preliminary test that triggers the running of additional checks. Second, the method is relatively easy to implement and does not require code disassembly or emulation. And, third, the method makes the malicious file record compact which is significant when compiling anti-virus databases.     

MACSPMD:基于恶意API调用序列模式挖掘的恶意代码检测

基于动态分析的恶意代码检测方法由于能有效对抗恶意代码的多态和代码混淆技术,而且可以检测新的未知恶意代码等,因此得到了研究者的青睐。在这种情况下,恶意代码的编写者通过在恶意代码中嵌入大量反检测功能来逃避现有恶意代码动态检测方法的检测。针对该问题,提出了基于恶意API调用序列模式挖掘的恶意代码检测方法MACSPMD。首先,使用真机模拟恶意代码的实际运行环境来获取文件的动态API调用序列;其次,引入面向目标关联挖掘的概念,以挖掘出能够代表潜在恶意行为模式的恶意API调用序列模式;最后,将挖掘到的恶意API调用序列模式作为异常行为特征进行恶意代码的检测。基于真实数据集的实验结果表明,MACSPMD对未知和逃避型恶意代码进行检测的准确率分别达到了94.55%和97.73%,比其他基于API调用数据的恶意代码检测方法 的准确率分别提高了2.47%和2.66%,且挖掘过程消耗的时间更少。因此,MACSPMD能有效检测包括逃避型在内的已知和未知恶意代码。     

A graph mining approach for detecting unknown malwares

Nowadays malware is one of the serious problems in the modern societies. Although the signature based malicious code detection is the standard technique in all commercial antivirus softwares, it can only achieve detection once the virus has already caused damage and it is registered. Therefore, it fails to detect new malwares (unknown malwares). Since most of malwares have similar behavior, a behavior based method can detect unknown malwares. The behavior of a program can be represented by a set of called API's (application programming interface). Therefore, a classifier can be employed to construct a learning model with a set of programs' API calls. Finally, an intelligent malware detection system is developed to detect unknown malwares automatically. On the other hand, we have an appealing representation model to visualize the executable files structure which is control flow graph (CFG). This model represents another semantic aspect of programs. This paper presents a robust semantic based method to detect unknown malwares based on combination of a visualize model (CFG) and called API's. The main contribution of this paper is extracting CFG from programs and combining it with extracted API calls to have more information about executable files. This new representation model is called API-CFG. In addition, to have fast learning and classification process, the control flow graphs are converted to a set of feature vectors by a nice trick. Our approach is capable of classifying unseen benign and malicious code with high accuracy. The results show a statistically significant improvement over n-grams based detection method.     

面向云平台的多样化恶意软件检测架构

近年来,恶意软件对物理机和云平台上虚拟机均构成巨大的安全威胁。在基础设施即服务（IaaS）云平台上部署传统的杀毒软件、防火墙等恶意软件检测工具存在以下问题：1）检测工具可能被破坏或者关闭;2）单一的检测工具效果不理想;3）检测工具可能被加壳等方式绕过;4）需要给每台客户机安装额外软件,难以部署实施。为此提出一种面向云平台的多样化恶意软件检测架构。该架构利用虚拟化技术截获客户机的特定行为,抓取客户机内软件释放的代码,通过多种杀毒软件多样化的扫描确定软件的恶意性。采用的动态内存提取的方式对客户机完全透明。最后在Xen上部署该架构并进行恶意软件检测测试,该架构对加壳恶意软件的检测率为85.7%,比杀毒软件静态扫描的检测率高14.3个百分点。实验结果表明,在云平台上采用多样化恶意软件检测框架能更好地保障客户机的安全。     

Packer identification using Byte plot and Markov plot

Malware is one of the major concerns in computer security. The availability of easy to use malware toolkits and internet popularity has led to the increase in number of malware attacks. Currently signature based malware detection techniques are widely used. However, malware authors use packing techniques to create new variants of existing malwares which defeat signature based malware detection. So, it is very important to identify packed malware and unpack it before analysis. Dynamic unpacking runs the packed executable and provides an unpacked version based on the system. This technique requires dedicated hardware and is computationally expensive. As each individual packer uses its own unpacking algorithm it is important to have a prior knowledge about the packer used, in order to assist in reverse engineering. In this paper, we propose an efficient framework for packer identification problem using Byte plot and Markov plot. First packed malware is converted to Byte plot and Markov plot. Later Gabor and wavelet based features are extracted from Byte plot and Markov plot. We used SVMs (Support Vector Machine) in our analysis. We performed our experiments on nine different packers and we obtained about 95 % accuracy for nine of the packers. Our results show features extracted from Markov plot outperformed features extracted from Byte plot by about 3 %. We compare the performance of Markov plot with PEID (Signature based PE identification tool). Our results show Markov plot produced better accuracy when compared to PEID. We also performed multi class classification using Random Forest and achieved 81 % accuracy using Markov plot based features.     

Music classification as a new approach for malware detection

Each year, a huge number of malicious programs are released which causes malware detection to become a critical task in computer security. Antiviruses use various methods for detecting malware, such as signature-based and heuristic-based techniques. Polymorphic and metamorphic malwares employ obfuscation techniques to bypass traditional detection methods used by antiviruses. Recently, the number of these malware has increased dramatically. Most of the previously proposed methods to detect malware are based on high-level features such as opcodes, function calls or program’s control flow graph (CFG). Due to new obfuscation techniques, extracting high-level features is tough, fallible and time-consuming; hence approaches using program’s bytes are quicker and more accurate. In this paper, a novel byte-level method for detecting malware by audio signal processing techniques is presented. In our proposed method, program’s bytes are converted to a meaningful audio signal, then Music Information Retrieval (MIR) techniques are employed to construct a machine learning music classification model from audio signals to detect new and unseen instances. Experiments evaluate the influence of different strategies converting bytes to audio signals and the effectiveness of the method.

     

一种基于多特征集成学习的恶意代码静态检测框架

伴随着互联网的普及和5G通信技术的快速发展,网络空间所面临的威胁日益增大,尤其是恶意软件的数量呈指数型上升,其所属家族的变种爆发式增加.传统的基于人工签名的恶意软件的检测方式速度太慢,难以处理每天数百万计新增的恶意软件,而普通的机器学习分类器的误报率和漏检率又明显过高.同时恶意软件的加壳、混淆等对抗技术对该情况造成了更大的困扰.基于此,提出一种基于多特征集成学习的恶意软件静态检测框架.通过提取恶意软件的非PE(Portable Executable)结构特征、可见字符串与汇编码序列特征、PE结构特征以及函数调用关系5部分特征,构建与各部分特征相匹配的模型,采用Bagging集成和Stacking集成算法,提升模型的稳定性,降低过拟合的风险.然后采取权重策略投票算法对5部分集成模型的输出结果做进一步聚合.经过测试,多特征多模型聚合的检测准确率可达96.99%,该结果表明:与其他静态检测方法相比,该方法具有更好的恶意软件鉴别能力,对加壳、混淆等恶意软件同样具备较高的识别率.     

A similarity metric method of obfuscated malware using function-call graph

Code obfuscating technique plays a significant role to produce new obfuscated malicious programs, generally called malware variants, from previously encountered malwares. However, the traditional signature-based malware detecting method is hard to recognize the up-to-the-minute obfuscated malwares. This paper proposes a method to identify the malware variants based on the function-call graph. Firstly, the function-call graphs were created from the disassembled codes of program; then the caller–callee relationships of functions and the operational code (opcode) information about functions, combining the graph coloring techniques were used to measure the similarity metric between two function-call graphs; at last, the similarity metric was utilized to identify the malware variants from known malwares. The experimental results show that the proposed method is able to identify the obfuscated malicious softwares effectively.     

基于灰度图纹理指纹的恶意软件分类

随着安卓恶意软件数量的快速增长,传统的恶意软件检测与分类机制存在检测率低、训练模型复杂度高等问题。为解决上述问题,结合图像纹理特征提取技术和机器学习分类器,提出基于灰度图纹理特征的恶意软件分类方法。该方法首先将恶意软件样本生成灰度图,设计并集成了包含GIST和Tamura特征提取算法在内的4种特征提取方法;然后将所得纹理特征集合作为源数据,基于Caffe高性能处理架构构造了5种分类学习模型,最终实现对恶意软件的检测和分类。实验结果表明,基于图像纹理特征的恶意软件分类具有较高的准确率,且Caffe架构能有效缩短学习时间,降低复杂度。     

改进的基于DNN的恶意软件检测方法

当前基于深度学习的恶意软件检测技术由于模型结构及样本预处理方式不够合理等原因,大多存在泛化性较差的问题,即训练好的恶意软件检测模型对不属于训练样本集的恶意软件或新出现的恶意软件的检出效果较差。提出一种改进的基于深度神经网络（Deep Neural Network,DNN）的恶意软件检测方法,使用多个全连接层构建恶意软件检测模型,并引入定向Dropout正则化方法,在模型训练过程中对神经网络中的权重进行剪枝。在Virusshare和lynx-project样本集上的实验结果表明,与同样基于DNN的恶意软件检测模型DeepMalNet相比,改进方法对恶意PE样本集的平均预测概率提高0.048,对被加壳的正常PE样本集的平均预测概率降低0.64。改进后的方法具有更好的泛化能力,对模型训练样本集外的恶意软件的检测效果更好。     

Hierarchical associative classifier (HAC) for malware detection from the large and imbalanced gray list

Nowadays, numerous attacks made by the malware (e.g., viruses, backdoors, spyware, trojans and worms) have presented a major security threat to computer users. Currently, the most significant line of defense against malware is anti-virus products which focus on authenticating valid software from a whitelist, blocking invalid software from a blacklist, and running any unknown software (i.e., the gray list) in a controlled manner. The gray list, containing unknown software programs which could be either normal or malicious, is usually authenticated or rejected manually by virus analysts. Unfortunately, along with the development of the malware writing techniques, the number of file samples in the gray list that need to be analyzed by virus analysts on a daily basis is constantly increasing. The gray list is not only large in size, but also has an imbalanced class distribution where malware is the minority class. In this paper, we describe our research effort on building automatic, effective, and interpretable classifiers resting on the analysis of Application Programming Interfaces (APIs) called by Windows Portable Executable (PE) files for detecting malware from the large and imbalanced gray list. Our effort is based on associative classifiers due to their high interpretability as well as their capability of discovering interesting relationships among API calls. We first adapt several different post-processing techniques of associative classification, including rule pruning and rule re-ordering, for building effective associative classifiers from large collections of training data. In order to help the virus analysts detect malware from the imbalanced gray list, we then develop the Hierarchical Associative Classifier (HAC). HAC constructs a two-level associative classifier to maximize precision and recall of the minority (malware) class: in the first level, it uses high precision rules of majority (benign file samples) class and low precision rules of minority class to achieve high recall; and in the second level, it ranks the minority class files and optimizes the precision. Finally, since our case studies are based on a large and real data collection obtained from the Anti-virus Lab of Kingsoft corporation, including 8,000,000 malware, 8,000,000 benign files, and 100,000 file samples from the gray list, we empirically examine the sampling strategy to build the classifiers for such a large data collection to avoid over-fitting and achieve great effectiveness as well as high efficiency. Promising experimental results demonstrate the effectiveness and efficiency of the HAC classifier. HAC has already been incorporated into the scanning tool of Kingsoft’s Anti-Virus software.     

February: Eventful horizon

February proved to be an interesting month on the virus front, in large part due to the controversy surrounding many of the newly discovered malware threats. Not only could vendors not agree on names, in one case they could not agree on whether the malware was a worm or a Trojan Horse — or even specifically what the malware proposed to do. Even the motive behind a worm became the subject of debate. However, the disagreements between anti-virus vendors were not the only common theme played out in the month of February.     

一种采用免疫原理的恶意软件检测方法

针对现有恶意软件检测方法的不足,提出一种采用免疫原理的恶意软件检测方法.该方法采用程序运行时产生的IRP请求序列作为抗原,定义系统中的正常程序为自体、恶意程序为非自体,通过选定数量的抗体,采用人工免疫原理识别非自体.实验结果表明,此方法在恶意软件的检测方面具有较高的准确率,且误报和漏报率较低.     

An intelligent PE-malware detection system based on association mining

The proliferation of malware has presented a serious threat to the security of computer systems. Traditional signature-based anti-virus systems fail to detect polymorphic/metamorphic and new, previously unseen malicious executables. Data mining methods such as Naive Bayes and Decision Tree have been studied on small collections of executables. In this paper, resting on the analysis of Windows APIs called by PE files, we develop the Intelligent Malware Detection System (IMDS) using Objective-Oriented Association (OOA) mining based classification. IMDS is an integrated system consisting of three major modules: PE parser, OOA rule generator, and rule based classifier. An OOA_Fast_FP-Growth algorithm is adapted to efficiently generate OOA rules for classification. A comprehensive experimental study on a large collection of PE files obtained from the anti-virus laboratory of KingSoft Corporation is performed to compare various malware detection approaches. Promising experimental results demonstrate that the accuracy and efficiency of our IMDS system outperform popular anti-virus software such as Norton AntiVirus and McAfee VirusScan, as well as previous data mining based detection systems which employed Naive Bayes, Support Vector Machine (SVM) and Decision Tree techniques. Our system has already been incorporated into the scanning tool of KingSoft’s Anti-Virus software. A short version of the paper is appeared in [33]. The work is partially supported by NSF IIS-0546280 and an IBM Faculty Research Award. The authors would also like to thank the members in the anti-virus laboratory at KingSoft Corporation for their helpful discussions and suggestions.     

Enhanced capsule network-based executable files malware detection and classification—deep learning approach

Malware has considerably increased recently, posing a serious security danger to both people and enterprises. In order to distinguish and stop the negative effects of malware, a variety of machine and deep learning approaches have been used to detect it. However, while extracting malware features, the feature-to-feature spatial hierarchy is not taken into account by the existing techniques and as a result, information is lost during the pooling operation. Hence, a modified capsule deep neural network was developed in which discriminative features are extracted from three channel image derived from malware binary with considering feature-to-feature spatial hierarchy. Also, conventional capsule deep neural network is modified by adding a global average pooling layer before fully connected layer thereby classified the dataset as malicious or benign without any loss of information. Moreover, these malwares were not accurately classified based on their families using existing variants of convolutional neural network (CNN) since malware family variants can modify due to minute changes in malware binaries. Hence, a hybrid deep convolutional neural network (DCNN) and long-short-term memory (LSTM) has been utilized that determine minute changes in malware binaries using LSTM without vanishing gradient issue and effectively perform malware family classification using DCNN. As a result, the proposed approach successfully identifies malware in executable files and categorizes malware into families with 98.5% accuracy.     

Systematisches Testen von Anti-Viren-Software

The application of anti-virus software (AV software) in companies is of increasing importance, caused by the enormous damages of different kinds of malware (malicious software). Features of different AV software systems vary in particular through the fast sequence of releases offered by different vendors. The reason for this release bombing is the still unbounded creativity of malware programmers. Therefore, it can only be analyzed through extensive and systematic tests, which software fits the current requirements regarding detection and disinfection of malware. In this paper first the potentials of damages caused by different kinds of malware will be described, followed by a presentation of a systematic test method for AV software.