A Survey on malware analysis and mitigation techniques

In recent days, malwares are advanced, sophisticatedly engineered to attack the target. Most of such advanced malwares are highly persistent and capable of escaping from the security systems. This paper explores such an advanced malware type called Advanced Persistent Threats (APTs). APTs pave the way for most of the Cyber espionages and sabotages. APTs are highly sophisticated, target specific and operate in a stealthy mode till the target is compromised. The intention of the APTs is to deploy target specific automated malwares in a host or network to initiate an on-demand attack based on continuous monitoring. Encrypted covert communication and advanced, sophisticated attack techniques make the identification of APTs more challenging. Conventional security systems like antivirus, anti-malware systems which depend on signatures and static analysis fail to identify these APTs. The Advanced Evasive Techniques (AET) used in APTs are capable of bypassing the stateful firewalls housed in the enterprise choke points at ease. Hence, this paper presents a detailed study on sophisticated attack and evasion techniques used by the contemporary malwares. Furthermore, existing malware analysis techniques, application hardening techniques and CPU assisted application security schemes are also discussed. Finally, the study concludes by presenting the System and Network Security Design (SNSD) using existing mitigation techniques.     

基于改进随机森林算法的Android恶意软件检测

Android系统的开放性和第三方应用市场的多样性,使其在取得高市场占有率的同时也带来了巨大的风险,导致Android恶意应用层出不穷并广泛传播,严重威胁了用户的隐私和经济安全. 如何有效检测Android恶意应用受到了研究人员的广泛关注. 根据是否运行应用程序,将现有的恶意应用检测方法分为静态检测和动态检测. 其中,静态检测的效率和代码覆盖率均优于动态检测,Drebin等静态检测工具取得了广泛应用. 为此,系统调研了Android恶意应用静态检测领域的研究进展,并进行了分析和总结. 首先,介绍了Android应用静态特征;然后,根据静态特征的不同,分别对基于权限、应用程序编程接口（application programming interface,API）和操作码等不同静态特征的Android恶意应用检测方法进行了分析,并总结了常用的Android应用数据集和评价Android恶意应用检测性能的常用指标;最后,对Android恶意应用静态检测技术的发展进行了总结和展望,以期为该领域的研究人员提供参考.

     

面向云平台的多样化恶意软件检测架构

近年来,恶意软件对物理机和云平台上虚拟机均构成巨大的安全威胁。在基础设施即服务（IaaS）云平台上部署传统的杀毒软件、防火墙等恶意软件检测工具存在以下问题：1）检测工具可能被破坏或者关闭;2）单一的检测工具效果不理想;3）检测工具可能被加壳等方式绕过;4）需要给每台客户机安装额外软件,难以部署实施。为此提出一种面向云平台的多样化恶意软件检测架构。该架构利用虚拟化技术截获客户机的特定行为,抓取客户机内软件释放的代码,通过多种杀毒软件多样化的扫描确定软件的恶意性。采用的动态内存提取的方式对客户机完全透明。最后在Xen上部署该架构并进行恶意软件检测测试,该架构对加壳恶意软件的检测率为85.7%,比杀毒软件静态扫描的检测率高14.3个百分点。实验结果表明,在云平台上采用多样化恶意软件检测框架能更好地保障客户机的安全。     

MarCHGen: A framework for generating a malware concept hierarchy

Automatic classification of virus instances into a concept hierarchy has been attracting much attention from malware research community. However, it is definitely not a trivial work, because malwares usually come in binary forms whose actions are complicated and obfuscated. Therefore, the typical data mining approaches based on feature extraction are not easily applied. In this paper, we tackle this problem by introducing a framework known as MarCHGen (Malware Concept Hierarchy Generation). In this framework, we first apply virus logical concept analysis, which incorporates formal concept analysis with temporal logic to capture malware behaviours and generalize a virus concept lattice accordingly. Second, we propose an on‐the‐fly conceptual clustering technique to generate a malware concept hierarchy. In the MarCHGen framework, the malware concept hierarchy will be monitored by the prelarge data set management technique to avoid reclustering several times unnecessarily. Our approach has been applied in a real data set of virus, and promising experimental results have been acquired.     

HDM-Analyser: a hybrid analysis approach based on data mining techniques for malware detection

Today’s security threats like malware are more sophisticated and targeted than ever, and they are growing at an unprecedented rate. To deal with them, various approaches are introduced. One of them is Signature-based detection, which is an effective method and widely used to detect malware; however, there is a substantial problem in detecting new instances. In other words, it is solely useful for the second malware attack. Due to the rapid proliferation of malware and the desperate need for human effort to extract some kinds of signature, this approach is a tedious solution; thus, an intelligent malware detection system is required to deal with new malware threats. Most of intelligent detection systems utilise some data mining techniques in order to distinguish malware from sane programs. One of the pivotal phases of these systems is extracting features from malware samples and benign ones in order to make at least a learning model. This phase is called “Malware Analysis” which plays a significant role in these systems. Since API call sequence is an effective feature for realising unknown malware, this paper is focused on extracting this feature from executable files. There are two major kinds of approach to analyse an executable file. The first type of analysis is “Static Analysis” which analyses a program in source code level. The second one is “Dynamic Analysis” that extracts features by observing program’s activities such as system requests during its execution time. Static analysis has to traverse the program’s execution path in order to find called APIs. Because it does not have sufficient information about decision making points in the given executable file, it is not able to extract the real sequence of called APIs. Although dynamic analysis does not have this drawback, it suffers from execution overhead. Thus, the feature extraction phase takes noticeable time. In this paper, a novel hybrid approach, HDM-Analyser, is presented which takes advantages of dynamic and static analysis methods for rising speed while preserving the accuracy in a reasonable level. HDM-Analyser is able to predict the majority of decision making points by utilising the statistical information which is gathered by dynamic analysis; therefore, there is no execution overhead. The main contribution of this paper is taking accuracy advantage of the dynamic analysis and incorporating it into static analysis in order to augment the accuracy of static analysis. In fact, the execution overhead has been tolerated in learning phase; thus, it does not impose on feature extraction phase which is performed in scanning operation. The experimental results demonstrate that HDM-Analyser attains better overall accuracy and time complexity than static and dynamic analysis methods.     

Applying NLP techniques to malware detection in a practical environment

International Journal of Information Security - Executable files still remain popular to compromise the endpoint computers. These executable files are often obfuscated to avoid anti-virus programs....     

DeepAM: a heterogeneous deep learning framework for intelligent malware detection

With computers and the Internet being essential in everyday life, malware poses serious and evolving threats to their security, making the detection of malware of utmost concern. Accordingly, there have been many researches on intelligent malware detection by applying data mining and machine learning techniques. Though great results have been achieved with these methods, most of them are built on shallow learning architectures. Due to its superior ability in feature learning through multilayer deep architecture, deep learning is starting to be leveraged in industrial and academic research for different applications. In this paper, based on the Windows application programming interface calls extracted from the portable executable files, we study how a deep learning architecture can be designed for intelligent malware detection. We propose a heterogeneous deep learning framework composed of an AutoEncoder stacked up with multilayer restricted Boltzmann machines and a layer of associative memory to detect newly unknown malware. The proposed deep learning model performs as a greedy layer-wise training operation for unsupervised feature learning, followed by supervised parameter fine-tuning. Different from the existing works which only made use of the files with class labels (either malicious or benign) during the training phase, we utilize both labeled and unlabeled file samples to pre-train multiple layers in the heterogeneous deep learning framework from bottom to up for feature learning. A comprehensive experimental study on a real and large file collection from Comodo Cloud Security Center is performed to compare various malware detection approaches. Promising experimental results demonstrate that our proposed deep learning framework can further improve the overall performance in malware detection compared with traditional shallow learning methods, deep learning methods with homogeneous framework, and other existing anti-malware scanners. The proposed heterogeneous deep learning framework can also be readily applied to other malware detection tasks.     

A defense framework against malware and vulnerability exploits

Current anti-malware tools have proved to be insufficient in combating ever-evolving malware attacks and vulnerability exploits due to inevitable vulnerabilities present in the complex software used today. In addition, the performance penalty incurred by anti-malware tools is magnified when security approaches designed for desktops are migrated to modern mobile devices, such as tablets and laptops, due to their relatively limited processing capabilities and battery capacities. In this paper, we propose a fine-grained anomaly detection defense framework that offers a cost-efficient way to detect malicious behavior and prevent vulnerability exploits in resource-constrained computing platforms. In this framework, a trusted third party (e.g., the publisher) first tests a new application by running it in a heavily monitored testing environment that emulates the target system and extracts a behavioral model from its execution paths. Extensive security policies are enforced during this process. In case of a violation, the program is denied release to the user. If the application passes the tests, the user can download the behavioral model along with the tested application binary. At run-time, the application is monitored against the behavioral model. In the unlikely event that a new execution path is encountered, conservative but lightweight security policies are applied. To reduce overhead at the user end, the behavioral model may be further reduced by the publisher through static analysis. We have implemented the defense framework using a netbook with the Intel Atom processor and evaluated it with a suite of 51 real-world Linux viruses and malware. Experiments demonstrate that our tool achieves a very high coverage (98 %) of considered malware and security threats. The four antivirus tools we compare our tool against were found to have poor virus coverage, especially of obfuscated viruses. By removing safe standard library blocks from the behavioral model, we reduce the model size by 8.4 \(\times \) and the user’s run-time overhead by 23 %.     

增强智能手机安全的动态恶意软件分析系统

分析了智能手机上恶意软件的现状和发展趋势,给出了当前防范手机恶意软件的措施及其不足之处.适应大多数手机用户的安全需求,提出了一种能够满足增强智能手机安全的方案--移动动态恶意软件分析系统.给出了恶意软件分析模块的具体设计和整个分析系统的实现构思.最后,指出了分析系统的不足并提出解决方法.智能手机恶意软件的现状使该方案具有很好的应用前景.     

HEMD: a highly efficient random forest-based malware detection framework for Android

Mobile phones are rapidly becoming the most widespread and popular form of communication; thus, they are also the most important attack target of malware. The amount of malware in mobile phones is increasing exponentially and poses a serious security threat. Google’s Android is the most popular smart phone platforms in the world and the mechanisms of permission declaration access control cannot identify the malware. In this paper, we proposed an ensemble machine learning system for the detection of malware on Android devices. More specifically, four groups of features including permissions, monitoring system events, sensitive API and permission rate are extracted to characterize each Android application (app). Then an ensemble random forest classifier is learned to detect whether an app is potentially malicious or not. The performance of our proposed method is evaluated on the actual data set using tenfold cross-validation. The experimental results demonstrate that the proposed method can achieve a highly accuracy of 89.91%. For further assessing the performance of our method, we compared it with the state-of-the-art support vector machine classifier. Comparison results demonstrate that the proposed method is extremely promising and could provide a cost-effective alternative for Android malware detection.

     

Malicious uses of blockchains by malware: from the analysis to Smart-Zephyrus

The permanent availability and relative obscurity of blockchains is the perfect ground for using them for malicious purposes. However, the use of blockchains by malwares has not been characterized yet. This paper analyses the current state of the art in this area. One of the lessons learned is that covert communications for malware have received little attention. To foster further defence-oriented research, a novel mechanism (dubbed Smart-Zephyrus) is built leveraging smart contracts written in Solidity. Our results show that it is possible to hide 4 Kb of secret in 41 s. While being expensive (around USD 1.82 per bit), the provided stealthiness might be worth the price for attackers.

     

Hybrid emulation for bypassing anti-reversing techniques and analyzing malware

The Journal of Supercomputing - Malware uses a variety of anti-reverse engineering techniques, which makes its analysis difficult. Dynamic analysis tools, e.g., debuggers, DBI (Dynamic Binary...     

The other guys: automated analysis of marginalized malware

In order to thwart dynamic analysis and bypass protection mechanisms, malware have been using several file formats and evasive techniques. While publicly available dynamic malware analysis systems are one of the main sources of information for researchers, security analysts and incident response professionals, they are unable to cope with all types of threats. Therefore, it is difficult to gather information from public systems about CPL, .NET/Mono, 64-bits, reboot-dependent, or malware targeting systems newer than Windows XP, which result in a lack of understanding about how current malware behave during infections on modern operating systems. In this paper, we discuss the challenges and issues faced during the development of this type of analysis system, mainly due to security features available in NT 6.x kernel versions of Windows OS. We also introduce a dynamic analysis system that addresses the aforementioned types of malware as well as present results obtained from their analyses.     

Graph-based malware detection using dynamic analysis

We introduce a novel malware detection algorithm based on the analysis of graphs constructed from dynamically collected instruction traces of the target executable. These graphs represent Markov chains, where the vertices are the instructions and the transition probabilities are estimated by the data contained in the trace. We use a combination of graph kernels to create a similarity matrix between the instruction trace graphs. The resulting graph kernel measures similarity between graphs on both local and global levels. Finally, the similarity matrix is sent to a support vector machine to perform classification. Our method is particularly appealing because we do not base our classifications on the raw n-gram data, but rather use our data representation to perform classification in graph space. We demonstrate the performance of our algorithm on two classification problems: benign software versus malware, and the Netbull virus with different packers versus other classes of viruses. Our results show a statistically significant improvement over signature-based and other machine learning-based detection methods.     

Entropy analysis to classify unknown packing algorithms for malware detection

The proportion of packed malware has been growing rapidly and now comprises more than 80 % of all existing malware. In this paper, we propose a method for classifying the packing algorithms of given unknown packed executables, regardless of whether they are malware or benign programs. First, we scale the entropy values of a given executable and convert the entropy values of a particular location of memory into symbolic representations. Our proposed method uses symbolic aggregate approximation (SAX), which is known to be effective for large data conversions. Second, we classify the distribution of symbols using supervised learning classification methods, i.e., naive Bayes and support vector machines for detecting packing algorithms. The results of our experiments involving a collection of 324 packed benign programs and 326 packed malware programs with 19 packing algorithms demonstrate that our method can identify packing algorithms of given executables with a high accuracy of 95.35 %, a recall of 95.83 %, and a precision of 94.13 %. We propose four similarity measurements for detecting packing algorithms based on SAX representations of the entropy values and an incremental aggregate analysis. Among these four metrics, the fidelity similarity measurement demonstrates the best matching result, i.e., a rate of accuracy ranging from 95.0 to 99.9 %, which is from 2 to 13  higher than that of the other three metrics. Our study confirms that packing algorithms can be identified through an entropy analysis based on a measure of the uncertainty of the running processes and without prior knowledge of the executables.     

A comprehensive survey on deep learning based malware detection techniques

Recent theoretical and practical studies have revealed that malware is one of the most harmful threats to the digital world. Malware mitigation techniques have evolved over the years to ensure security. Earlier, several classical methods were used for detecting malware embedded with various features like the signature, heuristic, and others. Traditional malware detection techniques were unable to defeat new generations of malware and their sophisticated obfuscation tactics. Deep Learning is increasingly used in malware detection as DL-based systems outperform conventional malware detection approaches at finding new malware variants. Furthermore, DL-based techniques provide rapid malware prediction with excellent detection rates and analysis of different malware types. Investigating recently proposed Deep Learning-based malware detection systems and their evolution is hence of interest to this work. It offers a thorough analysis of the recently developed DL-based malware detection techniques. Furthermore, current trending malwares are studied and detection techniques of Mobile malware (both Android and iOS), Windows malware, IoT malware, Advanced Persistent Threats (APTs), and Ransomware are precisely reviewed.     

Abstracting minimal security-relevant behaviors for malware analysis

Dynamic behavior-based malware analysis and detection is considered to be one of the most promising ways to combat with the obfuscated and unknown malwares. To perform such analysis, behavioral feature abstraction plays a fundamental role, because how to specify program formally to a large extend determines what kind of algorithm can be used. In existing research, graph-based methods keep a dominant position in specifying malware behaviors. However, they restrict the detection algorithm to be chosen from graph mining algorithm. In this paper, we build a complete virtual environment to capture malware behaviors, especially that to stimulate network behaviors of a malware. Then, we study the problem of abstracting constant behavioral features from API call sequences and propose a minimal security-relevant behavior abstraction way, which absorbs the advantages of prevalent graph-based methods in behavior representation and has the following advantages: first API calls are aggregated by data dependence, therefore it is resistent to redundant data and is a kind of more constant feature. Second, API call arguments are also abstracted particularly, this further contributes to common and constant behavioral features of malware variants. Third, it is a moderate degree aggregation of a small group of API calls with a constructing criterion that centering on an independent operation on a sensitive resource. Fourth, it is very easy to embed the extracted behaviors in a high dimensional vector space, so that it can be processed by almost all of the prevalent statistical learning algorithms. We then evaluate these minimal security-relevant behaviors in three kinds of test, including similarity comparison, clustering and classification. The experimental results show that our method has a capacity in distinguishing malwares from different families and also from benign programs, and it is useful for many statistical learning algorithms.     

A concise cost analysis of Internet malware

In this paper we present a cost model to analyze impacts of Internet malware in order to estimate the cost of incidents and risk caused by them. The model is useful in determining parameters needed to estimate recovery efficiency, probabilistic risk distributions, and cost of malware incidents. Many users tend to underestimate the cost of curiosity coming with stealth malware such as email-attachments, freeware/shareware, spyware (including keyloggers, password thieves, phishing-ware, network sniffers, stealth backdoors, and rootkits), popups, and peer-to-peer fileshares. We define two sets of functions to describe evolution of attacks and potential loss caused by malware, where the evolution functions analyze infection patterns, while the loss functions provide risk-impact analysis of failed systems. Due to a wide range of applications, such analyses have drawn the attention of many engineers and researchers. Analysis of malware propagation itself has little to contribute unless tied to analysis of system performance, economic loss, and risks.