“Andromaly”: a behavioral malware detection framework for android devices

This article presents Andromaly—a framework for detecting malware on Android mobile devices. The proposed framework realizes a Host-based Malware Detection System that continuously monitors various features and events obtained from the mobile device and then applies Machine Learning anomaly detectors to classify the collected data as normal (benign) or abnormal (malicious). Since no malicious applications are yet available for Android, we developed four malicious applications, and evaluated Andromaly’s ability to detect new malware based on samples of known malware. We evaluated several combinations of anomaly detection algorithms, feature selection method and the number of top features in order to find the combination that yields the best performance in detecting new malware on Android. Empirical results suggest that the proposed framework is effective in detecting malware on mobile devices in general and on Android in particular.     

User-mode memory scanning on 32-bit &; 64-bit windows

Memory scanning is an essential component in detecting and deactivating malware while the malware is still active in memory. The content here is confined to user-mode memory scanning for malware on 32-bit and 64-bit Windows NT based systems that are memory resident and/or persistent over reboots. Malware targeting 32-bit Windows are being created and deployed at an alarming rate today. While there are not many malware targeting 64-bit Windows yet, many of the existing Win32 malware for 32-bit Windows will work fine on 64-bit Windows due to the underlying WoW64 subsystem. Here, we will present an approach to implement user-mode memory scanning for Windows. This essentially means scanning the virtual address space of all processes in memory. In case of an infection, while the malware is still active in memory, it can significantly limit detection and disinfection. The real challenge hence actually lies in fully disinfecting the machine and restoring back to its clean state. Today’s malware apply complex anti-disinfection techniques making the task of restoring the machine to a clean state extremely difficult. Here, we will discuss some of these techniques with examples from real-world malware scenarios. Practical approaches for user-mode disinfection will be presented. By leveraging the abundance of redundant information available via various Win32 and Native API from user-mode, certain techniques to detect hidden processes will also be presented. Certain challenges in porting the memory scanner to 64-bit Windows and Vista will be discussed. The advantages and disadvantages of implementing a memory scanner in user-mode (rather than kernel-mode) will also be discussed.     

A comprehensive survey on deep learning based malware detection techniques

Recent theoretical and practical studies have revealed that malware is one of the most harmful threats to the digital world. Malware mitigation techniques have evolved over the years to ensure security. Earlier, several classical methods were used for detecting malware embedded with various features like the signature, heuristic, and others. Traditional malware detection techniques were unable to defeat new generations of malware and their sophisticated obfuscation tactics. Deep Learning is increasingly used in malware detection as DL-based systems outperform conventional malware detection approaches at finding new malware variants. Furthermore, DL-based techniques provide rapid malware prediction with excellent detection rates and analysis of different malware types. Investigating recently proposed Deep Learning-based malware detection systems and their evolution is hence of interest to this work. It offers a thorough analysis of the recently developed DL-based malware detection techniques. Furthermore, current trending malwares are studied and detection techniques of Mobile malware (both Android and iOS), Windows malware, IoT malware, Advanced Persistent Threats (APTs), and Ransomware are precisely reviewed.     

Metamorphic Malware Detection Using Code Metrics

ABSTRACT

Malware is becoming more and more aggressive and new techniques are emerging to allow malicious code to evade detection by antiviruses. Metamorphic malware is a particularly insidious kind of virus that changes its form at each infection. In this article, a technique for detecting metamorphic viruses is proposed that is based on identifying specific features of the assembly code, such as the instructions that change the contents of the registers, the instructions that change the control flow, and the potential code fragmentation. Such features have been derived by the analysis of a large dataset of malware. The experimentation suggests that the proposed technique produces very high precision (over 97%) in recognizing metamorphic malware, and allows also for distinguishing among different families of malware.     

“VANILLA” malware: vanishing antiviruses by interleaving layers and layers of attacks

Malware are persistent threats to any networked systems. Recent years increase in multi-core, distributed systems created new opportunities for malware authors to exploit such capabilities. In particular, the distributed execution of a malware in multiple cores may be used to evade currently widespread single-core-based detectors (e.g., antiviruses, or AVs) and malware analysis solutions that are unable to correlate data from multiple sources. In this paper, we propose a technique for distributing the malware functions in several distinct “vanilla” processes to show that AVs can be easily evaded. Therefore, our technique allows malware to interleave of layers of attacks to remain undetected by current AVs. Our goal is to expose a real menace and to discuss it so as to provide insights for the development of better AVs. We discuss the role of distributed and multicore-based malware in current and future threat scenarios with practical examples that we specially crafted for testing (e.g., a distributed sample synchronized via cache side channels). We (i) review multi-threaded/processed implementation issues (from kernel and userland) and present a multi-core-based monitoring solution; (ii) present strategies for code distribution, exemplified via DLL injectors, and discuss their weak and strong points; and (iii) evaluate how real security solutions perform when exposed to distributed malware. We converted real, serial malware to parallel code and showed that current AVs are not fully able to detect multi-core malware.

     

Multi-context features for detecting malicious programs

Malware detection is still an open problem. There are numerous attacks that take place every day where malware is used to steal private information, disrupt services, or sabotage industrial systems. In this paper, we combine three kinds of contextual information, namely static, dynamic, and instruction-based, for malware detection. This leads to the definition of more than thirty thousand features, which is a large features set that covers a wide range of a sample characteristics. Through experiments with one million files, we show that this features set leads to machine learning based models that can detect both malware seen roughly at the time when the models are built, and malware first seen even months after the models were built (i.e., the detection models remain effective months ahead of time). This may be due to the comprehensiveness of the features set.     

Behavior-based features model for malware detection

The sharing of malicious code libraries and techniques over the Internet has vastly increased the release of new malware variants in an unprecedented rate. Malware variants share similar behaviors yet they have different syntactic structure due to the incorporation of many obfuscation and code change techniques such as polymorphism and metamorphism. The different structure of malware variants poses a serious problem to signature-based detection technique, yet their similar exhibited behaviors and actions can be a remarkable feature to detect them by behavior-based techniques. Malware instances also largely depend on API calls provided by the operating system to achieve their malicious tasks. Therefore, behavior-based detection techniques that utilize API calls are promising for the detection of malware variants. In this paper, we propose a behavior-based features model that describes malicious action exhibited by malware instance. To extract the proposed model, we first perform dynamic analysis on a relatively recent malware dataset inside a controlled virtual environment and capture traces of API calls invoked by malware instances. The traces are then generalized into high-level features we refer to as actions. We assessed the viability of actions by various classification algorithms such as decision tree, random forests, and support vector machine. The experimental results demonstrate that the classifiers attain high accuracy and satisfactory results in the detection of malware variants.     

Software transformations to improve malware detection

Malware is code designed for a malicious purpose, such as obtaining root privilege on a host. A malware detector identifies malware and thus prevents it from adversely affecting a host. In order to evade detection, malware writers use various obfuscation techniques to transform their malware. There is strong evidence that commercial malware detectors are susceptible to these evasion tactics. In this paper, we describe the design and implementation of a malware transformer that reverses the obfuscations performed by a malware writer. Our experimental evaluation demonstrates that this malware transformer can drastically improve the detection rates of commercial malware detectors.     

An intelligent PE-malware detection system based on association mining

The proliferation of malware has presented a serious threat to the security of computer systems. Traditional signature-based anti-virus systems fail to detect polymorphic/metamorphic and new, previously unseen malicious executables. Data mining methods such as Naive Bayes and Decision Tree have been studied on small collections of executables. In this paper, resting on the analysis of Windows APIs called by PE files, we develop the Intelligent Malware Detection System (IMDS) using Objective-Oriented Association (OOA) mining based classification. IMDS is an integrated system consisting of three major modules: PE parser, OOA rule generator, and rule based classifier. An OOA_Fast_FP-Growth algorithm is adapted to efficiently generate OOA rules for classification. A comprehensive experimental study on a large collection of PE files obtained from the anti-virus laboratory of KingSoft Corporation is performed to compare various malware detection approaches. Promising experimental results demonstrate that the accuracy and efficiency of our IMDS system outperform popular anti-virus software such as Norton AntiVirus and McAfee VirusScan, as well as previous data mining based detection systems which employed Naive Bayes, Support Vector Machine (SVM) and Decision Tree techniques. Our system has already been incorporated into the scanning tool of KingSoft’s Anti-Virus software. A short version of the paper is appeared in [33]. The work is partially supported by NSF IIS-0546280 and an IBM Faculty Research Award. The authors would also like to thank the members in the anti-virus laboratory at KingSoft Corporation for their helpful discussions and suggestions.     

SBMDS: an interpretable string based malware detection system using SVM ensemble with bagging

Malicious executables are programs designed to infiltrate or damage a computer system without the owner’s consent, which have become a serious threat to the security of computer systems. There is an urgent need for effective techniques to detect polymorphic, metamorphic and previously unseen malicious executables of which detection fails in most of the commercial anti-virus software. In this paper, we develop interpretable string based malware detection system (SBMDS), which is based on interpretable string analysis and uses support vector machine (SVM) ensemble with Bagging to classify the file samples and predict the exact types of the malware. Interpretable strings contain both application programming interface (API) execution calls and important semantic strings reflecting an attacker’s intent and goal. Our SBMDS is carried out with four major steps: (1) first constructing the interpretable strings by developing a feature parser; (2) performing feature selection to select informative strings related to different types of malware; (3) followed by using SVM ensemble with bagging to construct the classifier; (4) and finally conducting the malware detector, which not only can detect whether a program is malicious or not, but also can predict the exact type of the malware. Our case study on the large collection of file samples collected by Kingsoft Anti-virus lab illustrate that: (1) The accuracy and efficiency of our SBMDS outperform several popular anti-virus software; (2) Based on the signatures of interpretable strings, our SBMDS outperforms data mining based detection systems which employ single SVM, Naive Bayes with bagging, Decision Trees with bagging; (3) Compared with the IMDS which utilizes the objective-oriented association (OOA) based classification on API calls, our SBMDS achieves better performance. Our SBMDS system has already been incorporated into the scanning tool of a commercial anti-virus software.     

一种基于模糊哈希的Android变种恶意软件检测方法

Android移动平台中恶意软件变种数量与日俱增,为了能够高效快速地检测出变种样本,提出一种能够根据Apk中字符串以及函数长度分布特征,来生成模糊哈希值的方法,使得同类变种的恶意软件间的哈希值相似。在对变种恶意软件进行检测时,首先利用k-means方法对已知病毒库所产生的模糊哈希值进行聚类,从而简化病毒库。再利用哈密顿距离来计算其与病毒库中各模糊哈希间哈密顿距离。当距离小于阈值,则表示检测到变种。实验结果表明,提出的方法具有检测速度快,抗干扰能力强等特点。     

恶意软件检测中解决样本不平衡问题的策略

互联网技术已经使人们的生活和工作发生了巨大的改变．然而,人们在享受互联网提供的便利的同时,也承受着恶意程序带来的威胁．在数字化时代的今天,与恶意程序的对抗已成为信息领域的焦点．由于恶意软件检测中的恶意软件样本难于获取,同时,标记大量的样本也需要花费大量的人力和物力,所获得的恶意软件样本远远少于正常软件样本,因此各类的训练样本之间存在分布不平衡的分类问题．为了解决该问题,本文提出采用SMOTE过采样方法,通过合理的增加少数类样本来解决样本不平衡问题．     

基于行为依赖特征的恶意代码相似性比较方法

恶意代码相似性比较是恶意代码分析和检测的基础性工作之一,现有方法主要是基于代码结构或行为序列进行比较.但恶意代码编写者常采用代码混淆、程序加壳等手段对恶意代码进行处理,导致传统的相似性比较方法失效.提出了一种基于行为之间控制依赖关系和数据依赖关系的恶意代码相似性比较方法,该方法利用动态污点传播分析识别恶意行为之间的依赖关系,然后,以此为基础构造控制依赖图和数据依赖图,根据两种依赖关系进行恶意代码的相似性比较.该方法充分利用了恶意代码行为之间内在的关联性,提高了比较的准确性,具有较强的抗干扰能力;通过循环消除、垃圾行为删除等方法对依赖图进行预处理,降低了相似性比较算法的复杂度,加快了比较速度.实验结果表明,与现有方法相比,该方法的准确性和抗干扰能力均呈现明显优势.     

Big Data Framework for Zero-Day Malware Detection

Malware has already been recognized as one of the most dominant cyber threats on the Internet today. It is growing exponentially in terms of volume, variety, and velocity, and thus overwhelms the traditional approaches used for malware detection and classification. Moreover, with the advent of Internet of Things, there is a huge growth in the volume of digital devices and in such scenario, malicious binaries are bound to grow even faster making it a big data problem. To analyze and detect unknown malware on a large scale, security analysts need to make use of machine learning algorithms along with big data technologies. These technologies help them to deal with current threat landscape consisting of complex and large flux of malicious binaries. This paper proposes the design of a scalable architecture built on the top of Apache Spark which uses its scalable machine learning library (MLlib) for detecting zero-day malware. The proposed platform is tested and evaluated on a dataset comprising of 0.2 million files consisting of 0.05 million clean files and 0.15 million malicious binaries covering a large number of malware families over a period of 7 years starting from 2010.     

Pushdown model checking for malware detection

The number of malware is growing extraordinarily fast. Therefore, it is important to have efficient malware detectors. Malware writers try to obfuscate their code by different techniques. Many well-known obfuscation techniques rely on operations on the stack such as inserting dead code by adding useless push and pop instructions, or hiding calls to the operating system, etc. Thus, it is important for malware detectors to be able to deal with the program’s stack. In this study, we propose a new model-checking approach for malware detection that takes into account the behavior of the stack. Our approach consists in: (1) Modeling the program using a pushdown system (PDS). (2) Introducing a new logic, called stack computation tree predicate logic (SCTPL), to represent the malicious behavior. SCTPL can be seen as an extension of the branching-time temporal logic CTL with variables, quantifiers, and predicates over the stack. (3) Reducing the malware detection problem to the model-checking problem of PDSs against SCTPL formulas. We show how our new logic can be used to precisely express malicious behaviors that could not be specified by existing specification formalisms. We then consider the model-checking problem of PDSs against SCTPL specifications. We reduce this problem to emptiness checking in Symbolic Alternating Büchi Pushdown Systems, and we provide an algorithm to solve this problem. We implemented our techniques in a tool and applied it to detect several viruses. Our results are encouraging.     

Metamorphic malware identification using engine-specific patterns based on co-opcode graphs

A metamorphic virus is a type of malware that modifies its code using a morphing engine. Morphing engines are used to generate a large number of metamorphic malware variants by performing different obfuscation techniques. Since each metamorphic malware has its own unique structure, signature based anti-virus programs are ineffective to detect these metamorphic variants. Therefore, detection of these kind of viruses becomes an increasingly important task. Recently, many researchers have focused on extracting common patterns of metamorphic variants that can be used as micro-signatures to identify the metamorphic malware executables. With the similar motivation, in this work, we propose a novel metamorphic malware identification method, named HLES-MMI (Higher-level Engine Signature based Metamorphic Malware Identification). The proposed method firstly constructs a unique graph structure, called as co-opcode graph, for each metamorphic family, then extracts engine-specific opcode patterns from the graphs. Finally, it generates higher-level signature belonging to each family by representing the extracted opcode-patterns with a binary vector. Experimental results on four datasets produced by different morphing engines demonstrate the effectiveness and efficiency of the proposed method by comparing with several existing malware identification methods.     

Hybrids of support vector machine wrapper and filter based framework for malware detection

Malware replicates itself and produces offspring with the same characteristics but different signatures by using code obfuscation techniques. Current generation Anti-Virus (AV) engines employ a signature-template type detection approach where malware can easily evade existing signatures in the database. This reduces the capability of current AV engines in detecting malware. In this paper we propose a hybrid framework for malware detection by using the hybrids of Support Vector Machines Wrapper, Maximum-Relevance–Minimum-Redundancy Filter heuristics where Application Program Interface (API) call statistics are used as a malware features. The novelty of our hybrid framework is that it injects the filter’s ranking score in the wrapper selection process and combines the properties of both wrapper and filters and API call statistics which can detect malware based on the nature of infectious actions instead of signature. To the best of our knowledge, this kind of hybrid approach has not been explored yet in the literature in the context of feature selection and malware detection. Knowledge about the intrinsic characteristics of malicious activities is determined by the API call statistics which is injected as a filter score into the wrapper’s backward elimination process in order to find the most significant APIs. While using the most significant APIs in the wrapper classification on both obfuscated and benign types malware datasets, the results show that the proposed hybrid framework clearly surpasses the existing models including the independent filters and wrappers using only a very compact set of significant APIs. The performances of the proposed and existing models have further been compared using binary logistic regression. Various goodness of fit comparison criteria such as Chi Square, Akaike’s Information Criterion (AIC) and Receiver Operating Characteristic Curve ROC are deployed to identify the best performing models. Experimental outcomes based on the above criteria also show that the proposed hybrid framework outperforms other existing models of signature types including independent wrapper and filter approaches to identify malware.     

Windows malware detection system based on LSVC recommended hybrid features

To combat exponentially evolved modern malware, an effective Malware Detection System and precise malware classification is highly essential. In this paper, the Linear Support Vector Classification (LSVC) recommended Hybrid Features based Malware Detection System (HF-MDS) has been proposed. It uses a combination of the static and dynamic features of the Portable Executable (PE) files as hybrid features to identify unknown malware. The application program interface calls invoked by the PE files during their execution along with their correspondent category are collected and considered as dynamic features from the PE file behavioural report produced by the Cuckoo Sandbox. The PE files’ header details such as optional header, disk operating system header, and file header are treated as static features. The LSVC is used as a feature selector to choose prominent static and dynamic features from their respective Original Feature Space. The features recommended by the LSVC are highly discriminative and used as final features for the classification process. Different sets of experiments were conducted using real-world malware samples to verify the combination of static and dynamic features, which encourage the classifier to attain high accuracy. The tenfold cross-validation experimental results demonstrate that the proposed HF-MDS is proficient in precisely detecting malware and benign PE files by attaining detection accuracy of 99.743% with sequential minimal optimization classifier consisting of hybrid features.

     

HDM-Analyser: a hybrid analysis approach based on data mining techniques for malware detection

Today’s security threats like malware are more sophisticated and targeted than ever, and they are growing at an unprecedented rate. To deal with them, various approaches are introduced. One of them is Signature-based detection, which is an effective method and widely used to detect malware; however, there is a substantial problem in detecting new instances. In other words, it is solely useful for the second malware attack. Due to the rapid proliferation of malware and the desperate need for human effort to extract some kinds of signature, this approach is a tedious solution; thus, an intelligent malware detection system is required to deal with new malware threats. Most of intelligent detection systems utilise some data mining techniques in order to distinguish malware from sane programs. One of the pivotal phases of these systems is extracting features from malware samples and benign ones in order to make at least a learning model. This phase is called “Malware Analysis” which plays a significant role in these systems. Since API call sequence is an effective feature for realising unknown malware, this paper is focused on extracting this feature from executable files. There are two major kinds of approach to analyse an executable file. The first type of analysis is “Static Analysis” which analyses a program in source code level. The second one is “Dynamic Analysis” that extracts features by observing program’s activities such as system requests during its execution time. Static analysis has to traverse the program’s execution path in order to find called APIs. Because it does not have sufficient information about decision making points in the given executable file, it is not able to extract the real sequence of called APIs. Although dynamic analysis does not have this drawback, it suffers from execution overhead. Thus, the feature extraction phase takes noticeable time. In this paper, a novel hybrid approach, HDM-Analyser, is presented which takes advantages of dynamic and static analysis methods for rising speed while preserving the accuracy in a reasonable level. HDM-Analyser is able to predict the majority of decision making points by utilising the statistical information which is gathered by dynamic analysis; therefore, there is no execution overhead. The main contribution of this paper is taking accuracy advantage of the dynamic analysis and incorporating it into static analysis in order to augment the accuracy of static analysis. In fact, the execution overhead has been tolerated in learning phase; thus, it does not impose on feature extraction phase which is performed in scanning operation. The experimental results demonstrate that HDM-Analyser attains better overall accuracy and time complexity than static and dynamic analysis methods.     

Malware classification based on call graph clustering

Each day, anti-virus companies receive tens of thousands samples of potentially harmful executables. Many of the malicious samples are variations of previously encountered malware, created by their authors to evade pattern-based detection. Dealing with these large amounts of data requires robust, automatic detection approaches. This paper studies malware classification based on call graph clustering. By representing malware samples as call graphs, it is possible to abstract certain variations away, enabling the detection of structural similarities between samples. The ability to cluster similar samples together will make more generic detection techniques possible, thereby targeting the commonalities of the samples within a cluster. To compare call graphs mutually, we compute pairwise graph similarity scores via graph matchings which approximately minimize the graph edit distance. Next, to facilitate the discovery of similar malware samples, we employ several clustering algorithms, including k-medoids and Density-Based Spatial Clustering of Applications with Noise (DBSCAN). Clustering experiments are conducted on a collection of real malware samples, and the results are evaluated against manual classifications provided by human malware analysts. Experiments show that it is indeed possible to accurately detect malware families via call graph clustering. We anticipate that in the future, call graphs can be used to analyse the emergence of new malware families, and ultimately to automate implementation of generic detection schemes.