A new machine learning-based method for android malware detection on imbalanced dataset

Nowadays, malware applications are dangerous threats to Android devices, users, developers, and application stores. Researchers are trying to discover new methods for malware detection because the complexity of malwares, their continuous changes, and damages caused by their attacks have increased. One of the most important challenges in detecting malware is to have a balanced dataset. In this paper, a detection method is proposed to identify malware to improve accuracy and reduce error rates by preprocessing the used dataset and balancing it. To attain these purposes, the static analysis is used to extract features of the applications. The ranking methods of features are used to preprocess the feature set and the low-effective features are removed. The proposed method also balances the dataset by using the techniques of undersampling, the Synthetic Minority Oversampling Technique (SMOTE), and a combination of both methods, which have not yet been studied among detection methods. Then, the classifiers of K-Nearest Neighbor (KNN), Support Vector Machine, and Iterative Dichotomiser 3 are used to create the detection model. The performance of KNN with SMOTE is better than the performance of the other classifiers. The obtained results indicate that the criteria of precision, recall, accuracy, F-measure, and Matthews Correlation Coefficient are over 97%. The proposed method is effective in detecting 99.49% of the malware’s existing in the used dataset and new malware.

     

A New Hybrid Machine Learning for Cybersecurity Threat Detection Based on Adaptive Boosting

A hybrid machine learning is a combination of multiple types of machine learning algorithms for improving the performance of single classifiers. Currently, cyber intrusion detection systems require high-performance methods for classifications because attackers can develop invasive methods and evade the detection tools. In this paper, the cyber intrusion detection architecture based on new hybrid machine learning is proposed for multiple cyber intrusion detection. In addition, the correlation-based feature selection is adopted for reducing the irrelevant features and the weight vote of adaptive boosting that is adopted to combine multiple classifiers is concentrated. In the experiments, UNB-CICT or network traffic dataset is used for evaluating the performance of the proposed method. The results show that the proposed method can achieve higher efficiency in every attack type detection. Furthermore, the experiments with Phishing website dataset UNSW-NB 15 dataset NSL-KDD dataset and KDD Cup’99 dataset are also conducted, and the results show that the proposed method can produce higher efficiency as well.     

Discovering optimal features using static analysis and a genetic search based method for Android malware detection

Mobile device manufacturers are rapidly producing miscellaneous Android versions worldwide. Simultaneously, cyber criminals are executing malicious actions, such as tracking user activities, stealing personal data, and committing bank fraud. These criminals gain numerous benefits as too many people use Android for their daily routines, including important communi-cations. With this in mind, security practitioners have conducted static and dynamic analyses to identify malware. This study used static analysis because of its overall code coverage, low resource consumption, and rapid processing. However, static analysis requires a minimum number of features to efficiently classify malware. Therefore, we used genetic search (GS), which is a search based on a genetic algorithm (GA), to select the features among 106 strings. To evaluate the best features determined by GS, we used five machine learning classifiers, namely, Naïve Bayes (NB), functional trees (FT), J48, random forest (RF), and multilayer perceptron (MLP). Among these classifiers, FT gave the highest accuracy (95%) and true positive rate (TPR) (96.7%) with the use of only six features.     

基于数据流深度学习算法的Android恶意应用检测方法

目前针对未知的Android恶意应用可以采用机器学习算法进行检测,但传统的机器学习算法具有少于三层的计算单元,无法充分挖掘Android应用程序特征深层次的表达。文中首次提出了一种基于深度学习的算法DDBN （Data-flow Deep BeliefNetwork）对Android应用程序数据流特征进行分析,从而检测Android未知恶意应用。首先,使用分析工具FlowDroid和SUSI提取能够反映Android应用恶意行为的静态数据流特征;然后,针对该特征设计了数据流深度学习算法DDBN,该算法通过构建深层的模型结构,并进行逐层特征变换,将数据流在原空间的特征表示变换到新的特征空间,从而使分类更加准确;最后,基于DDBN实现了Android恶意应用检测工具Flowdect,并对现实中的大量安全应用和恶意应用进行检测。实验结果表明,Flowdect能够充分学习Android应用程序的数据流特征,用于检测未知的Android恶意应用。通过与其他基于传统机器学习算法的检测方案对比,DDBN算法具有更优的检测效果。     

HEMD: a highly efficient random forest-based malware detection framework for Android

Mobile phones are rapidly becoming the most widespread and popular form of communication; thus, they are also the most important attack target of malware. The amount of malware in mobile phones is increasing exponentially and poses a serious security threat. Google’s Android is the most popular smart phone platforms in the world and the mechanisms of permission declaration access control cannot identify the malware. In this paper, we proposed an ensemble machine learning system for the detection of malware on Android devices. More specifically, four groups of features including permissions, monitoring system events, sensitive API and permission rate are extracted to characterize each Android application (app). Then an ensemble random forest classifier is learned to detect whether an app is potentially malicious or not. The performance of our proposed method is evaluated on the actual data set using tenfold cross-validation. The experimental results demonstrate that the proposed method can achieve a highly accuracy of 89.91%. For further assessing the performance of our method, we compared it with the state-of-the-art support vector machine classifier. Comparison results demonstrate that the proposed method is extremely promising and could provide a cost-effective alternative for Android malware detection.

     

“Andromaly”: a behavioral malware detection framework for android devices

This article presents Andromaly—a framework for detecting malware on Android mobile devices. The proposed framework realizes a Host-based Malware Detection System that continuously monitors various features and events obtained from the mobile device and then applies Machine Learning anomaly detectors to classify the collected data as normal (benign) or abnormal (malicious). Since no malicious applications are yet available for Android, we developed four malicious applications, and evaluated Andromaly’s ability to detect new malware based on samples of known malware. We evaluated several combinations of anomaly detection algorithms, feature selection method and the number of top features in order to find the combination that yields the best performance in detecting new malware on Android. Empirical results suggest that the proposed framework is effective in detecting malware on mobile devices in general and on Android in particular.     

Bio-inspired computational paradigm for feature investigation and malware detection: interactive analytics

Recently, people rely on mobile devices to conduct their daily fundamental activities. Simultaneously, most of the people prefer devices with Android operating system. As the demand expands, deceitful authors develop malware to compromise Android for private and money purposes. Consequently, security analysts have to conduct static and dynamic analyses to counter malware violation. In this paper, we adopt static analysis which only requests minimal resource consumption and rapid processing. However, finding a minimum set of features in the static analysis are vital because it removes irrelevant data, reduces the runtime of machine learning detection and reduces the dimensionality of datasets. Therefore, in this paper, we investigate three categories of features, which are permissions, directory path, and telephony. This investigation considers the features frequency as well as repeatedly used in each application. Subsequently, this study evaluates the proposed features in three bio-inspired machine learning classifiers in artificial neural network (ANN) category to signify the usefulness of ANN type in uncovering unknown malware. The classifiers are multilayer perceptron (MLP), voted perceptron (VP) and radial basis function network (RBFN). Among all these three classifiers, the outstanding outcomes acquire is the MLP, which achieves 90% in accuracy and 87% in true positive rate (TPR), as well as 97% accuracy in our Bio Analyzer prediction system.     

基于BPSO-NB算法的Android恶意应用检测方法

为了提高Android恶意应用检测效率,将二值粒子群算法(BPSO,Binary Particle Swarm Optimization)用于原始特征全集的优化选择,并结合朴素贝叶斯(NB,Nave Bayesian)分类算法,提出一种基于BPSO-NB的Android恶意应用检测方法。该方法首先对未知应用进行静态分析,提取AndroidManifest.xml文件中的权限信息作为特征。然后,采用BPSO算法优化选择分类特征,并使用NB算法的分类精度作为评价函数。最后采用NB分类算法构建Android恶意应用分类器。实验结果表明,通过二值粒子群优化选择分类特征可以有效提高分类精度,缩短检测时间。      

基于深度信念网络的Android恶意应用检测方法

传统的机器学习算法无法有效地从海量的行为特征中选择出有本质的行为特征来对未知的Android恶意应用进行检测。为了解决这个问题,提出DBNSel,一种基于深度信念网络模型的Android恶意应用检测方法。为了实现该方法,首先通过静态分析方法从Android应用中提取5类不同的属性。其次,建立深度信念网络模型从提取到的属性中进行选择和学习。最后,使用学习到的属性来对未知类型的Android恶意应用进行检测。在实验阶段,使用一个由3 986个Android正常应用和3 986个Android恶意应用组成的数据集来验证DBNSel的有效性。实验结果表明,DBNSel的检测结果要优于其他几种已有的检测方法,并可以达到99.4%的检测准确率。此外,DBNSel具有较低的运行开销,可以适应于更大规模的真实环境下的Android恶意应用检测。     

面向安卓恶意软件检测的对抗攻击技术综述

随着对Android恶意软件检测精度和性能要求的提高,越来越多的Android恶意软件检测引擎使用人工智能算法.与此同时,攻击者开始尝试对Android恶意软件进行一定的修改,使得Android恶意软件可以在保留本身的功能的前提下绕过这些基于人工智能算法的检测.上述过程即是Android恶意软件检测领域的对抗攻击.本文...     

Investigation of Android Malware with Machine Learning Classifiers using Enhanced PCA Algorithm

Android devices are popularly available in the commercial market at different price levels for various levels of customers. The Android stack is more vulnerable compared to other platforms because of its open-source nature. There are many android malware detection techniques available to exploit the source code and find associated components during execution time. To obtain a better result we create a hybrid technique merging static and dynamic processes. In this paper, in the first part, we have proposed a technique to check for correlation between features and classify using a supervised learning approach to avoid Multicollinearity problem is one of the drawbacks in the existing system. In the proposed work, a novel PCA (Principal Component Analysis) based feature reduction technique is implemented with conditional dependency features by gathering the functionalities of the application which adds novelty for the given approach. The Android Sensitive Permission is one major key point to be considered while detecting malware. We select vulnerable columns based on features like sensitive permissions, application program interface calls, services requested through the kernel, and the relationship between the variables henceforth build the model using machine learning classifiers and identify whether the given application is malicious or benign. The final goal of this paper is to check benchmarking datasets collected from various repositories like virus share, Github, and the Canadian Institute of cyber security, compare with models ensuring zero-day exploits can be monitored and detected with better accuracy rate.     

Empirical assessment of machine learning-based malware detectors for Android

To address the issue of malware detection through large sets of applications, researchers have recently started to investigate the capabilities of machine-learning techniques for proposing effective approaches. So far, several promising results were recorded in the literature, many approaches being assessed with what we call in the lab validation scenarios. This paper revisits the purpose of malware detection to discuss whether such in the lab validation scenarios provide reliable indications on the performance of malware detectors in real-world settings, aka in the wild. To this end, we have devised several Machine Learning classifiers that rely on a set of features built from applications’ CFGs. We use a sizeable dataset of over 50 000 Android applications collected from sources where state-of-the art approaches have selected their data. We show that, in the lab, our approach outperforms existing machine learning-based approaches. However, this high performance does not translate in high performance in the wild. The performance gap we observed—F-measures dropping from over 0.9 in the lab to below 0.1 in the wild—raises one important question: How do state-of-the-art approaches perform in the wild?     

一种基于多特征集成学习的恶意代码静态检测框架

伴随着互联网的普及和5G通信技术的快速发展,网络空间所面临的威胁日益增大,尤其是恶意软件的数量呈指数型上升,其所属家族的变种爆发式增加.传统的基于人工签名的恶意软件的检测方式速度太慢,难以处理每天数百万计新增的恶意软件,而普通的机器学习分类器的误报率和漏检率又明显过高.同时恶意软件的加壳、混淆等对抗技术对该情况造成了更大的困扰.基于此,提出一种基于多特征集成学习的恶意软件静态检测框架.通过提取恶意软件的非PE(Portable Executable)结构特征、可见字符串与汇编码序列特征、PE结构特征以及函数调用关系5部分特征,构建与各部分特征相匹配的模型,采用Bagging集成和Stacking集成算法,提升模型的稳定性,降低过拟合的风险.然后采取权重策略投票算法对5部分集成模型的输出结果做进一步聚合.经过测试,多特征多模型聚合的检测准确率可达96.99%,该结果表明:与其他静态检测方法相比,该方法具有更好的恶意软件鉴别能力,对加壳、混淆等恶意软件同样具备较高的识别率.     

基于机器学习算法的Android恶意程序检测系统

对于传统的恶意程序检测方法存在的缺点,针对将数据挖掘和机器学习算法被应用在未知恶意程序的检测方法进行研究。当前使用单一特征的机器学习算法无法充分发挥其数据处理能力,检测效果不佳。文中将语音识别模型与随机森林算法相结合,首次提出了综和APK文件多类特征统一建立N-gram模型,并应用随机森林算法用于未知恶意程序检测。首先,采用多种方式提取可以反映Android恶意程序行为的3类特征,包括敏感权限、DVM函数调用序列以及OpCodes特征;然后,针对每类特征建立N-gram模型,每个模型可以独立评判恶意程序行为;最后,3类特征模型统一加入随机森林算法进行学习,从而对Android程序进行检测。基于该方法实现了Android恶意程序检测系统,并对811个非恶意程序及826个恶意程序进行检测,准确率较高。综合各个评价指标,与其他相关工作对比,实验结果表明该系统在恶意程序检测准确率和有效性上表现更优。     

安卓恶意软件的静态检测方法

近几年,Android平台的恶意软件数量几乎以几何式的速度增长,故提出一种恶意软件检测方法是必要的.本文利用现如今疯涨的Android恶意样本量和机器学习算法建立分类预测模型实现对恶意软件的静态检测.首先,通过反编译APK文件获取AndroidManifest.xml文件中权限特征,baksmali工具反编译class.dex成smali文件得到危险API特征.然后运用机器学习中多种分类和预处理算法比较每一特征和联合特征检测的准确率.实验结果表明,联合特征检测准确率高于单独特征,准确率达到97.5%.     

Android恶意软件的多特征协作决策检测方法

由于智能手机使用率持续上升促使移动恶意软件在规模和复杂性方面发展更加迅速。作为免费和开源的系统,目前Android已经超越其他移动平台成为最流行的操作系统,使得针对Android平台的恶意软件数量也显著增加。针对Android平台应用软件安全问题,提出了一种基于多特征协作决策的Android恶意软件检测方法,该方法主要通过对Android 应用程序进行分析、提取特征属性以及根据机器学习模型和分类算法判断其是否为恶意软件。通过实验表明,使用该方法对Android应用软件数据集进行分类后,相比其他分类器或算法分类的结果,其各项评估指标均大幅提高。因此,提出的基于多特征协作决策的方式来对Android恶意软件进行检测的方法可以有效地用于对未知应用的恶意性进行检测,避免恶意应用对用户所造成的损害等。     

Android平台恶意应用程序静态检测方法

本文构建的静态检测系统主要用于检测Android平台未知恶意应用程序.首先,对待检测应用程序进行预处理,从Android Manifest.xml文件中提取权限申请信息作为一类特征属性;如待检测应用程序存在动态共享库,则提取从第三方调用的函数名作为另一类特征属性.对选取的两类特征属性分别选择最优分类算法,最后根据上述的两个最优分类算法对待检测应用程序的分类结果判定待检测应用程序是否为恶意应用程序.实验结果表明:该静态检测系统能够有效地检测出Android未知恶意应用程序,准确率达到95.4%,具有良好的应用前景.     

基于集成学习投票算法的Android恶意应用检测

针对Android平台恶意应用的检测技术,提出一种基于集成学习投票算法的Android恶意程序检测方法MASV（Soft-Voting Algorithm）,以有效地对未知应用程序进行分类。从已知开源的数据集中获取了实验的基础数据,使用的应用程序集包含213 256个良性应用程序以及18 363个恶意应用程序。使用SVM-RFE特征选择算法对特征进行降维。使用多个分类器的集合,即SVM（Support Vector Machine）、[K]-NN[（K]-Nearest Neighbor）、NB（Na?ve Bayes）、CART（Classification and Regression Tree）和RF（Random Forest）,以检测恶意应用程序和良性应用程序。使用梯度上升算法确定集成学习软投票的基分类器权重参数。实验结果表明,该方法在恶意应用程序检测中达到了99.27%的准确率。     

一种基于组合事件行为触发的Android恶意行为检测方法

当前Android恶意应用程序在传播环节缺乏有效的识别手段,对此提出了一种基于自动化测试技术和动态分析技术的Android恶意行为检测方法。 通过自动化测试技术触发Android应用程序的行为,同时构建虚拟的沙箱监控这些行为。设计了一种组合事件行为触发模型——DroidRunner,提高了Android应用程序的代码覆盖率、恶意行为的触发率以及Android恶意应用的检测率。经过实际部署测试,该方法对未知恶意应用具有较高的检测率,能帮助用户发现和分析未知恶意应用。     

Intelligent OS X malware threat detection with code inspection

With the increasing market share of Mac OS X operating system, there is a corresponding increase in the number of malicious programs (malware) designed to exploit vulnerabilities on Mac OS X platforms. However, existing manual and heuristic OS X malware detection techniques are not capable of coping with such a high rate of malware. While machine learning techniques offer promising results in automated detection of Windows and Android malware, there have been limited efforts in extending them to OS X malware detection. In this paper, we propose a supervised machine learning model. The model applies kernel base Support Vector Machine and a novel weighting measure based on application library calls to detect OS X malware. For training and evaluating the model, a dataset with a combination of 152 malware and 450 benign were created. Using common supervised Machine Learning algorithm on the dataset, we obtain over 91% detection accuracy with 3.9% false alarm rate. We also utilize Synthetic Minority Over-sampling Technique (SMOTE) to create three synthetic datasets with different distributions based on the refined version of collected dataset to investigate impact of different sample sizes on accuracy of malware detection. Using SMOTE datasets we could achieve over 96% detection accuracy and false alarm of less than 4%. All malware classification experiments are tested using cross validation technique. Our results reflect that increasing sample size in synthetic datasets has direct positive effect on detection accuracy while increases false alarm rate in compare to the original dataset.