基于统计学习的挂马网页实时检测

近年来挂马网页对Web安全造成严重威胁,客户端的主要防御手段包括反病毒软件与恶意站点黑名单。反病毒软件采用特征码匹配方法,无法有效检测经过加密与混淆变形的网页脚本代码;黑名单无法防御最新出现的恶意站点。提出一种新型的、与网页内容代码无关的挂马网页实时检测方法。该方法主要提取访问网页时HTTP会话过程的各种统计特征,利用决策树机器学习方法构建挂马网页分类模型并用于在线实时检测。实验证明,该方法能够达到89. 7%的挂马网页检测率与0. 3%的误检率。     

Novel active learning methods for enhanced PC malware detection in windows OS

The formation of new malwares every day poses a significant challenge to anti-virus vendors since antivirus tools, using manually crafted signatures, are only capable of identifying known malware instances and their relatively similar variants. To identify new and unknown malwares for updating their anti-virus signature repository, anti-virus vendors must daily collect new, suspicious files that need to be analyzed manually by information security experts who then label them as malware or benign. Analyzing suspected files is a time-consuming task and it is impossible to manually analyze all of them. Consequently, anti-virus vendors use machine learning algorithms and heuristics in order to reduce the number of suspect files that must be inspected manually. These techniques, however, lack an essential element – they cannot be daily updated. In this work we introduce a solution for this updatability gap. We present an active learning (AL) framework and introduce two new AL methods that will assist anti-virus vendors to focus their analytical efforts by acquiring those files that are most probably malicious. Those new AL methods are designed and oriented towards new malware acquisition. To test the capability of our methods for acquiring new malwares from a stream of unknown files, we conducted a series of experiments over a ten-day period. A comparison of our methods to existing high performance AL methods and to random selection, which is the naïve method, indicates that the AL methods outperformed random selection for all performance measures. Our AL methods outperformed existing AL method in two respects, both related to the number of new malwares acquired daily, the core measure in this study. First, our best performing AL method, termed “Exploitation”, acquired on the 9th day of the experiment about 2.6 times more malwares than the existing AL method and 7.8 more times than the random selection. Secondly, while the existing AL method showed a decrease in the number of new malwares acquired over 10 days, our AL methods showed an increase and a daily improvement in the number of new malwares acquired. Both results point towards increased efficiency that can possibly assist anti-virus vendors.     

基于Stacking的恶意网页集成检测方法

针对目前主流恶意网页检测技术耗费资源多、检测周期长和分类效果低等问题，提出一种基于Stacking的恶意网页集成检测方法，将异质分类器集成的方法应用在恶意网页检测识别领域。通过对网页特征提取分析相关因素和分类集成学习来得到检测模型，其中初级分类器分别使用K近邻（KNN）算法、逻辑回归算法和决策树算法建立，而次级的元分类器由支持向量机（SVM）算法建立。与传统恶意网页检测手段相比，此方法在资源消耗少、速度快的情况下使识别准确率提高了0.7%，获得了98.12%的高准确率。实验结果表明，所提方法构造的检测模型可高效准确地对恶意网页进行识别。     

基于行为特征分析的微博恶意用户识别

近年来,社交网络数据挖掘作为物理网络空间数据挖掘的一大热点,目前在用户行为分析、兴趣识别、产品推荐等方面都取得了令人可喜的成果。随着社交网络商业契机的到来,出现了很多恶意用户及恶意行为,给数据挖掘的效果产生了极大的影响。基于此,提出基于用户行为特征分析的恶意用户识别方法,该方法引入主成分分析方法对微博网络用户行为数据进行挖掘,对各维度特征的权重进行排序,选取前六维主成分特征可以有效识别恶意用户,主成分特征之间拟合出的新特征也能提升系统的识别性能。实验结果表明,引入的方法对微博用户特征进行了有效的排序,很好地识别出了微博社交网络中的恶意用户,为其他方向的社交网络数据挖掘提供了良好的数据清洗技术。     

Practical real-time intrusion detection using machine learning approaches

The growing prevalence of network attacks is a well-known problem which can impact the availability, confidentiality, and integrity of critical information for both individuals and enterprises. In this paper, we propose a real-time intrusion detection approach using a supervised machine learning technique. Our approach is simple and efficient, and can be used with many machine learning techniques. We applied different well-known machine learning techniques to evaluate the performance of our IDS approach. Our experimental results show that the Decision Tree technique can outperform the other techniques. Therefore, we further developed a real-time intrusion detection system (RT-IDS) using the Decision Tree technique to classify on-line network data as normal or attack data. We also identified 12 essential features of network data which are relevant to detecting network attacks using the information gain as our feature selection criterions. Our RT-IDS can distinguish normal network activities from main attack types (Probe and Denial of Service (DoS)) with a detection rate higher than 98% within 2 s. We also developed a new post-processing procedure to reduce the false-alarm rate as well as increase the reliability and detection accuracy of the intrusion detection system.     

Steels classification by machine learning and Calphad methods

Steels of different classes (austenitic, martensitic, pearlitic, etc.) have different applications and characteristic areas of properties. In the present work two methods are used to predict steel class, based on the composition and heat treatment parameters: the physically-based Calphad method and data-driven machine learning method. They are applied to the same dataset, collected from open sources (mostly steels for high-temperature applications). Classification accuracy of 93.6% is achieved by machine learning model, trained on the concentration of three elements (C, Cr, Ni) and heat treatment parameters (heating temperatures). Calphad method gives 76% accuracy, based on the temperature and cooling rate. The reasons for misclassification by both methods are discussed, and it is shown that the part of them caused by ambiguity/inaccuracy in the data or limitations of the models used. For the rest of cases reasonable classification accuracy is demonstrated. We suggest that the reason of the supremacy of machine learning classifier is the small variation in the data used, which indeed does not change the steel class: the properties of steel should be insensitive to the details of the manufacturing process.     

静动态结合的恶意Android应用自动检测技术

随着移动互联网的快速发展,移动终端及移动应用在人们日常生活中越来越重要,与此同时,恶意移动应用给网络和信息安全带来了严峻的挑战。Android平台由于其开放性和应用市场审查机制不够完善,使其成为了移动互联网时代恶意应用的主要传播平台。现有的恶意应用检测方法主要有静态分析和动态测试两种。一般而言,静态分析方法代码覆盖率高、时间开销小,但存在误报率较高的问题;而动态测试准确度较高,但需要实际运行应用,所需的时间和计算资源开销较大。针对上述情况,本文基于静动态结合的方法,自动检测恶意Android应用。首先,使用静态分析技术获取应用API的调用情况来判定其是否为疑似恶意应用,特别是可有效检测试图通过反射机制调用API躲避静态分析的恶意应用;然后,根据疑似恶意应用UI控件的可疑度进行有针对性的动态测试,来自动确认疑似恶意应用中是否存在恶意行为。基于此方法,我们实现了原型检测工具框架,并针对吸费短信类恶意行为,对由465个恶意应用和1085个正常应用组成的数据集进行了对比实验。实验结果表明,该方法在提高恶意应用检测效率的同时,有效地降低了误报率。     

A hybrid machine learning approach to network anomaly detection

Zero-day cyber attacks such as worms and spy-ware are becoming increasingly widespread and dangerous. The existing signature-based intrusion detection mechanisms are often not sufficient in detecting these types of attacks. As a result, anomaly intrusion detection methods have been developed to cope with such attacks. Among the variety of anomaly detection approaches, the Support Vector Machine (SVM) is known to be one of the best machine learning algorithms to classify abnormal behaviors. The soft-margin SVM is one of the well-known basic SVM methods using supervised learning. However, it is not appropriate to use the soft-margin SVM method for detecting novel attacks in Internet traffic since it requires pre-acquired learning information for supervised learning procedure. Such pre-acquired learning information is divided into normal and attack traffic with labels separately. Furthermore, we apply the one-class SVM approach using unsupervised learning for detecting anomalies. This means one-class SVM does not require the labeled information. However, there is downside to using one-class SVM: it is difficult to use the one-class SVM in the real world, due to its high false positive rate. In this paper, we propose a new SVM approach, named Enhanced SVM, which combines these two methods in order to provide unsupervised learning and low false alarm capability, similar to that of a supervised SVM approach.We use the following additional techniques to improve the performance of the proposed approach (referred to as Anomaly Detector using Enhanced SVM): First, we create a profile of normal packets using Self-Organized Feature Map (SOFM), for SVM learning without pre-existing knowledge. Second, we use a packet filtering scheme based on Passive TCP/IP Fingerprinting (PTF), in order to reject incomplete network traffic that either violates the TCP/IP standard or generation policy inside of well-known platforms. Third, a feature selection technique using a Genetic Algorithm (GA) is used for extracting optimized information from raw internet packets. Fourth, we use the flow of packets based on temporal relationships during data preprocessing, for considering the temporal relationships among the inputs used in SVM learning. Lastly, we demonstrate the effectiveness of the Enhanced SVM approach using the above-mentioned techniques, such as SOFM, PTF, and GA on MIT Lincoln Lab datasets, and a live dataset captured from a real network. The experimental results are verified by m-fold cross validation, and the proposed approach is compared with real world Network Intrusion Detection Systems (NIDS).     

一种恶意代码变种检测的有效方法

为了改变基于特征码病毒查杀存在的滞后性,以及对于恶意代码变种的无效性,提出了一种基于支持向量机和模糊推理技术的恶意代码及其变种的检测方法。基于Radux原型系统,通过使用多分类机,将恶意程序进一步细分为病毒、蠕虫和木马程序,然后进行恶意代码判定的模糊推理,使得未知病毒的检测概率进一步提升,对于已有恶意程序的检测率高达99.03%,对于恶意程序变种的检测率达到93.38%。     

显示器电磁信息泄漏的机器学习检测方法研究

根据电磁学原理,在操作电子信息设备的过程中会产生无意的电磁辐射.电磁辐射会引发信息泄漏,给信息安全造成严重威胁.面向计算机显示器的电磁信息安全问题,提出基于机器学习的电磁信息泄漏检测方法.针对电磁泄漏信号的特点,设计了MGCNN卷积神经网络.利用其独特的卷积和池化处理能力,提取显示器电磁频谱信号中图像信息的多层次特征,...     

A taxonomy and survey of attacks against machine learning

The majority of machine learning methodologies operate with the assumption that their environment is benign. However, this assumption does not always hold, as it is often advantageous to adversaries to maliciously modify the training (poisoning attacks) or test data (evasion attacks). Such attacks can be catastrophic given the growth and the penetration of machine learning applications in society. Therefore, there is a need to secure machine learning enabling the safe adoption of it in adversarial cases, such as spam filtering, malware detection, and biometric recognition. This paper presents a taxonomy and survey of attacks against systems that use machine learning. It organizes the body of knowledge in adversarial machine learning so as to identify the aspects where researchers from different fields can contribute to. The taxonomy identifies attacks which share key characteristics and as such can potentially be addressed by the same defence approaches. Thus, the proposed taxonomy makes it easier to understand the existing attack landscape towards developing defence mechanisms, which are not investigated in this survey. The taxonomy is also leveraged to identify open problems that can lead to new research areas within the field of adversarial machine learning.     

A comprehensive survey on regularization strategies in machine learning

In machine learning, the model is not as complicated as possible. Good generalization ability means that the model not only performs well on the training data set, but also can make good prediction on new data. Regularization imposes a penalty on model’s complexity or smoothness, allowing for good generalization to unseen data even when training on a finite training set or with an inadequate iteration. Deep learning has developed rapidly in recent years. Then the regularization has a broader definition: regularization is a technology aimed at improving the generalization ability of a model. This paper gave a comprehensive study and a state-of-the-art review of the regularization strategies in machine learning. Then the characteristics and comparisons of regularizations were presented. In addition, it discussed how to choose a regularization for the specific task. For specific tasks, it is necessary for regularization technology to have good mathematical characteristics. Meanwhile, new regularization techniques can be constructed by extending and combining existing regularization techniques. Finally, it concluded current opportunities and challenges of regularization technologies, as well as many open concerns and research trends.     

基于多通道图像深度学习的恶意代码检测

现有基于深度学习的恶意代码检测方法存在深层次特征提取能力偏弱、模型相对复杂、模型泛化能力不足等问题。同时,代码复用现象在同一类恶意样本中大量存在,而代码复用会导致代码的视觉特征相似,这种相似性可以被用来进行恶意代码检测。因此,提出一种基于多通道图像视觉特征和AlexNet神经网络的恶意代码检测方法。该方法首先将待检测的代码转化为多通道图像,然后利用AlexNet神经网络提取其彩色纹理特征并对这些特征进行分类从而检测出可能的恶意代码;同时通过综合运用多通道图像特征提取、局部响应归一化（LRN）等技术,在有效降低模型复杂度的基础上提升了模型的泛化能力。利用均衡处理后的Malimg数据集进行测试,结果显示该方法的平均分类准确率达到97.8%;相较于VGGNet方法在准确率上提升了1.8%,在检测效率上提升了60.2%。实验结果表明,多通道图像彩色纹理特征能较好地反映恶意代码的类别信息,AlexNet神经网络相对简单的结构能有效地提升检测效率,而局部响应归一化能提升模型的泛化能力与检测效果。     

Linking software testing results with a machine learning approach

Software testing techniques and criteria are considered complementary since they can reveal different kinds of faults and test distinct aspects of the program. The functional criteria, such as Category Partition, are difficult to be automated and are usually manually applied. Structural and fault-based criteria generally provide measures to evaluate test sets. The existing supporting tools produce a lot of information including: input and produced output, structural coverage, mutation score, faults revealed, etc. However, such information is not linked to functional aspects of the software. In this work, we present an approach based on machine learning techniques to link test results from the application of different testing techniques. The approach groups test data into similar functional clusters. After this, according to the tester's goals, it generates classifiers (rules) that have different uses, including selection and prioritization of test cases. The paper also presents results from experimental evaluations and illustrates such uses.     

一种基于组合事件行为触发的Android恶意行为检测方法

当前Android恶意应用程序在传播环节缺乏有效的识别手段,对此提出了一种基于自动化测试技术和动态分析技术的Android恶意行为检测方法。 通过自动化测试技术触发Android应用程序的行为,同时构建虚拟的沙箱监控这些行为。设计了一种组合事件行为触发模型——DroidRunner,提高了Android应用程序的代码覆盖率、恶意行为的触发率以及Android恶意应用的检测率。经过实际部署测试,该方法对未知恶意应用具有较高的检测率,能帮助用户发现和分析未知恶意应用。     

基于CNN的恶意Web请求检测技术

目前,基于卷积神经网络的Web恶意请求检测技术领域内只有针对URL部分进行恶意检测的研究,并且各研究对原始数据的数字化表示方法不同,这会造成检测效率和检测准确率较低。为提高卷积神经网络在Web恶意请求检测领域的性能,在现有工作的基础上将其他多个HTTP请求参数与URL合并,将数据集HTTP data set CSIC 2010和DEV_ACCESS作为原始数据,设计对比实验。首先采用6种数据数字向量化方法对字符串格式的原始输入进行处理;然后将其分别输入所设计的卷积神经网络,训练后可得到6个不同的模型,同时使用相同的训练数据集对经典算法HMM,SVM和RNN进行训练,得到对照组模型;最后在同一验证集上对9个模型进行评估。实验结果表明,采用多参数的Web恶意请求检测方法将词汇表映射与卷积神经网络内部嵌入层相结合对原始数据进行表示,可使卷积神经网络取得99.87%的准确率和98.92%的F1值。相比其他8个模型,所提方法在准确率上提升了0.4～7.7个百分点,在F1值上提升了0.3～13个百分点。实验充分说明,基于卷积神经网络的多参数Web恶意请求检测技术具有明显的优势,...     

Comparison of machine learning methods for classifying aphasic and non-aphasic speakers

The performance of eight machine learning classifiers were compared with three aphasia related classification problems. The first problem contained naming data of aphasic and non-aphasic speakers tested with the Philadelphia Naming Test. The second problem included the naming data of Alzheimer and vascular disease patients tested with Finnish version of the Boston Naming Test. The third problem included aphasia test data of patients suffering from four different aphasic syndromes tested with the Aachen Aphasia Test. The first two data sets were small. Therefore, the data used in the tests were artificially generated from the original confrontation naming data of 23 and 22 subjects, respectively. The third set contained aphasia test data of 146 aphasic speakers and was used as such in the experiments. With the first and the third data set the classifiers could successfully be used for the task, while the results with the second data set were less encouraging. However, based on the results, no single classifier performed exceptionally well with all data sets, suggesting that the selection of the classifier used for classification of aphasic data should be based on the experiments performed with the data set at hand.     

A comparison of machine learning techniques for customer churn prediction

We present a comparative study on the most popular machine learning methods applied to the challenging problem of customer churning prediction in the telecommunications industry. In the first phase of our experiments, all models were applied and evaluated using cross-validation on a popular, public domain dataset. In the second phase, the performance improvement offered by boosting was studied. In order to determine the most efficient parameter combinations we performed a series of Monte Carlo simulations for each method and for a wide range of parameters. Our results demonstrate clear superiority of the boosted versions of the models against the plain (non-boosted) versions. The best overall classifier was the SVM-POLY using AdaBoost with accuracy of almost 97% and F-measure over 84%.     

基于极限学习机的网页分类应用

极限学习机ELM不同于传统的神经网络学习算法（如BP算法）,是一种高效的单隐层前馈神经网络（SLFNs）学习算法。将极限学习机引入到中文网页分类任务中。对中文网页进行预处理,提取其特性信息,从而形成网页特征树,产生定长编码作为极限学习机的输入数据。实验结果表明该方法能够有效地分类网页。