Detecting P2P botnets by discovering flow dependency in C&C traffic

Botnets are widely used by attackers and they have evolved from centralized structures to distributed structures. Most of the modern P2P bots launch attacks in a stealthy way and the detection approaches based on the malicious traffic of bots are inefficient. In this paper, an approach that aims to detect Peer-to-Peer (P2P) botnets is proposed. Unlike previous works, the approach is independent of any malicious traffic generated by bots and does not require bots’ information provided by external systems. It detects P2P bots by focusing on the instinct characteristics of their Command and Control (C&C) communications, which are identified by discovering flow dependencies in C&C traffic. After discovering the flow dependencies, our approach distinguishes P2P bots and normal hosts by clustering technique. Experimental results on real-world network traces merged with synthetic P2P botnet traces indicate that 1) flow dependency can be used to detect P2P botnets, and 2) the proposed approach can detect P2P botnets with a high detection rate and a low false positive rate.     

BiLSTM在JavaScript恶意代码检测中的应用

传统的机器学习方法在检测JavaScript恶意代码时,存在提取特征过程复杂、计算量大、代码被恶意混淆导致难以检测的问题,不利于当前JavaScript恶意代码检测准确性和实时性的要求.基于此,提出一种基于双向长短时神经网络(BiLSTM)的JavaScript恶意代码检测方法.首先,将得到的样本数据经过代码反混淆,数据分词,代码向量化后得到适应于神经网络输入的标准化数据.其次,利用BiLSTM算法对向量化数据进行训练,学习JavaScript恶意代码的抽象特征.最后,利用学习到的特征对代码进行分类.将本文方法与深度学习方法和主流机器学习方法进行比较,结果表明该方法具有较高的准确率和较低的误报率.     

Decision tree based light weight intrusion detection using a wrapper approach

The objective of this paper is to construct a lightweight Intrusion Detection System (IDS) aimed at detecting anomalies in networks. The crucial part of building lightweight IDS depends on preprocessing of network data, identifying important features and in the design of efficient learning algorithm that classify normal and anomalous patterns. Therefore in this work, the design of IDS is investigated from these three perspectives. The goals of this paper are (i) removing redundant instances that causes the learning algorithm to be unbiased (ii) identifying suitable subset of features by employing a wrapper based feature selection algorithm (iii) realizing proposed IDS with neurotree to achieve better detection accuracy. The lightweight IDS has been developed by using a wrapper based feature selection algorithm that maximizes the specificity and sensitivity of the IDS as well as by employing a neural ensemble decision tree iterative procedure to evolve optimal features. An extensive experimental evaluation of the proposed approach with a family of six decision tree classifiers namely Decision Stump, C4.5, Naive Baye’s Tree, Random Forest, Random Tree and Representative Tree model to perform the detection of anomalous network pattern has been introduced.     

基于多类特征的JavaScript恶意脚本检测算法*

针对脚本样本集具有混淆、统计、语义等不同层面上的特征,设计基于多类特征的JavaScript恶意脚本检测算法,实现针对恶意JavaScript脚本的离线分析系统JCAD.首先提取脚本的混淆特征,使用C4.5决策树分析被混淆的脚本并解除混淆.然后提取脚本的静态统计特征,根据语义进行脚本序列化,构造危险序列树,提取脚本的危险序列特征.最后以三类特征作为输入,采用对脚本样本集的非均匀性与不断增加的特点具有较强适应能力的概率神经网络构造分类器,判断恶意脚本.实验表明,该算法具有较好的检测准确率与稳定性.     

A novel method of mining network flow to detect P2P botnets

Botnets are a serious threat to cyber-security. As a consequence, botnet detection has become an important research topic in network protection and cyber-crime prevention. P2P botnets are one of the most malicious zombie networks, as their architecture imitates P2P software. Characteristics of P2P botnets include (1) the use of multiple controllers to avoid single-point failure; (2) the use of encryption to evade misuse detection technologies; and (3) the capacity to evade anomaly detection, usually by initiating numerous sessions without consuming substantial bandwidth. To overcome these difficulties, we propose a novel data mining method. First, we identify the differences between P2P botnet behavior and normal network behavior. Then, we use these differences to tune the data-mining parameters to cluster and distinguish normal Internet behavior from that lurking P2P botnets. This method can identify a P2P botnet without breaking the encryption. Furthermore, the detection system can be deployed without altering the existing network architecture, and it can detect the existence of botnets in a complex traffic mix before they attack. The experimental results reveal that the method is effective in recognizing the existence of botnets. Accordingly, the results of this study will be of value to information security academics and practitioners.     

一种基于特征编码技术的恶意代码检测方法

在对恶意代码进行检测和分类时,由于传统的灰度编码方法将特征转换为图像的过程中,会产生特征分裂和精度损失等问题,严重影响了恶意代码的检测性能.同时,传统的恶意代码检测和分类的数据集中只使用了单一的恶意样本,并没有考虑到良性样本.因此,文中采用了一个包含良性样本和恶意样本的数据集,同时提出了一种双字节特征编码方法.首先将待...     

基于神经网络的僵尸网络检测

目前主流的僵尸网络检测方法主要利用网络流量分析技术,这往往需要数据包的内部信息,或者依赖于外部系统提供的信息或僵尸主机的恶意行为,并且大多数方法不能自动存储僵尸网络的流量特征,不具有联想记忆功能.为此提出了一种基于BP神经网络的僵尸网络检测方法,通过大量的僵尸网络和正常流量样本训练BP神经网络分类器,使其学会辨认僵尸网络的流量,自动记忆僵尸流量特征,从而有效检测出被感染的主机.该神经网络分类器以主机对为分析对象,提取2个主机间通信的流量特征,将主机对的特征向量作为输入,有效地区分出正常主机和僵尸主机.实验表明,该方法的检测率达到99%,误报率在1%以下,具有良好的性能.     

基于主成分分析禁忌搜索和决策树分类的异常流量检测方法

真实网络流量包括大量特征属性,现有基于特征分析的异常流量检测方法无法满足高维特征分析要求。提出一种基于主成分分析和禁忌搜索(PCA-TS)的流量特征选择算法结合决策树分类的异常流量检测方法,通过PCA-TS对高维特征进行特征约减和近优特征子集选择,为决策树分类方法提供有效的低维特征属性,结合决策树分类精度和处理效率高的优点,采用半监督学习方式进行异常流量实时检测。实验表明,与传统异常检测方法相比,此方法具有更高的检测精度和更低的误检率,其检测性能受样本规模影响较小,且对未知异常可以进行有效检测     

RP-NBSR: A Novel Network Attack Detection Model Based on Machine Learning

The rapid progress of the Internet has exposed networks to an increased number of threats. Intrusion detection technology can effectively protect network security against malicious attacks. In this paper, we propose a ReliefF-P-Naive Bayes and softmax regression (RP-NBSR) model based on machine learning for network attack detection to improve the false detection rate and F1 score of unknown intrusion behavior. In the proposed model, the Pearson correlation coefficient is introduced to compensate for deficiencies in correlation analysis between features by the ReliefF feature selection algorithm, and a ReliefF-Pearson correlation coefficient (ReliefF-P) algorithm is proposed. Then, the Relief-P algorithm is used to preprocess the UNSW-NB15 dataset to remove irrelevant features and obtain a new feature subset. Finally, naïve Bayes and softmax regression (NBSR) classifier is constructed by cascading the naïve Bayes classifier and softmax regression classifier, and an attack detection model based on RP-NBSR is established. The experimental results on the UNSW-NB15 dataset show that the attack detection model based on RP-NBSR has a lower false detection rate and higher F1 score than other detection models.     

一种基于多移动Agent的P2P网络主动免疫机制

针对对等计算(P2P)环境中日益严峻的恶意代码传播及攻击问题,通过引入多移动Agent技术,提出一种适合P2P网络系统的主动免疫机制.基于多移动Agent的P2P网络主动免疫机制借鉴了生物免疫原理,并利用多Agent技术构建了面向不同功能的Agent,在中枢免疫节点与普通Peer之间、普通Peer与普通Peer之间实现了一种联合防御恶意代码的协作关系;还利用移动Agent技术实现了可以在整个P2P网络环境中漫游、承担主动探测恶意代码功能的Agent和携带免疫疫苗进行远程免疫的Agent,从而实现了对恶意代码的快速响应、分析处理和有效抵御,降低了恶意代码的危害程度.为了高效率地将免疫疫苗分发于网络各节点,还提出一种新的ET+扩散树模型以及基于ET+树的疫苗分发算法.首先分析了P2P网络恶意代码传播模型,然后介绍了基于多移动Agent的P2P网络主动免疫模型的体系结构及组件,以及基于ET+树的免疫疫苗分发算法,最后对算法性能进行了对比仿真验证.     

P2P技术在僵尸网络中的应用研究

与传统集中式僵尸网络相比,P2P僵尸网络鲁棒性更好、拓扑结构更复杂,因此更难防御。针对上述情况,将P2P网络按拓扑结构分为4类,即中心化拓扑、全分布式非结构化拓扑、全分布式结构化拓扑和半分布式拓扑。对4类P2P技术从流量、消息传播速度和网络鲁棒性3个方面进行分析比较和实验验证,并指出以半分布式结构为代表的新型P2P网络具有较好的综合性能,是未来僵尸网络的发展方向之一。     

基于决策树的二维码恶意网址检测方法

二维码技术应用已经进入大众生活,同时也逐渐成为恶意软件传播的新途径。面向二维码中URL,提出二维码恶意网址决策树智能检测方法。利用恶意网址和正规网址,提取网址特征,构建特征向量,进而构建决策树。进一步对网址特征提取及决策树选择进行了优化,实例测试结果表明系统在对恶意网址识别的响应速度和准确率方面取得了良好的效果。     

基于PCA-概率神经网络的P2P流量分类方法研究

随着P2P快速增长带来的网络拥塞等诸多问题,准确识别P2P流量对流量控制具有重要的实际意义.提出利用PCA特征选择方法选择最优特征子集,使用概率神经网络方法对P2P流量与常规流量进行分类.实验结果表明,该方法的分类精确度与准确度有了明显的提高.     

Construction and evaluation of the new heuristic malware detection mechanism based on executable files static analysis

The paper presents the application justification of a new set of features collected at the stage of the static analysis of the executable files to address the problem of malicious code detection. In the course of study the following problems were solved: the development of the executable files classifier in the absence of a priori data concerning their functionality; designing the class models of uninfected files and malware during the learning process; the development of malicious code detection procedure using the neural networks mathematical apparatus and decision tree composition relating to the set of features specified on the basis of the executable files static analysis. The paper contains the results of experimental evaluation of the developed detection mechanism efficiency on the basis of neural networks (accuracy was 0.99125) and decision tree composition (accuracy was 0.99240). The obtained data confirmed the hypothesis about the possibility of constructing the heuristic malware analyzer on the basis of features selected during the static analysis of the executable files.     

基于统计学习的挂马网页实时检测

近年来挂马网页对Web安全造成严重威胁,客户端的主要防御手段包括反病毒软件与恶意站点黑名单。反病毒软件采用特征码匹配方法,无法有效检测经过加密与混淆变形的网页脚本代码;黑名单无法防御最新出现的恶意站点。提出一种新型的、与网页内容代码无关的挂马网页实时检测方法。该方法主要提取访问网页时HTTP会话过程的各种统计特征,利用决策树机器学习方法构建挂马网页分类模型并用于在线实时检测。实验证明,该方法能够达到89. 7%的挂马网页检测率与0. 3%的误检率。     

网页内容链接层次语义树的恶意网页检测方法

针对攻击者利用URL缩短服务导致仅依赖于URL特征的恶意网页检测失效的问题,及恶意网页检测中恶意与良性网页高度不均衡的问题,提出一种融合网页内容层次语义树特征的成本敏感学习的恶意网页检测方法。该方法通过构建网页内容链接层次语义树,提取基于语义树的特征,解决了URL缩短服务导致特征失效的问题;并通过构建成本敏感学习的检测模型,解决了数据类别不均衡的问题。实验结果表明,与现有的方法相比,提出的方法不仅能应对缩短服务的问题,还能在类别不均衡的恶意网页检测任务中表现出较低的漏报率2.1%和误报率3.3%。此外,在25万条无标签数据集上,该方法比反病毒工具VirusTotal的查全率提升了38.2%。     

基于神经网络的模糊决策树改进算法

传统决策树通过对特征空间的递归划分寻找决策边界,给出特征空间的“硬”划分。但对于处理大数据和复杂模式问题时,这种精确决策边界降低了决策树的泛化能力。为了让决策树算法获得对不精确知识的自动获取,把模糊理论引进了决策树,并在建树过程中,引入神经网络作为决策树叶节点,提出了一种基于神经网络的模糊决策树改进算法。在神经网络模糊决策树中,分类器学习包含两个阶段：第一阶段采用不确定性降低的启发式算法对大数据进行划分,直到节点划分能力低于真实度阈值[ε]停止模糊决策树的增长;第二阶段对该模糊决策树叶节点利用神经网络做具有泛化能力的分类。实验结果表明,相较于传统的分类学习算法,该算法准确率高,对识别大数据和复杂模式的分类问题能够通过结构自适应确定决策树规模。     

基于Bi-LSTM和自注意力的恶意代码检测方法

针对当前恶意代码检测方法严重依赖人工提取特征和无法提取恶意代码深层特征的问题,提出一种基于双向长短时记忆(Bidirectional Long Short Term Memory,Bi-LSTM)模型和自注意力的恶意代码检测方法。采用Bi-LSTM自动学习恶意代码样本字节流序列,输出各时间步的隐状态;利用自注意力机制计算各时间步隐状态的线性加权和作为序列的深层特征;通过全连接神经网络层和Softmax层输出深层特征的预测概率。实验结果表明该方法切实可行,相较于次优结果,准确率提高了12.32%,误报率降低了66.42%。     

Network intrusion detection based on deep learning model optimized with rule-based hybrid feature selection

ABSTRACT

Network Intrusion Detection System (NIDS) is often used to classify network traffic in an attempt to protect computer systems from various network attacks. A major component for building an efficient intrusion detection system is the preprocessing of network traffic and identification of essential features which is essential for building robust classifier. In this study, a NIDS based on deep learning model optimized with rule-based hybrid feature selection is proposed. The architecture is divided into three phases namely: hybrid feature selection, rule evaluation and detection. Several search methods and attribute evaluators were combined for features selection to enhance experimentation and comparison. The results obtained showed that the number of selected features will not affect the detection accuracy of the feature selection algorithms, but directly proportional to the performance of the base classifier. Results from the performance comparison proved that the proposed method outperforms other related methods with reduction of false alarm rate, high accuracy rate, reduced training and testing time of 1.2%, 98.8%, 7.17s and 3.11s, respectively. Finally, the simulation experiments on standard evaluation metrics showed that the proposed method is suitable for attack classification in NIDS.