基于逻辑回归模型的流量异常检测方法研究

网络流量作为异常检测的基本数据源,其行为特征的准确描述,是网络异常行为实时检测的重要依据.本文针对流量异常检测问题,提出了一种基于逻辑回归模型的网络流量异常检测方法.通过分析源IP、目的 IP等多个网络流量基本特征,构造了网络异常行为和正常行为的训练机,并且在此基础上采用逻辑回归建立起网络异常流量挖掘模型.利用实验室所采集的真实网络流量对所构建的模型进行检测,以验证该模型的有效性.实验结果表明本文所建立的网络模型在异常流量挖掘方面准确度高、实时性好.     

BotSward: Centrality Measures for Graph-Based Bot Detection Using Machine Learning

The number of botnet malware attacks on Internet devices has grown at an equivalent rate to the number of Internet devices that are connected to the Internet. Bot detection using machine learning (ML) with flow-based features has been extensively studied in the literature. Existing flow-based detection methods involve significant computational overhead that does not completely capture network communication patterns that might reveal other features of malicious hosts. Recently, Graph-Based Bot Detection methods using ML have gained attention to overcome these limitations, as graphs provide a real representation of network communications. The purpose of this study is to build a botnet malware detection system utilizing centrality measures for graph-based botnet detection and ML. We propose BotSward, a graph-based bot detection system that is based on ML. We apply the efficient centrality measures, which are Closeness Centrality (CC), Degree Centrality (CC), and PageRank (PR), and compare them with others used in the state-of-the-art. The efficiency of the proposed method is verified on the available Czech Technical University 13 dataset (CTU-13). The CTU-13 dataset contains 13 real botnet traffic scenarios that are connected to a command-and-control (C&C) channel and that cause malicious actions such as phishing, distributed denial-of-service (DDoS) attacks, spam attacks, etc. BotSward is robust to zero-day attacks, suitable for large-scale datasets, and is intended to produce better accuracy than state-of-the-art techniques. The proposed BotSward solution achieved 99% accuracy in botnet attack detection with a false positive rate as low as 0.0001%.     

An IoT Environment Based Framework for Intelligent Intrusion Detection

Software-defined networking (SDN) represents a paradigm shift in network traffic management. It distinguishes between the data and control planes. APIs are then used to communicate between these planes. The controller is central to the management of an SDN network and is subject to security concerns. This research shows how a deep learning algorithm can detect intrusions in SDN-based IoT networks. Overfitting, low accuracy, and efficient feature selection is all discussed. We propose a hybrid machine learning-based approach based on Random Forest and Long Short-Term Memory (LSTM). In this study, a new dataset based specifically on Software Defined Networks is used in SDN. To obtain the best and most relevant features, a feature selection technique is used. Several experiments have revealed that the proposed solution is a superior method for detecting flow-based anomalies. The performance of our proposed model is also measured in terms of accuracy, recall, and precision. F1 rating and detection time Furthermore, a lightweight model for training is proposed, which selects fewer features while maintaining the model’s performance. Experiments show that the adopted methodology outperforms existing models.     

Diagnosis of multiple faults with flow-based functional models: the functional diagnosis with efforts and flows approach

This paper presents the Functional Diagnosis with efforts and flows (FDef) approach to model-based diagnosis. First, it motivates our research by considering the field of functional flow-based diagnostic techniques, pointing out some of their relevant limitations, which are overcome by FDef. Then, it summarizes and exemplifies the main FDef concepts and techniques in a tutorial way. A real-world diagnostic problem in the marine engineering domain has been used to evaluate the basic FDef engine. The paper summarizes the results of the evaluation and discusses the extensions it has prompted, i.e., the introduction of qualitative deviations of variables and the exploitation of a hierarchical representation of the functional model. The ideas proposed in FDef can be easily adapted to other flow-based modeling formalisms, such as MFM.     

YATA: Yet Another Proposal for Traffic Analysis and Anomaly Detection

Network traffic anomaly detection has gained considerable attention over the years in many areas of great importance. Traditional methods used for detecting anomalies produce quantitative results derived from multi-source information. This makes it difficult for administrators to comprehend and deal with the underlying situations. This study proposes another method to yet determine traffic anomaly (YATA), based on the cloud model. YATA adopts forward and backward cloud transformation algorithms to fuse the quantitative value of acquisitions into the qualitative concept of anomaly degree. This method achieves rapid and direct perspective of network traffic. Experimental results with standard dataset indicate that using the proposed method to detect attacking traffic could meet preferable and expected requirements.     

A max-algebra approach to the robust distributed control of repetitive AGV systems

This paper presents a new approach for distributed control of automatic guided vehicle systems (AGVS) that uses max-algebra formalism to model system operation. The proposed method differs from previous works on performance analysis and control of such systems in that it constructs a feasible schedule by exploiting the repetitive character of the flow of automated guided vehicles. The AGVS is treated as a distributed system composed of autonomous and repetitive processes of AGV flows. Its periodic behaviour follows from the type of operation characterizing the job shops considered. The proposed approach employs a concept of asynchronous traffic semaphores that provide the local mechanism for vehicle flow synchronization. Setting the timing of particular semaphores results in distributed control of vehicle flows possessing self-synchronization capabilities.     

基于统计的网络流量模型及异常流量发现

基于从网络流量的大小和流量曲线的形状两个方面的研究,建立了一种基于统计的网络流量模型,提出了计算正常情况网络流量曲线的算法.通过对比正常网络流量曲线和异常网络流量曲线之间的差距,实现了对异常数据流的自动检测.实验表明,该模型不仅可以模拟与网络实测数据相似的网络流量,而且具有一定的异常流量发现能力.     

基于状态反馈的网络分布式拥塞管理方案

提出了一种新型的面向区分服务网络的分布式拥塞管理方案。其基本思想是利用拥塞状态反馈信息在边缘节点或主机上实施拥塞管理，该方案主要包括三个组成部分；拥塞状态控制分组，早期拥塞检测和流量控制算法，实验结果表明，与标准的区分服务网络相比，该方案能在TCP和UDP聚集之间公平地分配带宽并能显著地降低分组丢失率。     

Malicious Traffic Detection in IoT and Local Networks Using Stacked Ensemble Classifier

Malicious traffic detection over the internet is one of the challenging areas for researchers to protect network infrastructures from any malicious activity. Several shortcomings of a network system can be leveraged by an attacker to get unauthorized access through malicious traffic. Safeguard from such attacks requires an efficient automatic system that can detect malicious traffic timely and avoid system damage. Currently, many automated systems can detect malicious activity, however, the efficacy and accuracy need further improvement to detect malicious traffic from multi-domain systems. The present study focuses on the detection of malicious traffic with high accuracy using machine learning techniques. The proposed approach used two datasets UNSW-NB15 and IoTID20 which contain the data for IoT-based traffic and local network traffic, respectively. Both datasets were combined to increase the capability of the proposed approach in detecting malicious traffic from local and IoT networks, with high accuracy. Horizontally merging both datasets requires an equal number of features which was achieved by reducing feature count to 30 for each dataset by leveraging principal component analysis (PCA). The proposed model incorporates stacked ensemble model extra boosting forest (EBF) which is a combination of tree-based models such as extra tree classifier, gradient boosting classifier, and random forest using a stacked ensemble approach. Empirical results show that EBF performed significantly better and achieved the highest accuracy score of 0.985 and 0.984 on the multi-domain dataset for two and four classes, respectively.     

Establishing relationships between accidents and flows at urban priority road junctions

This papers explores the effects of traffic stream flows on accident potential at urban priority-controlled (i.e. unsignalised), four-arm junctions. Forty-three urban priority junctions were carefully selected so that other than flow parameters expected to influence accident potential have similar values at the junctions considered. Using traffic accident data for five-year time periods and the corresponding 24-hour flows, a new exposure index is proposed consisting of an expression of the flows of the junction's interacting traffic streams. The regression of this exposure index on the expected number of accidents per year at junctions of the type examined yields a quite satisfactory correlation coefficient, better than those achieved when other proposed indices are used.     

On the Analysis of Communication and Computer Networks by Traffic Flow Measurements

This paper investigates the potential contributions of traffic flow measurements in monitoring and network diagnostics. The basic idea is that information of diagnostic relevance may be obtained from the detection of local anomalies in a traffic trace. For this purpose, this paper introduces a novel approach in the analysis of aggregate traffic, based on the determination of empirical rate-interval curves (RICs). These curves allow an analysis of flow quantiles versus time scale, which is helpful both in the investigation of the scaling properties of network traffic and in diagnostics. RIC-based analyses of traffic measurements taken from different networks are presented and show that the proposed approach appears to be effective in evidencing potential flow anomalies.     

Deep Learning-Based Hybrid Intelligent Intrusion Detection System

Machine learning (ML) algorithms are often used to design effective intrusion detection (ID) systems for appropriate mitigation and effective detection of malicious cyber threats at the host and network levels. However, cybersecurity attacks are still increasing. An ID system can play a vital role in detecting such threats. Existing ID systems are unable to detect malicious threats, primarily because they adopt approaches that are based on traditional ML techniques, which are less concerned with the accurate classification and feature selection. Thus, developing an accurate and intelligent ID system is a priority. The main objective of this study was to develop a hybrid intelligent intrusion detection system (HIIDS) to learn crucial features representation efficiently and automatically from massive unlabeled raw network traffic data. Many ID datasets are publicly available to the cybersecurity research community. As such, we used a spark MLlib (machine learning library)-based robust classifier, such as logistic regression (LR), extreme gradient boosting (XGB) was used for anomaly detection, and a state-of-the-art DL, such as a long short-term memory autoencoder (LSTMAE) for misuse attack was used to develop an efficient and HIIDS to detect and classify unpredictable attacks. Our approach utilized LSTM to detect temporal features and an AE to more efficiently detect global features. Therefore, to evaluate the efficacy of our proposed approach, experiments were conducted on a publicly existing dataset, the contemporary real-life ISCX-UNB dataset. The simulation results demonstrate that our proposed spark MLlib and LSTMAE-based HIIDS significantly outperformed existing ID approaches, achieving a high accuracy rate of up to 97.52% for the ISCX-UNB dataset respectively 10-fold cross-validation test. It is quite promising to use our proposed HIIDS in real-world circumstances on a large-scale.     

Network Intrusion Detection Using CFAR Abrupt-Change Detectors

In this paper, the constant false alarm rate (CFAR) detectors are proposed for network intrusion detection. By using an autoregressive system to model the network traffic, predictor error is shown to closely follow a Gaussian distribution. CFAR detector approaches are then developed on the prediction error distribution. In the present study, we consider the optimal CFAR, the cell-averaging CFAR, and the order statistics CFAR. The use of these CFAR techniques can significantly improve the detection performance. In addition, we propose the use of fusion of these CFAR detectors by using Dempster-Shafer and Bayesian techniques. Computer simulations based on the DARPA traffic data show that the proposed approach achieves higher detection probabilities than the conventional detection method. Even under different types of attacks, the intrusion detection performances based on the proposed CFAR detectors shows consistent improvement.     

Diagnosing Anomalies and Identifying Faulty Nodes in Sensor Networks

In this paper, an anomaly detection approach that fuses data gathered from different nodes in a distributed sensor network is proposed and evaluated. The emphasis of this work is placed on the data integrity and accuracy problem caused by compromised or malfunctioning nodes. The proposed approach utilizes and applies Principal Component Analysis simultaneously on multiple metrics received from various sensors. One of the key features of the proposed approach is that it provides an integrated methodology of taking into consideration and combining effectively correlated sensor data, in a distributed fashion, in order to reveal anomalies that span through a number of neighboring sensors. Furthermore, it allows the integration of results from neighboring network areas to detect correlated anomalies/attacks that involve multiple groups of nodes. The efficiency and effectiveness of the proposed approach is demonstrated for a real use case that utilizes meteorological data collected from a distributed set of sensor nodes     

基于多链差分进化的贝叶斯有限元模型修正方法

针对传统贝叶斯算法在高维参数下采样效率低且收敛难的问题,建立了基于多链差分进化算法的贝叶斯有限元模型修正方法。在标准马尔可夫链蒙特卡罗（MCMC）方法的基础上,引入差分进化算法,通过多条马氏链间的随机差分运算来自适应选择条件分布的大小和方向以快速逼近目标分布;引入子空间采样算法,通过自适应选择优良的参数维度进行采样以提高采样效率;引入异常链检测算法,通过在采样的非平稳期对马氏链进行异常检测与剔除以提高在平稳期的采样效率。简支梁理论模型和实验室4层框架结构的模型修正结果表明:该方法修正精度较高,且具有良好的抗噪性,在高阶频率以及振型下的修正效果均优于DRAM算法,为解决不确定性模型修正中的计算精度提供了一种新手段。     

Traffic Sign Detection with Low Complexity for Intelligent Vehicles Based on Hybrid Features

Globally traffic signs are used by all countries for healthier traffic flow and to protect drivers and pedestrians. Consequently, traffic signs have been of great importance for every civilized country, which makes researchers give more focus on the automatic detection of traffic signs. Detecting these traffic signs is challenging due to being in the dark, far away, partially occluded, and affected by the lighting or the presence of similar objects. An innovative traffic sign detection method for red and blue signs in color images is proposed to resolve these issues. This technique aimed to devise an efficient, robust and accurate approach. To attain this, initially, the approach presented a new formula, inspired by existing work, to enhance the image using red and green channels instead of blue, which segmented using a threshold calculated from the correlational property of the image. Next, a new set of features is proposed, motivated by existing features. Texture and color features are fused after getting extracted on the channel of Red, Green, and Blue (RGB), Hue, Saturation, and Value (HSV), and YCbCr color models of images. Later, the set of features is employed on different classification frameworks, from which quadratic support vector machine (SVM) outnumbered the others with an accuracy of 98.5%. The proposed method is tested on German Traffic Sign Detection Benchmark (GTSDB) images. The results are satisfactory when compared to the preceding work.     

MalDetect: A Structure of Encrypted Malware Traffic Detection

Recently, TLS protocol has been widely used to secure the application data carried in network traffic. It becomes more difficult for attackers to decipher messages through capturing the traffic generated from communications of hosts. On the other hand, malwares adopt TLS protocol when accessing to internet, which makes most malware traffic detection methods, such as DPI (Deep Packet Inspection), ineffective. Some literatures use statistical method with extracting the observable data fields exposed in TLS connections to train machine learning classifiers so as to infer whether a traffic flow is malware or not. However, most of them adopt the features based on the complete flow, such as flow duration, but seldom consider that the detection result should be given out as soon as possible. In this paper, we propose MalDetect, a structure of encrypted malware traffic detection. MalDetect only extracts features from approximately 8 packets (the number varies in different flows) at the beginning of traffic flows, which makes it capable of detecting malware traffic before the malware behaviors take practical impacts. In addition, observing that it is inefficient and time-consuming to re-train the offline classifier when new flow samples arrive, we deploy Online Random Forest in MalDetect. This enables the classifier to update its parameters in online mode and gets rid of the re-training process. MalDetect is coded in C++ language and open in Github. Furthermore, MalDetect is thoroughly evaluated from three aspects: effectiveness, timeliness and performance.     

Charge-coupled device operated in a time-delayed integration mode as an approach to high-throughput flow-based single molecule analysis

Single molecule detection (SMD) readouts are particularly attractive for assays geared toward high-throughput processing, because they can potentially reduce assay time by eliminating various processing steps. Unfortunately, most flow-based SMD experiments have generated low throughputs due primarily to the fact that they are configured in single assay formats. The use of a charge-coupled device (CCD) with flow-based SMD can image multiple single molecule assays simultaneously to realize high-throughput processing capabilities. We present, for the first time, the ability to simultaneously track and detect single molecules in multiple microfluidic channels by employing a CCD camera operated in time-delayed integration (TDI) mode as a means for increasing the throughput of any single molecule measurement. As an example of the technology, we have configured a CCD to operate in a TDI mode to detect single double-stranded DNA molecules (lambda and pBR322) labeled with an intercalating dye (TOTO-3) in a series of microfluidic channels poised on a poly(methyl methacrylate), PMMA, chip. A laser beam was launched into the side of the chip, which irradiated a series of fluidic channels (eight) with the resulting fluorescence imaged onto a CCD. Using this system, we were able to identify single DNA molecules based on the fluorescence burst intensity arising from differences in the extent of dye labeling associated with the DNA molecule length. The CCD/TDI approach allowed increasing sample throughput by a factor of 8 compared to a single-assay SMD experiment. A sampling throughput of 276 molecules s (-1) per channel and 2208 molecules s (-1) for an eight channel microfluidic system was demonstrated. Operated in its full capacity, this multichannel format was projected to yield a sample throughput of 1.7 x 10 (7) molecules s (-1), which represents a 170-fold improvement over previously reported single molecule sampling rates.     

Enhanced token bucket policer using reduced fair queuing for Ethernet access networks

An enhanced traffic policer capable of adaptive rate control to provide fair distribution of available network bandwidth is proposed. The policer also performs maximum rate control, limiting flow's use of network resources within a pre-defined bound. Since the enhanced policer is based on a token bucket policer, a flow is guaranteed to receive service at its reserved rate. The proposed policer partially employs a reduced fair queuing algorithm, which is designed for adaptive rate control based on the perceived traffic congestion level. The adaptive rate control requires measurement of the congestion level, which is approximated through examination of local buffer usage. As a result, the allocated service rate increase provided to the flows by the policer is inversely proportional to the network congestion level. In addition, far-travelling flows in multi-hop networks receive the same service rate enhancements as short-travelling flows. The proposed scheme is useful in Ethernet networks, especially access networks, where quality of service is not well organised.