Multi-domain anomaly detection in spatial datasets

A spatial anomaly captures a phenomenon occurring in a region which is vastly deviant in behavior with respect to the other normal observations. However, in reality this anomaly may impact other phenomena in the region across multiple domains, for example, crime is often linked to other sociopolitical factors or phenomenon such as poverty and education. Similarly, accidents in the region may be linked to other environmental factors such as weather and surface condition. So, finding anomalies across multiple domains is important in various applications. In this paper, we propose an approach for finding such a tangible anomalous window across multiple domains where window refers to the set of contiguous points in space, and since the window is multi-domain, there are several overlapping windows in the same space across domains. Our approach for finding anomalous window across the domains comprises the following steps: (1) single-domain anomaly detection: discovering anomalous window in each domain; (2) association rule mining: discovering relationship between the anomalous windows across domains using association rule mining; and (3) validation: validating the result using (a) Monte Carlo simulation, (b) correlation using lift and (c) ground truth evaluation. In addition, we also provide a probabilistic framework to evaluate the relationships between the spatial nodes as a postprocessing step. Finally, we provide a visualization technique for viewing the multi-domain anomalous window and the probabilistic relationships between the nodes. We provide detailed experimental results and comparisons with other approaches using real-world health ranking [51] and transportation datasets [50] with known ground truth windows. The results show that our approach is effective in finding the anomalies in multiple domains as compared to other approaches.     

Distributed anomaly detection for industrial wireless sensor networks based on fuzzy data modelling

Modern infrastructure increasingly depends on large computerized systems for their reliable operation. Supervisory Control and Data Acquisition (SCADA) systems are being deployed to monitor and control large scale distributed infrastructures (e.g. power plants, water distribution systems). A recent trend is to incorporate Wireless Sensor Networks (WSNs) to sense and gather data. However, due to the broadcast nature of the network and inherent limitations in the sensor nodes themselves, they are vulnerable to different types of security attacks. Given the critical aspects of the underlying infrastructure it is an extremely important research challenge to provide effective methods to detect malicious activities on these networks. This paper proposes a robust and scalable mechanism that aims to detect malicious anomalies accurately and efficiently using distributed in-network processing in a hierarchical framework. Unsupervised data partitioning is performed distributively adapting fuzzy c-means clustering in an incremental model. Non-parametric and non-probabilistic anomaly detection is performed through fuzzy membership evaluations and thresholds on observed inter-cluster distances. Robust thresholds are determined adaptively using second order statistical knowledge at each evaluation stage. Extensive experiments were performed and the results demonstrate that the proposed framework achieves high detection accuracy compared to existing data clustering approaches with more than 96% less communication overheads opposed to a centralized approach.     

基于K-近邻树的离群检测算法

为适应数据集分布形状多样性以及克服数据集密度问题,针对已有算法对离群簇检测效果欠佳的现状,提出了一种基于K-近邻树的离群检测算法KNMOD(outlier detection based on K-nearest neighborhood MST).算法结合密度与方向因素,提出一种基于K-近邻的不相似性度量,然后带约束切割基于此度量构建的最小生成树从而获得离群点.算法可以有效地检测出局部离群点以及局部离群簇,与LOF、COF、KNN及INFLO算法的对比结果也证实了算法的优越性能.     

基于邻域粗糙集的入侵检测

针对入侵检测系统存在的高漏报率和误报率,提出了一种基于邻域粗糙集的入侵检测方法.该方法在粗糙集理论的基础上引入邻域概念,这样便无需对数据进行离散化处理,可以减少信息损失.实验结果表明:该方法可选择出更为重要的属性组合,从而获得较高的检测率和较低的漏报率与误报率.     

一种基于聚类的异常检测方法

利用数据挖掘技术对网络中的海量数据进行分析从而发现入侵行为已成为目前异常检测研究的重点.为了进一步提高入侵行为检测的质量,提出了一种改进的异常检测算法.该方法首先将训练数据集转换为标准的单位特征度量空间,然后利用改进算法对数据进行划分,以找到聚类中心.最后对改进算法进行了性能分析与比较,实验结果表明:算法具有良好的稳定...     

Unsupervised and non-parametric learning-based anomaly detection system using vibration sensor data

Multimedia Tools and Applications - In this paper, we propose an anomaly detection system of machines using a hybrid learning mechanism that combines two kinds of machine learning approaches,...     

一种基于自回归模型的网络异常检测方法

随着网络技术的不断发展,计算机病毒、网络攻击等问题也日益严峻.维护网络的安全和稳定,是一个亟须解决的问题.针对该问题,介绍了一种基于自回归模型的网络异常检测方法,该方法将局部的网络流量看作统计学上近似的平稳.OPNET上的仿真实验表明,该方法能有效检测出网络异常,误报率低.     

基于异常因子的异常模式探测算法

时间序列挖掘中不同的数据集中的异常模式的长度未必相同.提出的算法使用异常因子作为模式的异常度量,利用模式的k-距离和中位数来计算异常因子,使用二次回归算法来探测时间序列中的所有模式和其长度范围,在这个范围内使用变长方法来判断一个模式是否异常,然后合并相邻的异常模式.为了验证算法的有效性和健壮性,使用人工合成数据和标准数据集对算法进行了测试,得到了较为满意的效果.     

Hyperspectral anomaly detection based on uniformly partitioned pixel

The objective of this paper is to develop an algorithm to detect anomaly in a hyperspectral image. The algorithm is based on a subspace model that is derived statistically. The anomaly detector is defined as the Mahalanobis distance of a residual from a pixel that is partitioned uniformly. The high correlation among adjacent components of the pixel is exploited by partitioning the pixel uniformly to improve anomaly detection. The residual is obtained by partialling out the main background from the pixel by predicting a linear combination of each partition of the pixel with a linear combination of the random variables representing the main background. Experimental results show that the anomaly detector outperforms conventional anomaly detectors.     

基于有效载荷的异常入侵检测技术研究

分析了目前入侵检测存在的问题,提出了一种基于有效载荷的异常入侵检测技术.该技术选取网络数据包有效载荷的位分布作为系统特征值,采用统计学中的马哈拉诺比斯距离作为区分合法访问与非法入侵的算法,降低了误报率,提高了检测精度.实验结果表明,该检测技术是有效的,具备一定的识别未知入侵的能力,可以实现实时高效的异常入侵检测.     

基于W-Kmeans算法的DNS流量异常检测

为了对DNS查询进行有效检测,及时发现DNS流量异常,提出了适合于检测DNS流量异常的权重Kmeans (WKmeans)算法.对CN顶级域2009年5月19日的原始查询日志抽取有用信息,提取相关的向量特征,对不同的向量特征赋予不同的权重值.利用W-Kmeans算法对查询日志进行聚类检测,并分析了算法各种参数选择的影响.5.19事件的DNS查询检测结果表明,W-Kmeans算法可以有效检测DNS流量异常的发生.     

基于异常与误用的入侵检测系统

入侵检测系统近年来得到长足的发展,但功能都不够完善.为此将基于误用的入侵检测与基于异常的检测结合为一体.在误用检测上,将检测规则进行分类排序,从而极大地提高了检测效率.异常检测则采用人工免疫技术,使系统对已知的攻击和新型攻击均有较强检测能力.     

基于网络处理器的多维统计异常检测系统

基于网络处理器开发的网络设备能够很好地解决灵活性和高性能之间的矛盾.基于网络处理器IXP2400自身的特点,设计了多维异常检测系统.该系统可以有效地检测和防御DDOS攻击.根据TCP/IP协议簇,对数据包进行多维解析,统计以及异常标记.仿真和硬件实验的验证数据表明,该系统能准确无误地按照设计目标一一分解数据包,并标记出异常值,从而为后续的网络安全的研究和防御工作提供可靠的数据保证.     

Network anomaly detection based on selective ensemble algorithm

The Journal of Supercomputing - In order to reduce the loss of information of the majority class samples in the resampling process, combining the distribution of class samples and the...     

Spatio-temporal trajectory anomaly detection based on common sub-sequence

Applied Intelligence - With the rapid development of GPS positioning and wireless communication, more and more trajectories are collected. How to accurately and efficiently detect abnormal...     

基于集成学习的云平台异常点检测

为提高云平台异常点检测的精度,解决单一检测系统误报率与漏报率高的问题,提出基于集成学习的异常点检测系统。为解决异常检测对象多样性的问题,构造监测序列的特征矩阵,采用自组织映射神经网络对监测序列进行聚类;对监测序列进行过采样,解决异常发生频率很低的问题;对异构的异常点检测器进行基于委员会的学习,集成各检测器的检测优点,提高检测的精度。通过带有标注的监测序列对异常点检测系统进行验证,结果表明,该系统效果优于单一检测系统,验证了设计的有效性。     

基于多维度特征分析的KPI异常检测

为帮助运维人员提前发现未知风险,减少因异常风险带来的损失,提出多种特征融合的异常检测方法.对关键性能指标(KPI)进行多维度的特征提取,使用主成分分析方法(PCA)进行降维,对降维后的数据按照时序模式,使用小波分解提取出高频特征与低频特征,使用极限梯度提升(XGBoost)模型进行异常检测.实验结果表明,该方法有较好的...