基于局部信息熵的加权子空间离群点检测算法

离群点检测作为数据挖掘的一个重要研究方向,可以从大量数据中发现少量与多数数据有明显区别的数据对象.“维度灾殃”现象的存在使得很多已有的离群点检测算法对高维数据不再有效.针对这一问题,提出基于局部信息熵的加权子空间离群点检测算法SPOD.通过对数据对象在各维进行邻域信息熵分析,生成数据对象相应的离群子空间和属性权向量,对离群子空间中的属性赋以较高的权值,进一步提出子空间加权距离等概念.采用基于密度离群点检测的思想,分析计算数据对象的子空间离群影响因子,判断是否为离群点.算法能够有效地适应于高维数据离群点检测,理论分析和实验结果表明算法是有效可行的.     

基于信息熵加权的局部离群点检测算法

离群点检测是数据挖掘领域的重要研究方向之一,可以从大量数据中发现少量与多数数据有明显区别的数据对象。在诸如网络入侵、无线传感器网络异常事件等检测应用中,离群点检测是一项具有很高应用价值的技术。为了提高离群点检测准确度,文中在局部离群测度（SLOM）算法的基础上,作了一些改进,提出了一种基于密度的局部离群点检测算法ESLOM。引入信息熵确定数据对象的离群属性,并对对象距离采用加权距离,以提高离群点检测准确度。理论分析和实验表明该算法是可行有效的。     

基于Voronoi和空间自相关的离群点检测

为了提高空间数据挖掘的效率和准确度,在分析传统的离群点检测算法优、缺点的基础上,提出一种空间离群点检测算法。用Voronoi来确定空间对象间的邻近关系,在空间邻域内利用空间自相关性来计算局部Moran指数,并将其作为离群因子进而判断离群点。实验结果表明,该算法能够高效、准确地检测出空间离群点,具有对用户依赖性少和可伸缩性强等优点。     

基于Voronoi和空间自相关的离群点检测

为了提高空间数据挖掘的效率和准确度,在分析传统的离群点检测算法优、缺点的基础上,提出一种空间离群点检测算法。用Voronoi来确定空间对象间的邻近关系,在空间邻域内利用空间自相关性来计算局部Moran指数,并将其作为离群因子进而判断离群点。实验结果表明,该算法能够高效、准确地检测出空间离群点,具有对用户依赖性少和可伸缩性强等优点。     

动态数据环境下基于信息熵的相对离群点检测算法

在基于信息熵的离群点检测算法的基础上,提出一种适用于动态数据环境的检测算法。该算法在有数据对象插入或删除的时候,不必计算所有数据对象的相对离群点因子(ROF)值,而只需重新计算受影响的点的ROF值。实验结果表明,该算法在动态数据环境下的运行时间小于原来的算法。     

空间离群点的检测算法

空间离群点是指与其邻居具有明显区别的属性值的空间对象。已有的空间离散点检测算法一个主要的缺陷就是这些方法导致一些真正的离群点被忽略而把一些非离群点当成了空间离群点。本文提出了一种迭代算法,该算法通过多次迭代检测离群点,取得较好效果。实验表明该算法具有较好的实用性。     

空间离群点的检测算法

空间离群点是指与其邻居具有明显区别的属性值的空间对象。已有的空间离散点检测算法一个主要的缺陷就是这些方法导致一些真正的离群点被忽略而把一些非离群点当成了空间离群点。提出了一种迭代算法,该算法通过多次迭代检测离群点,取得较好效果。实验表明该算法具有较好的实用性。     

基于空间约束的离群点挖掘

由于现有的空间离群点检测算法没有很好地解决空间数据的自相关性和异质性约束问题,提出用计算邻域距离的方法解决空间自相关性约束问题,用计算空间局部离群系数的方法解决空间异质性约束问题。用离群系数表示对象的离群程度,并将离群系数按降序排列,取离群系数最大的前m个对象为离群点,据此提出基于空间约束的离群点挖掘算法。实验结果表明,所提算法比已有算法具有更高的检测精度、更低的用户依赖性和更高的效率。     

空间离群点的检测算法

空间离群点是指与其邻居具有明显区别的属性值的空间对象.已有的空间离散点检测算法一个主要的缺陷就是这些方法导致一些真正的离群点被忽略而把一些非离群点当成了空问离群点.提出了一种迭代算法,该算法通过多次迭代检测离群点,取得较好效果.实验表明该算法具有较好的实用性.     

基于密度的局部离群点检测算法

基于统计学和基于距离的离群点检测都依赖与给定数据点集的全局分布,然而数据通常并非都是均匀分布的。当分析分布密度相差很大的数据时,基于密度的局部离群点检测方法有着很好的识别局部离群点的能力。但存在时间复杂度较大,文章提出了一种改进的算法,能降低时间复杂度,实现有效的局部离群点的检测。     

基于熵度量的空间邻域离群点查找

离群点的查找算法主要有两类：第一类是面向统计数据,把各种数据都看成是多维空间,没有区分空间维与非空间维,这类算法可能产生错误的判断或找到的是无意义的离群点;第二类算法面向空间数据,区分空间维与非空间维,但该类算法查找效率太低或不能查找邻域离群点。引入熵权的概念,提出了一种新的基于熵权的空间邻域离群点度量算法。算法面向空间数据,区分空间维与非空间维,利用空间索引划分空间邻域,用非空间属性计算空间偏离因子,由此度量空间邻域的离群点。理论分析表明,该算法是合理的。实验结果表明,算法具有对用户依赖性小、检测精度和计算效率高的优点。     

Distance-based outliers: algorithms and applications

This paper deals with finding outliers (exceptions) in large, multidimensional datasets. The identification of outliers can lead to the discovery of truly unexpected knowledge in areas such as electronic commerce, credit card fraud, and even the analysis of performance statistics of professional athletes. Existing methods that we have seen for finding outliers can only deal efficiently with two dimensions/attributes of a dataset. In this paper, we study the notion of DB (distance-based) outliers. Specifically, we show that (i) outlier detection can be done efficiently for large datasets, and for k-dimensional datasets with large values of k (e.g., ); and (ii), outlier detection is a meaningful and important knowledge discovery task. First, we present two simple algorithms, both having a complexity of , k being the dimensionality and N being the number of objects in the dataset. These algorithms readily support datasets with many more than two attributes. Second, we present an optimized cell-based algorithm that has a complexity that is linear with respect to N, but exponential with respect to k. We provide experimental results indicating that this algorithm significantly outperforms the two simple algorithms for . Third, for datasets that are mainly disk-resident, we present another version of the cell-based algorithm that guarantees at most three passes over a dataset. Again, experimental results show that this algorithm is by far the best for . Finally, we discuss our work on three real-life applications, including one on spatio-temporal data (e.g., a video surveillance application), in order to confirm the relevance and broad applicability of DB outliers. Received February 15, 1999 / Accepted August 1, 1999     

Approximate voronoi cell computation on spatial data streams

Several studies have exploited the properties of Voronoi diagrams to improve the efficiency of variations of the nearest neighbor search on stored datasets. However, the significance of Voronoi diagrams and their basic building blocks, Voronoi cells, has been neglected when the geometry data is incrementally becoming available as a data stream. In this paper, we study the problem of Voronoi cell computation for fixed 2-d site points when the locations of the neighboring sites arrive as a spatial data stream. We show that the non-streaming solution to the problem does not meet the memory requirements of many realistic scenarios over a sliding window. Hence, we propose AVC-SW, an approximate streaming algorithm that computes (1 + ε)-approximations to the actual exact Voronoi cell in O(κ) where κ is its sample size. With the sliding window model and random arrival of points, we show both analytically and experimentally that for given window size w and parameter k, AVC-SW reduces the expected memory requirements of the classic algorithm from O(w) to regardless of the distribution of the points in the 2-d space. This is a significant improvement for most of the real-world scenarios where w ≫ k.     

Efficient algorithms for spatial configuration information retrieval

The problem of spatial configuration information retrieval is a constraint satisfaction problem (CSP), which can be solved using traditional CSP algorithms. But the spatial data can be reorganized using index techniques like R-tree and the spatial data are approximated by their minimum bounding rectangles (MBRs), so the spatial configuration information retrieval is actually based on the MBRs and some special techniques can be studied. This paper studies the mapping relationships among the spatial relations for real spatial objects, the corresponding spatial relations for their MBRs and the corresponding spatial relations between the intermediate nodes and the MBRs in R-tree. Three algorithms are designed and studied, and their performances are compared.     

基于空间复用的信号检测算法研究

为了在接收端恢复出发送端的原始数据,需要在接收端进行信号检测。对几种经典的传统信号检测算法进行了详细阐述和分析,并对各种算法进行了Matlab仿真和性能比较。由此得出,改进型的V-BLAST算法可以用于TD-LTE无线综合测试仪的开发。     

基于轨迹信息熵分布的异常轨迹检测方法

针对异常轨迹检测多特征检测和检测单元造成的检测效率低等问题。提出一种基于轨迹信息熵分布的异常轨迹检测方法。该算法根据轨迹偏转角与速度将轨迹分割成若干轨迹段,计算轨迹段间加权多特征距离判断轨迹间相似度,进而完成轨迹聚类并计算出每类代表性轨迹,然后对待检测轨迹进行分割,利用代表性轨迹计算每个轨迹段的信息熵,通过比较轨迹信息熵大小及其分布特点实现异常轨迹检测。大西洋飓风数据仿真实验结果表明该方法提高了聚类效果,克服以整条轨迹检测效率低的缺点,提升了异常轨迹检测算法的有效性。     

基于修正熵权算法的信息披露质量评价

基于信息披露质量与信息披露违规的对偶关系,从违规行为直接评价与违规影响间接评价两个方面,构建了三级层次的信息披露质量评价指标体系,并利用建立在一般熵权算法基础上的修正熵权算法,评价了深交所1 385家上市公司的信息披露质量。实证计算、t检验及Wilcoxon检验的结果表明,修正熵权算法既保证评价结果无显著性差异,也比一般熵权算法更具适用性和有效性。     

基于距离和密度的分布式孤立点探测算法

在数据挖掘中,对待数据集中的孤立点有两种策略:一种是将它们作为噪声而从数据集中清洗出去;另一种则将它们作为研究对象,称为孤立点探测.孤立点如此不同于集合中的其他数据点,以至于我们怀疑这些孤立点是否基于不同的机制产生.基于Bay算法和LOF(local outlier factor)算法,设计了两种分布式孤立点探测算法,一种基于距离探测,另一种基于密度.估计的这两种算法的工作量显示出它们的通讯量可以有效的控制,工作量上限为二次阶.     

Adaptive detection of spatial point event outliers using multilevel constrained Delaunay triangulation

Spatial outlier detection is a research hot spot in the field of spatial data mining. Because of the lack of specific research on spatial point events, this study presents an adaptive approach for spatial point events outlier detection (SPEOD) using multilevel constrained Delaunay triangulation. First, the spatial proximity relationships between spatial point events are roughly captured by Delaunay triangulation. Then, three-level constraints are described and used to refine spatial proximity relationships with the consideration of statistical characteristics. Finally, those spatial point events connected by remaining edges are gathered to form a series of subgraphs. Those subgraphs containing very few point events are regarded as spatial outliers. Experiments on both synthetic and real-world spatial data sets are used to show that the proposed SPEOD algorithm can detect various types of spatial point event outliers with high efficiency. Moreover, there is no need to input any parameter in SPEOD.     

基于信息熵SVM的ICMP隐蔽通道检测研究

许多网络设备考虑ICMP流量是良性,对其负载部分不进行检测,因此,攻击者可以将生成的任意信息隐藏在ICMP的有效负载中。将信息熵引入到支持向量机建模中,分析了高熵点与支持向量的关系,进一步构造出用于检测ICMP负载隐蔽通道的信息熵支持向量机模型,最后给出了相关实验,实验结果表明使用信息熵支持向量机检测ICMP负载隐蔽通道具有较快的分类速度和较高的分类精度。