基于近邻传播的离群点检测算法

离群点是与其他正常点属性不同的一类对象,其检测技术在各行业上均有维护数据纯度、保障业内安全等重要应用,现有算法大多是基于距离、密度等传统方法判断检测离群点.本算法给每个对象分配一个"孤立度",即该点相对其邻点的孤立程度,通过排序进行判定,比传统算法效率更高.在AP(affinity propagation)聚类算法的基础上进行改进与优化,提出能检测异常数据点的算法APO(outlier detection algorithm based on affinity propagation).通过加入孤立度模块并计算处理样本点的孤立信息,并引入放大因子,使其与正常点之间的差异更明显,通过增大算法对离群点的敏感性,提高算法的准确性.分别在模拟数据集和真实数据集上进行对比实验,结果表明:该算法与AP算法相比,对离群点的敏感性更加强烈,且本算法检测离群点的同时也能聚类,是其他检测算法所不具备的.

Outlier detection algorithm based on affinity propagation

Outliers are a class of objects with different properties from other normal points, whose detection technology in various industries has a wide application to maintain the purity of data and ensure the safety of the industry. Most of the existing algorithms are based on distance, density, and other traditional methods to detect outliers. This paper assigned each object an "isolation degree", the degree of isolation of the point relative to adjacent points, which could identify outliers by sorting, that''s more efficient. It proposed the detection technology APO by improving and optimizing the AP clustering algorithm. It introduced the outlier module and processed the isolated information of points. In addition, it added the amplification factor to enlarge the isolation of outliers, so that the difference between the outliers and the normal points is more obvious, the sensitivity and the accuracy of the algorithm is much higher. The experiment used simulated dataset real datasets, who''s the results showed that the algorithm was more sensitive and it detected outliers more accurately than AP algorithm. In addition, this algorithm can cluster outliers while detecting outliers, which is not available in other detection algorithms.