基于熵距离的离群点检测及其应用

离群数据检测是找出与正常数据不一致的数据。由于某种原因,会出现一些噪声数据。针对噪声数据的特征,提出了一个有效的离群点检测算法。通过层次k-means算法对数据集进行聚类,从包括离群点可能性最大的簇开始进行检测,在检测过程中提出基于熵值距离来衡量数据点的离群程度,并通过剪枝规则来减少检测次数,从而提高了检测的效率。仿真结果表明该算法对出现的噪声数据具有较好的过滤效果。

Entropy Distance-based Outlier Detection and Its Application

The outlier detection is to identify the inconsistent data different to the normal data. For some reason, there will be some noise data in the student ratings of teaching evaluation. Based on the characteristics of noise data in teaching evaluation, this paper proposes an effective entropy distance-based outlier detection algorithm. This algorithm uses k-means algorithm to implement data clustering and then detects from the cluster with the highest possibility of including outlier. During the detection, the degree of outlier data is measured based on entropy distance. The simulation results show that this algorithm can have better filtering effect for noise data in student ratings of teaching effectiveness evaluation.