基于信息论的高维海量数据离群点挖掘

针对高维海量数据集离群点挖掘存在“维数灾难”的问题,提出了基于信息论的高维海量数据的离群点挖掘算法。该算法采用属性选择,去除冗余属性降维。利用信息嫡作为离群点判断的度量标准,消除距离和密度量纲的弊端。在真实数据集上的实验结果表明,算法对高维海量数据离群点挖掘是有效可行的,其效率和精度得到了明显提高。

Outlier Mining of the High-dimension Datasets Based on Information Theory

Phenomena of "curse of dimensionality" deteriorate lots of existing outlier mining algorithms validity. Conconing thw problem, the outlier mining algorithm of high-dimension and large datasets based on information theory was proposed. This algorithm used the concept of information entropy and the mutual information in the information theory,carried on the feature selection after using estimated mutual information value objective basis entropy power sorting, and eliminated redundant attribute for dimensionality reduction. Outlier mining using information entropy as a measure standard to judge eliminated the drawbacks of distance and density metric. The experimental result in the real data sets indicates that the algorithm for outlicr mining in high-dimensional mass data is effective and feasible, its efficiency and accuracy arc significantly improved.