基于属性相关分析的离群数据并行挖掘算法

针对高维海量数据集中的局部离群数据,利用并行计算和属性相关性分析思想,给出了一种离群数据并行挖掘算法。该算法首先由主节点分配属性相关分析任务,各个子节点并行查找数据集中的冗余属性,将其冗余属性传回主节点,并由主节点删除;其次,主节点分配搜索任务,各子节点采用微粒群算法,并行搜索局部离群子空间;再次,由主节点对局部离群子空间合并计算后,确定全局离群数据;最后,在MPICH2-1.0.3的并行计算环境下,采用恒星光谱数据作为数据集,实验结果验证了算法的正确性和有效性。

A Local Outlier Mining Parallel Algorithm Based on Attribute Relevance Analysis

For high-dimensional and large data set,an outlier parallel mining algorithm is presented by taking attribute relevance analysis.Firstly,main node distributes attribute relevance analysis task,then each sub-node finds out irrelevant attributes of data set in parallel,and these attributes are returned to the main node.The irrelevant attributes are removed by the main node.Secondly,the main node assigns search task,and each sub-node takes particle swarm optimization algorithm to search local outlier spaces in parallel.The main node works out the outlier spaces to establish the global outliers.In the end,the experimental results validate the accuracy and the effectiveness of the algorithm by using star spectrum data set in parallel computing environment.