云平台下基于粗糙集的并行增量知识更新算法

日益复杂和动态变化的海量数据处理,是当前人们普遍关注的问题,其核心内容之一是研究如何利用已有的信息实现快速的知识更新.粒计算是近年来新兴的一个研究领域,是信息处理的一种新的概念和计算范式,主要用于描述和处理不确定的、模糊的、不完整的和海量的信息,以及提供一种基于粒与粒间关系的问题求解方法.作为粒计算理论中的一个重要组成部分,粗糙集是一种处理不确定性和不精确性问题的有效数学工具.根据云计算中的并行模型MapReduce,给出了并行计算粗糙集中等价类、决策类和两者之间相关性的算法;然后,设计了用于处理大规模数据的并行粗糙近似集求解算法.为应对动态变化的海量数据,结合MapReduce模型和增量更新方法,根据不同的增量策略,设计了两种并行增量更新粗糙近似集的算法.实验结果表明,该算法可以有效地快速更新知识;而且数据量越大,效果越明显.

Parallel and Incremental Algorithm for Knowledge Update Based on Rough Sets in Cloud Platform

The increasing complexity and dynamic change of massive data processing currently receive widespread attention. One of its core content is to study how to use the existing information to achieve rapid updating of knowledge. Granular computing (GrC), a new computing paradigm of information processing, is an emerging research field which is mainly used to describe and deal with uncertain, vague, incomplete and massive data, and provides a solution based on the granularity and the relationship between the granularities. As an important part of GrC, rough set theory is an effective mathematical tool to deal with the uncertainty and imprecise problems. Based on the MapReduce model in cloud computing, this paper first presents a parallel algorithm for computing the equivalence classes, decision classes and the association between them in rough set theory. A parallel algorithm is then designed for computing rough set approximations from large-scale data. To adapt to the dynamic real-time system, the MapReduce model and incremental method are combined to build two parallel incremental algorithms for updating rough set approximations in different incremental strategies. An extensive experimental evaluation on big data sets show that the proposed algorithms are very effective and have better performance with the increasing size of the data.