Fuzzy OLAP association rules mining-based modular reinforcement learning approach for multiagent systems.

Multiagent systems and data mining have recently attracted considerable attention in the field of computing. Reinforcement learning is the most commonly used learning process for multiagent systems. However, it still has some drawbacks, including modeling other learning agents present in the domain as part of the state of the environment, and some states are experienced much less than others, or some state-action pairs are never visited during the learning phase. Further, before completing the learning process, an agent cannot exhibit a certain behavior in some states that may be experienced sufficiently. In this study, we propose a novel multiagent learning approach to handle these problems. Our approach is based on utilizing the mining process for modular cooperative learning systems. It incorporates fuzziness and online analytical processing (OLAP) based mining to effectively process the information reported by agents. First, we describe a fuzzy data cube OLAP architecture which facilitates effective storage and processing of the state information reported by agents. This way, the action of the other agent, not even in the visual environment. of the agent under consideration, can simply be predicted by extracting online association rules, a well-known data mining technique, from the constructed data cube. Second, we present a new action selection model, which is also based on association rules mining. Finally, we generalize not sufficiently experienced states, by mining multilevel association rules from the proposed fuzzy data cube. Experimental results obtained on two different versions of a well-known pursuit domain show the robustness and effectiveness of the proposed fuzzy OLAP mining based modular learning approach. Finally, we tested the scalability of the approach presented in this paper and compared it with our previous work on modular-fuzzy Q-learning and ordinary Q-learning.     

Flexible online association rule mining based on multidimensional pattern relations

Most incremental mining and online mining algorithms concentrate on finding association rules or patterns consistent with entire current sets of data. Users cannot easily obtain results from only interesting portion of data. This may prevent the usage of mining from online decision support for multidimensional data. To provide ad-hoc, query-driven, and online mining support, we first propose a relation called the multidimensional pattern relation to structurally and systematically store context and mining information for later analysis. Each tuple in the relation comes from an inserted dataset in the database. We then develop an online mining approach called three-phase online association rule mining (TOARM) based on this proposed multidimensional pattern relation to support online generation of association rules under multidimensional considerations. The TOARM approach consists of three phases during which final sets of patterns satisfying various mining requests are found. It first selects and integrates related mining information in the multidimensional pattern relation, and then if necessary, re-processes itemsets without sufficient information against the underlying datasets. Some implementation considerations for the algorithm are also stated in detail. Experiments on homogeneous and heterogeneous datasets were made and the results show the effectiveness of the proposed approach.     

基于数据立方体的多维关联规则的挖掘方法

总结了现有基于数据立方体的多维关联规则挖掘算法，在此基础上将联机分析处理(OLAP)的钻取操作引入关联规则挖掘过程，提出Apriori_cubc算法的改进算法。通过动态调整立方体的维层次，来挖掘出用户感兴趣的关联规则。     

High Performance OLAP and Data Mining on Parallel Computers

On-Line Analytical Processing (OLAP) techniques are increasingly being used in decision support systems to provide analysis of data. Queries posed on such systems are quite complex and require different views of data. Analytical models need to capture the multidimensionality of the underlying data, a task for which multidimensional databases are well suited. Multidimensional OLAP systems store data in multidimensional arrays on which analytical operations are performed. Knowledge discovery and data mining requires complex operations on the underlying data which can be very expensive in terms of computation time. High performance parallel systems can reduce this analysis time. Precomputed aggregate calculations in a Data Cube can provide efficient query processing for OLAP applications. In this article, we present algorithms for construction of data cubes on distributed-memory parallel computers. Data is loaded from a relational database into a multidimensional array. We present two methods, sort-based and hash-based for loading the base cube and compare their performances. Data cubes are used to perform consolidation queries used in roll-up operations using dimension hierarchies. Finally, we show how data cubes are used for data mining using Attribute Focusing techniques. We present results for these on the IBM-SP2 parallel machine. Results show that our algorithms and techniques for OLAP and data mining on parallel systems are scalable to a large number of processors, providing a high performance platform for such applications.     

关联规则挖掘的基本算法

介绍了加权模糊关联规则挖掘算法的基本思想及实现步骤，并给出挖掘算法的多种策略。在此基础上，分析了加权模糊关联规则与模糊关联规则、布尔型属性加权关联规则、布尔型属性关联规则之间的内在联系，并指出加权模糊关联规则挖掘算法是一种最基本的关联规则挖掘算法，蕴涵了其它3种关联规则挖掘算法。     

A Complexity Guided Algorithm for Association Rule Extraction on Fuzzy DataCubes

The use of online analytical processing (OLAP) systems as data sources for data mining techniques has been widely studied and has resulted in what is known as online analytical mining (OLAM). As a result of both the use of OLAP technology in new fields of knowledge and the merging of data from different sources, it has become necessary for models to support imprecision. We, therefore, need OLAM methods which are able to deal with this imprecision. Association rules are one of the most used data mining techniques. There are several proposals that enable the extraction of association rules on DataCubes but few of these deal with imprecision in the process. The main problem observed in these proposals is the complexity of the rule set obtained. In this paper, we present a novel association rule extraction method that works over a fuzzy multidimensional model which is capable of representing and managing imprecise data. Our method deals with the problem of reducing the complexity of the result obtained by using fuzzy concepts and a hierarchical relation between them.     

PARSIMONY: An Infrastructure for Parallel Multidimensional Analysis and Data Mining

Multidimensional analysis and online analytical processing (OLAP) operations require summary information on multidimensional data sets. Most common are aggregate operations along one or more dimensions of numerical data values. Simultaneous calculation of multidimensional aggregates are provided by the Data Cube operator, used to calculate and store summary information on a number of dimensions. This is computed only partially if the number of dimensions is large. Query processing for these applications requires different views of data to gain insight and for effective decision support. Queries may either be answered from a materialized cube in the data cube or calculated on the fly.  The multidimensionality of the underlying problem can be represented both in relational and in multidimensional databases, the latter being a better fit when query performance is the criteria for judgment. Relational databases are scalable in size for OLAP and multidimensional analysis and efforts are on to make their performance acceptable. On the other hand multidimensional databases have proven to provide good performance for such queries, although they are not very scalable. In this article we address (1) scalability in multidimensional systems for OLAP and multidimensional analysis and (2) integration of data mining with the OLAP framework. We describe our system PARSIMONY, parallel and scalable infrastructure for multidimensional online analytical processing, used for both OLAP and data mining. Sparsity of data sets is handled by using chunks to store data either as a dense block using multidimensional arrays or as sparse representation using a bit encoded sparse structure. Chunks provide a multidimensional index structure for efficient dimension oriented data accesses much the same as multidimensional arrays do. Operations within chunks and between chunks are a combination of relational and multidimensional operations depending on whether the chunk is sparse or dense. Further, we develop parallel algorithms for data mining on the multidimensional cube structure for attribute-oriented association rules and decision-tree-based classification. These take advantage of the data organization provided by the multidimensional data model.  Performance results for high dimensional data sets on a distributed memory parallel machine (IBM SP-2) show good speedup and scalability.     

OLAP中基于FP-增长的关联规则挖掘

关联规则挖掘是一种发现属性问关系的方法，主要用于在商务事务记录中挖掘事务问关系。本文将已经广泛使用的FP-增长(frequent-pattern growth，频繁模式增长)算法进行改进，实现了OLAP中的关联规则挖掘。改进算法分别针对单维、多维、混合维三种关联规则，将多维立方体转化成不同的关系表，通过关系表产生关联规则．并利用立方体中的事实值作为进一步约束，生成了更有价值的规则。     

基于数据仓库的联机分析挖掘系统

基于数据仓库的联机分析挖掘系统的应用已成为趋势，设计并实现了一个应用模型。该模型建立了一个B／S架构的双服务器数据仓库，将数据挖掘与OLAP思想相结合，提供多维度事务内、事务间关联规则的挖掘。介绍了系统设计思想和体系结构，对系统的各部分功能进行了分析，并在此基础上提出了传统OLAP系统到OLAM系统平滑过渡的解决方案。     

特殊网络通信行为的数据挖掘与分析

为了挖掘可疑通信的行为模式,定位发生了可疑通信行为的上网账户,本文首先分析了可疑通信行为特点。然后针对已有关联规则挖掘算法不能同时满足多层次数据挖掘和加权关联规则挖掘的问题,分析对比两种典型的基本关联规则算法,以FP-tree为基础,提出了ML-WFP多层次加权关联规则挖掘算法。针对算法中数据项权重的确定问题,由用户设置数据项间的重要性比较关系,借鉴模糊一致矩阵的概念,利用模糊层次分析法计算数据项的权重。最后将该算法应用于可疑通信行为的挖掘。实验测试结果表明可疑通信行为挖掘方案合理有效。     

基于数据立方体的关联规则挖掘及其实现

联机分析处理和数据挖掘是两种重要的数据分析方法。使用数据立方体作为数据存储结构,将两者集成起来,使得用户可以从不同角度、不同抽象层次分析数据。针对数据立方体的特点,本文提出了挖掘维间关联规则的算法,并编程实现了该算法,取得满意的结果。     

二维立方体中关联规则挖掘算法研究

针对二维数据立方体的结构特点 ,通过对传统的关联规则挖掘算法的改进 ,提出了一种二维立方体关联规则挖掘的新颖算法。该算法通过有效组织挖掘过程中的数据结构 ,降低对立方体的扫描次数 ,并充分利用联机分析处理技术 ,从而大大降低了执行时间 ,提高了执行效率     

OLAP关联规则挖掘

该文提出一种新的关联规则挖掘方法,OLAP关联规则挖掘。OLAP关联规则挖掘是OLAP技术和一些高效的关联规则挖掘算法的结合。OLAP关联规则挖掘方法是一种灵活的、多维的、多层次的高性能方法。该文首先介绍了O-LAP关联规则挖掘的结构,最后详述了OLAP关联规则挖掘的具体实现。     

模糊关联规则挖掘模型及其在分解炉中的应用

研究了模糊关联规则挖掘模型在分解炉中的应用,并阐述了模糊聚类算法和关联规则的相关内容,提出运用模糊聚类KFCM算法对实际数据进行计算,得到数据归属于不同类别的隶属度.同时,利用规则挖掘算法MFAR对模糊化的参数进行处理,得到了有价值的模糊规则,解决了实际中专家经验获取的瓶颈问题.试验表明,该方法为水泥生产环节中分解炉的温度控制提供了理论依据和生产优化指导.     

联机分析关联规则挖掘的研究

在对关联挖掘、数据仓库、OLAP研究的基础之上，提出了联机分析关联规则挖掘的方法，并给出了针对该方法的特定算法。研究表明，同孤立的关联规则挖掘方法相比，该方法具有较大的灵活性和更高的效率。     

Efficient aggregation algorithms for compressed data warehouses

Aggregation and cube are important operations for online analytical processing (OLAP). Many efficient algorithms to compute aggregation and cube for relational OLAP have been developed. Some work has been done on efficiently computing cube for multidimensional data warehouses that store data sets in multidimensional arrays rather than in tables. However, to our knowledge, there is nothing to date in the literature describing aggregation algorithms on compressed data warehouses for multidimensional OLAP. This paper presents a set of aggregation algorithms on compressed data warehouses for multidimensional OLAP. These algorithms operate directly on compressed data sets, which are compressed by the mapping-complete compression methods, without the need to first decompress them. The algorithms have different performance behaviors as a function of the data set parameters, sizes of outputs and main memory availability. The algorithms are described and the I/O and CPU cost functions are presented in this paper. A decision procedure to select the most efficient algorithm for a given aggregation request is also proposed. The analysis and experimental results show that the algorithms have better performance on sparse data than the previous aggregation algorithms     

基于多维标度的快速挖掘关联规则算法

挖掘关联规则是数据挖掘研究的一个重要方面．文章在分析其基本模型和研究多维标度基本性质的基础上,提出一个新的基于多维标度的挖掘关联规则算法．该算法以数据项间的关联度量为依据,将各个数据项投影到多维空间上,进行降维处理,最后将数据项集间的关联关系以可视结果提供给用户.     

OLAP数据挖掘引擎算法的设计与实现

基于OLAP的数据挖掘，是数据挖掘的一个新的发展方向。对于如何把OLAP(联机分析处理技术)和DM(数据挖掘)统一起来，从而在数据库或数据仓库的不同层次进行挖掘，提出了OLAP数据挖掘系统的结构。通过研究数据挖掘方法和OLAP操作的特点，以及数据立方的构建和物化，对传统的DM算法进行了改进，设计并实现了更能适应OLAP数据挖掘引擎的算法。     

数据立方梯度挖掘的研究

1 前言随着人们生成、收集和存储数字化数据能力的极大提高,当今世界面临着各种原始数据的爆炸性增长。数据库技术的巨大进步创建了对大量数据的有效存储,成千上万的大型数据库被广泛地应用在商业、政府和科研等等部门。大量数据资源的积累为人们从历史数据中发现有用信息提供了基础,人们期望数据库能够提供智能化或者至少是半自动化的数据分析处理的能力。于是,数据仓库技术(Data Warehouse)、联机分析处理技术(On Line Analysis Processing)以及数据挖掘技术(Data Mining)应运而生。     

Multi-objective genetic algorithms based automated clustering for fuzzy association rules mining

Researchers realized the importance of integrating fuzziness into association rules mining in databases with binary and quantitative attributes. However, most of the earlier algorithms proposed for fuzzy association rules mining either assume that fuzzy sets are given or employ a clustering algorithm, like CURE, to decide on fuzzy sets; for both cases the number of fuzzy sets is pre-specified. In this paper, we propose an automated method to decide on the number of fuzzy sets and for the autonomous mining of both fuzzy sets and fuzzy association rules. We achieve this by developing an automated clustering method based on multi-objective Genetic Algorithms (GA); the aim of the proposed approach is to automatically cluster values of a quantitative attribute in order to obtain large number of large itemsets in less time. We compare the proposed multi-objective GA based approach with two other approaches, namely: 1) CURE-based approach, which is known as one of the most efficient clustering algorithms; 2) Chien et al. clustering approach, which is an automatic interval partition method based on variation of density. Experimental results on 100 K transactions extracted from the adult data of USA census in year 2000 showed that the proposed automated clustering method exhibits good performance over both CURE-based approach and Chien et al.’s work in terms of runtime, number of large itemsets and number of association rules.