基于垂直数据分布的大型稠密数据库快速关联规则挖掘算法

为进一步解决对大型事务数据库进行关联规则挖掘时产生的CPU时间开销大和I/O操作频繁的问题,给出了一种基于垂直数据分布的改进关联规则挖掘算法,称为VARMLDb算法。该算法首先有效地把数据库分为内存可以满足要求的若干划分,然后结合有向无环图和垂直数据形式diffse、差集来存储和计算频繁项集,极大地减少了存储中间结果所需的内存大小,解决了传统垂直数据挖掘算法对稠密数据库挖掘效率低下的问题,使该算法可有效地适用于大型稠密数据库的关联规则挖掘。整个算法吸取CARMA算法的优势,只需扫描两次数据库便可完成挖掘过程。实验结果表明该算法是正确的,在大型稠密数据库中,VARMLDb算法具有较高的执行效率。     

基于项编码的关联规则挖掘算法

关联规则挖掘是发现大量数据中项集之间有趣的关联或相关联系的技术方法,关联规则挖掘Apriori算法需要多次扫描数据库,时空复杂度过高。针对该算法的局限性,本文提出了基于项编码的关联规则挖掘算法CA（Coding-based Apriori）,只需要第一遍扫描数据库并对每个项完成编码,以后的过程都是针对编码进行,不需要多次扫描数据库。相同条件下的实验结果表明,优化后的算法能有效地提高关联规则挖掘的效率。     

一种改进的Apriori算法

数据挖掘中的关联规则挖掘能够发现大量数据中项集之间有趣的关联或相关联系,特别是随着大量数据不停地收集和存储,从数据库中挖掘关联规则就越来越有其必要性。通过对关联规则挖掘技术及其相关算法Apriori进行分析,发现该技术存在的问题。Apriori算法是关联规则挖掘中的经典算法。对Apriori算法做了改进。借助0—1矩阵给出了计算项集的支持度计数的更快方法,同时还简化了Apriori算法中的连接和剪枝操作,从而在时间和空间上提高了Apriori算法的效率。     

一种改进的Apriori算法

数据挖掘中的关联规则挖掘能够发现大量数据中项集之间有趣的关联或相关联系,特别是随着大量数据不停地收集和存储,从数据库中挖掘关联规则就越来越有其必要性.通过对关联规则挖掘技术及其相关算法Apaod进行分析,发现该技术存在的问题.Apriori算法是关联规则挖掘中的经典算法.对Apriori算法做了改进.借助0-1矩阵给出了计算项集的支持度计数的更快方法,同时还简化了Apriori算法中的连接和剪枝操作,从而在时间和空间上提高了Apriori算法的效率.     

Web日志中有趣关联规则的发现

关联规则挖掘是Web用法挖掘的一个重要研究课题。目前的Web日志关联规则挖掘算法忽略了用户对规则是否感兴趣这一重要问题。对Web日志关联规则挖掘算法进行了研究，结合网络拓扑结构，提出了Web拓扑概率模型和有趣关联规则（IAR）算法。利用Web拓扑概率模型对关联规则进行有趣度评价，得出有趣度高的规则，用于改善网络性能。实验显示了IAR算法如何提高规则的利用率和有效地改善网络拓扑，它可以成功地应用到Web用法挖掘中。     

基于片上多核的频繁项集并行挖掘算法

关联规则挖掘中最主要的工作是如何高效地挖掘频繁项集。目前在单机平台上,由于计算量大等原因,大数据集上的关联规则挖掘很难得到理想结果。在分析现有频繁项集挖掘算法的基础上,结合Eclat和dEclat挖掘算法优点,针对大数据集和片上多核共享内存计算环境,提出一种高效的并行频繁项集挖掘算法PEclat,算法实现了任务级并行挖掘频繁项集,并在大数据集上进行了多项测试。实验结果表明,无论数据稠密程度如何,该算法均能取得较好的性能。     

旅游信息数据挖掘中数量型数据的转化和分析研究

数据挖掘就是从大量数据中,抽取隐含的、但又是潜在有用的关联信息和知识的发现过程。其中关联规则(AssociationRules)的挖掘是一个重要的问题,是大量数据中项集之间有趣的关联或相关联系。本文论述了如何改进关联规则算法,使之可以挖掘数量型数据,并利用改进的关联规则算法对游客数据进行数据挖掘,从而得到一些对旅游经营有用的知识。     

一种改进的Apriori算法

关联规则挖掘是数据挖掘研究领域中的一个重要任务,旨在挖掘事务数据库中有趣的关联。Apriori算法是关联规则挖掘中的经典算法。然而Apriori算法存在着产生候选项目集效率低和频繁扫描数据等缺点。提出了一种新的Apriori的改进算法,该算法在生成k(k>1)项频繁集时,不需要重新扫描数据库,只是在生成1项频集时,才需要扫描事务数据库,有效地减少了对事务数据库的读操作,在时间复杂度上较经典的Apriori算法有更加优越的性能。     

一种基于BUC的水平加权关联规则挖掘算法

关联规则挖掘可以从大量数据中发现项集间潜在而有趣的相互联系。针对用户对每个项目感兴趣的程度不同,一些学者提出了水平加权关联规则。然而每次生成新候选集后对整个数据库事物的扫描成为算法效率的一大瓶颈。为进一步提高加权关联规则的挖掘效率,在原有的水平加权关联规则算法的基础上,采取了深度优先的策略,提出了一种基于BUC的水平加权关联规则挖掘算法——BUc—MINWAL。改进算法可以大大减少对数据库的扫描范围。实验结果表明,改进的算法有更好的执行效率。     

一种基于矩阵的多值关联规则的挖掘算法

关联规则是数据挖掘研究的主要模式之一,其中布尔型关联规则的挖掘已经有比较成熟的系统和方法,而多值关联规则的挖掘则不然。本文提出的QARMM算法利用矩阵存储数据,将频繁项目集的产生过程转化为项目集的关系矩阵中向量的运算过程,同时克服了SLIG算法和矩阵算法不能挖掘多值关联规则的弱点,只需运行一次便可挖掘出所有关联规则。实验证明,在等价的数据集上挖掘关联规则,QARMM算法比Apriori算法具有更高的效率。     

空间关联规则的增量维护

为了得到有趣且有效的空间关联规则通常需要多次执行挖掘操作,可以使用增量维护算法来提高挖掘效率。然而,能够直接使用空间数据的关联规则增量更新算法尚属空白。为解决这一问题,对挖掘阈值改变和空间数据集更新后通过筛选或增量挖掘等方法实现规则维护的策略进行了分析,并提出适用于支持度阈值减小和空间图层增加这两类情况的增量挖掘算法——ISA。ISA算法不依赖于空间事务表的构建与更新,可以直接使用空间图层作为输入数据。在基于实际数据的实验中,采用ISA算法所得结果与类Apriori算法一致,耗时则相对缩短20.0%至71.0%;此外,对1372772条规则进行了基于筛选的更新,耗时低于0.1s。实验结果表明,所提出的空间关联规则增量维护策略和算法是可行、正确且高效的。     

基于Apriori算法的水平加权关联规则挖掘

关联规则挖掘可以发现大量数据中项集之间有趣的关联或相关联系,并已在许多领域得到了广泛的应用。目前业界已经提出了许多发现关联规则的算法,这些算法都认为每个数据对规则的重要性相同。但在实际应用中,用户会比较倾向于自己最感兴趣或认为最重要的那部分项目,因此有必要加强这些项目对规则的影响,同时减弱另一些用户兴趣不大或认为不重要的项目对规则的影响。为此,论文提出了水平加权关联规则的问题,并结合Apriori算法,加以改进,给出了关于该问题的解决方案及有效算法New_Apriori。     

PARM—An Efficient Algorithm to Mine Association Rules From Spatial Data

Association rule mining, originally proposed for market basket data, has potential applications in many areas. Spatial data, such as remote sensed imagery (RSI) data, is one of the promising application areas. Extracting interesting patterns and rules from spatial data sets, composed of images and associated ground data, can be of importance in precision agriculture, resource discovery, and other areas. However, in most cases, the sizes of the spatial data sets are too large to be mined in a reasonable amount of time using existing algorithms. In this paper, we propose an efficient approach to derive association rules from spatial data using Peano Count Tree (P-tree) structure. P-tree structure provides a lossless and compressed representation of spatial data. Based on P-trees, an efficient association rule mining algorithm PARM with fast support calculation and significant pruning techniques is introduced to improve the efficiency of the rule mining process. The P-tree based Association Rule Mining (PARM) algorithm is implemented and compared with FP-growth and Apriori algorithms. Experimental results showed that our algorithm is superior for association rule mining on RSI spatial data.      

Mining potentially more interesting association rules with fuzzy interest measure

The association rules, discovered by traditional support–confidence based algorithms, provide us with concise statements of potentially useful information hidden in databases. However, only considering the constraints of minimum support and minimum confidence is far from satisfying in many cases. In this paper, we propose a fuzzy method to formulate how interesting an association rule may be. It is indicated by the membership values belonging to two fuzzy sets (i.e., the stronger rule set and the weaker rule set), and thus provides much more flexibility than traditional methods to discover some potentially more interesting association rules. Furthermore, revised algorithms based on Apriori algorithm and matrix structure are designed under this framework.     

Mining data streams with concept drifts using genetic algorithm

Recent research shows that rule based models perform well while classifying large data sets such as data streams with concept drifts. A genetic algorithm is a strong rule based classification algorithm which is used only for mining static small data sets. If the genetic algorithm can be made scalable and adaptable by reducing its I/O intensity, it will become an efficient and effective tool for mining large data sets like data streams. In this paper a scalable and adaptable online genetic algorithm is proposed to mine classification rules for the data streams with concept drifts. Since the data streams are generated continuously in a rapid rate, the proposed method does not use a fixed static data set for fitness calculation. Instead, it extracts a small snapshot of the training example from the current part of data stream whenever data is required for the fitness calculation. The proposed method also builds rules for all the classes separately in a parallel independent iterative manner. This makes the proposed method scalable to the data streams and also adaptable to the concept drifts that occur in the data stream in a fast and more natural way without storing the whole stream or a part of the stream in a compressed form as done by the other rule based algorithms. The results of the proposed method are comparable with the other standard methods which are used for mining the data streams.     

Mining optimized support rules for numeric attributes

Mining association rules on large data sets have received considerable attention in recent years. Association rules are useful for determining correlations between attributes of a relation and have applications in marketing, financial and retail sectors. Furthermore, optimized association rules are an effective way to focus on the most interesting characteristics involving certain attributes. Optimized association rules are permitted to contain uninstantiated attributes and the problem is to determine instantiations such that either the support, confidence or gain of the rule is maximized. In this paper, we generalize the optimized support association rule problem by permitting rules to contain disjunctions over uninstantiated numeric attributes. Our generalized association rules enable us to extract more useful information about seasonal and local patterns involving the uninstantiated attribute. For rules containing a single numeric attribute, we present a dynamic programming algorithm for computing optimized association rules. Furthermore, we propose bucketing technique for reducing the input size, and a divide and conquer strategy that improves the performance significantly without sacrificing optimality. We also present approximation algorithms based on dynamic programming for two numeric attributes. Our experimental results for a single numeric attribute indicate that our bucketing and divide and conquer enhancements are very effective in reducing the execution times and memory requirements of our dynamic programming algorithm. Furthermore, they show that our algorithms scale up almost linearly with the attribute's domain size as well as the number of disjunctions.     

A greedy classification algorithm based on association rule

Classification and association rule discovery are important data mining tasks. Using association rule discovery to construct classification systems, also known as associative classification, is a promising approach. In this paper, a new associative classification technique, Ranked Multilabel Rule (RMR) algorithm is introduced, which generates rules with multiple labels. Rules derived by current associative classification algorithms overlap in their training objects, resulting in many redundant and useless rules. However, the proposed algorithm resolves the overlapping between rules in the classifier by generating rules that does not share training objects during the training phase, resulting in a more accurate classifier. Results obtained from experimenting on 20 binary, multi-class and multi-label data sets show that the proposed technique is able to produce classifiers that contain rules associated with multiple classes. Furthermore, the results reveal that removing overlapping of training objects between the derived rules produces highly competitive classifiers if compared with those extracted by decision trees and other associative classification techniques, with respect to error rate.     

Fast algorithms for frequent itemset mining using FP-trees

Efficient algorithms for mining frequent itemsets are crucial for mining association rules as well as for many other data mining tasks. Methods for mining frequent itemsets have been implemented using a prefix-tree structure, known as an FP-tree, for storing compressed information about frequent itemsets. Numerous experimental results have demonstrated that these algorithms perform extremely well. In this paper, we present a novel FP-array technique that greatly reduces the need to traverse FP-trees, thus obtaining significantly improved performance for FP-tree-based algorithms. Our technique works especially well for sparse data sets. Furthermore, we present new algorithms for mining all, maximal, and closed frequent itemsets. Our algorithms use the FP-tree data structure in combination with the FP-array technique efficiently and incorporate various optimization techniques. We also present experimental results comparing our methods with existing algorithms. The results show that our methods are the fastest for many cases. Even though the algorithms consume much memory when the data sets are sparse, they are still the fastest ones when the minimum support is low. Moreover, they are always among the fastest algorithms and consume less memory than other methods when the data sets are dense.