Mining spatial association rules in image databases

In this paper, we propose a novel spatial mining algorithm, called 9DLT-Miner, to mine the spatial association rules from an image database, where every image is represented by the 9DLT representation. The proposed method consists of two phases. First, we find all frequent patterns of length one. Next, we use frequent k-patterns (k ? 1) to generate all candidate (k + 1)-patterns. For each candidate pattern generated, we scan the database to count the pattern’s support and check if it is frequent. The steps in the second phase are repeated until no more frequent patterns can be found. Since our proposed algorithm prunes most of impossible candidates, it is more efficient than the Apriori algorithm. The experiment results show that 9DLT-Miner runs 2-5 times faster than the Apriori algorithm.     

基于项编码的分布式频繁项集挖掘算法

Apriori算法是解决频繁项集挖掘最常用的算法之一,但多轮迭代扫描完整数据集的计算方式,严重影响算法效率且难以并行化处理。随着数据规模的持续增大,这一问题日益严重。针对这一问题,提出了一种基于项编码和Spark计算框架的Apriori并行化处理方法——IEBDA算法,利用项编码完整保存项集信息,在不重复扫描完整数据集的情况下完成频繁项集挖掘,同时利用Spark的广播变量实现并行化处理。与其他分布式Apriori算法在不同规模的数据集上进行性能比较,发现IEBDA算法从第一轮迭代后加速效果明显。结果表明,该算法可以提高大数据环境下的多轮迭代的频繁项集挖掘效率。     

HFIM: a Spark-based hybrid frequent itemset mining algorithm for big data processing

Frequent itemset mining is one of the data mining techniques applied to discover frequent patterns, used in prediction, association rule mining, classification, etc. Apriori algorithm is an iterative algorithm, which is used to find frequent itemsets from transactional dataset. It scans complete dataset in each iteration to generate the large frequent itemsets of different cardinality, which seems better for small data but not feasible for big data. The MapReduce framework provides the distributed environment to run the Apriori on big transactional data. However, MapReduce is not suitable for iterative process and declines the performance. We introduce a novel algorithm named Hybrid Frequent Itemset Mining (HFIM), which utilizes the vertical layout of dataset to solve the problem of scanning the dataset in each iteration. Vertical dataset carries information to find support of each itemsets. Moreover, we also include some enhancements to reduce number of candidate itemsets. The proposed algorithm is implemented over Spark framework, which incorporates the concept of resilient distributed datasets and performs in-memory processing to optimize the execution time of operation. We compare the performance of HFIM with another Spark-based implementation of Apriori algorithm for various datasets. Experimental results show that the HFIM performs better in terms of execution time and space consumption.     

New and efficient knowledge discovery of partial periodic patterns with multiple minimum supports

The problem of mining partial periodic patterns is an important issue with many applications. Previous studies to find these patterns encounter efficiency and effectiveness problem. The efficiency problem is that most previous methods were proposed to find frequent partial periodic patterns by extending the well-known Apriori-like algorithm. However, these methods generate many candidate partial periodic patterns to calculate the patterns’ supports, spending much time for discovering patterns. The effective problem is that only one minimum support threshold is set to find frequent partial periodic patterns but the results is not practical for real-world. In real-life circumstances, some rare or specific events may occur with lower frequencies but their occurrences may offer some vital information to be referred in decision making. If the minimum support is set too high, the associations between events along with higher and lower frequencies cannot be evaluated so that significant knowledge will be ignored. In this study, an algorithm to overcome these two problems has been proposed to generating redundant candidate patterns and setting only one minimum support threshold. The algorithm greatly improves the efficiency and effectiveness. First, it eliminates the need to generate numerous candidate partial periodic patterns thus reducing database scanning. Second, the minimum support threshold of each event can be specified based in its real-life occurring frequency.     

基于BIPL的Web频繁访问模式挖掘

挖掘频繁访问模式是Web日志挖掘的一个重要任务。针对类Apriori算法和GITC算法的不足,提出了基于双亲链的单次扫描求交的Web频繁访问模式挖掘算法—BIPL,该算法首先对用户的访问模式两两进行交集运算,生成候选访问模式,并在求交集过程中保存各个候选访问模式的双亲模式,然后通过简单的求和运算,计算出各个候选访问模式的支持数。最后通过理论分析和实验验证,该算法是稳定的和高效的。     

FP-growth算法的实现方法研究

事务数据库中频繁模式的挖掘研究作为关联规则等许多数据挖掘问题的核心工作,已经研究了许多年。早期算法大都是Apriori型算法,即首先产生候选集,然后在候选集的基础上找出频繁模式,候选集的产生往往是耗时的,特别是挖掘富模式或长模式时。JianweiHan等人提出了一种新颖的数据结构FP-tree及基于其上的FP-growth算法,用于有效的富模式与长模式挖掘。由于不同的实现方法可能会导致不同的挖掘效率,该文在讨论FP-growth算法的基础上,采用了几种不同的方法来实现它,并用几个数据库对它们的性能进行了比较。     

挖掘关联规则中Apriori算法的改进与优化

在所有的关联规则的挖掘算法中Apriori算法是最为经典的一个,但Apriofi算法有两个缺陷,即要扫描多次数据库以及生成大量的候选集。本文提出一种利用对项进行编码的方法对该算法进行改进,通过对项编码来减少扫描数据库次数并通过删除项来减少生成候选集的数量,从而提高算法的效率。相同条件下的实验结果表明,该优化后的算法能有效地提高关联规则挖掘的效率。     

考虑多粒度属性约简的关联规则挖掘研究

大数据时代，人们获取所需信息的困难度提高，而数据挖掘是当下解决此问题的关键技术。Apriori算法作为数据挖掘中的常用算法，通过挖掘数据背后的潜在关联规则。考虑到传统Apriori算法执行过程中，数据扫描频繁、候选集获取繁琐等问题，提出采用加权Apriori算法，即将冗余记录存储一次，并将记录的重复次数占全部记录数的比值作为权重，压缩空间；采用二进制的布尔矩阵替代原有数据集，通过矩阵内部“与运算”，获取最大频繁集，降低时间复杂度。考虑到原始数据冗余性以及粗糙集属性约简的不精确性，在提取关联规则前，提出采用多粒度粗糙集的属性约简算法，通过知识粒度细化属性值来提高约简精度，降低空间复杂度。最后，将所提方法与基于频繁矩阵的Apriori算法以及原始Apriori算法进行比较，验证所提方法的实用性和有效性。     

基于FC—tree的频繁闭项目集挖掘算法

目前提出的频繁项目集挖掘算法大多基于Apriori算法思想,但这类算法会产生巨大的候选集并且重复扫描数据库.本文针对这一问题,给出了一种基于FC-tree的频繁闭项目集挖掘算法Max-FCIA,该算法将频繁项目集存储在哈希表中,节省了程序的搜索时间.此外,利用广度优先搜索和有效的剪枝策略,大大限制了候选项目集的生成,缩小了搜索空间从而提高了程序的性能.实验结果表明该算法是快速有效的.     

Frequent itemset mining using cellular learning automata

A core issue of the association rule extracting process in the data mining field is to find the frequent patterns in the database of operational transactions. If these patterns discovered, the decision making process and determining strategies in organizations will be accomplished with greater precision. Frequent pattern is a pattern seen in a significant number of transactions. Due to the properties of these data models which are unlimited and high-speed production, these data could not be stored in memory and for this reason it is necessary to develop techniques that enable them to be processed online and find repetitive patterns. Several mining methods have been proposed in the literature which attempt to efficiently extract a complete or a closed set of different types of frequent patterns from a dataset. In this paper, a method underpinned upon Cellular Learning Automata (CLA) is presented for mining frequent itemsets. The proposed method is compared with Apriori, FP-Growth and BitTable methods and it is ultimately concluded that the frequent itemset mining could be achieved in less running time. The experiments are conducted on several experimental data sets with different amounts of minsup for all the algorithms as well as the presented method individually. Eventually the results prod to the effectiveness of the proposed method.     

Mining frequent patterns in image databases with 9D-SPA representation

In this paper, we propose a novel algorithm, called 9DSPA-Miner, to mine frequent patterns from an image database, where every image is represented by the 9D-SPA representation. Our proposed method consists of three phases. First, we scan the database once and create an index structure. Next, the index structure is scanned to find all frequent patterns of length two. Finally, we use the frequent k-patterns (k ? 2) to generate candidate (k + 1)-patterns and check if the support of each candidate generated is not less than the user-specified minimum support threshold by using the index structure. Then, the steps in the third phase are repeated until no more frequent patterns can be found. Since the 9DSPA-Miner algorithm uses the characteristics of the 9D-SPA representation to prune most of impossible candidates, the experiment results demonstrate that it is more efficient and scalable than the modified Apriori method.     

分布环境中的并行频繁模式挖掘算法

频繁模式的并行挖掘算法是数据挖掘中重要的研究课题。目前已经提出的并行算法大多是基于Apriori或基于FP-tree。由于两者的固有局限性,而且在计算过程中需要多次同步,因而具有较低的性能。文章提出了一种基于分布数据库的并行挖掘算法。该算法尽可能地让每个处理器独立地挖掘,每个处理器基于前缀树采用深度优先搜索的策略挖掘局部频繁模式集,并通过相关性质尽量减少候选全局频繁模式的规模,减少网络的通信量和同步次数以提高挖掘效率。     

基于频繁2项集支持矩阵的Apriori改进算法

捕要：Apriori算法在关联规则挖掘过程中需要多次扫描事务数据库,产生大量候选项目集,导致计算量过大。为解决该问题,提出一种基于频繁2项集支持矩阵的Apriori改进算法,通过分析频繁k＋1项集的生成机制,将支持矩阵与频繁2项集矩阵相结合实现快速剪枝,并大幅减少频繁k项集验证的计算量。实验结果表明,与Apriori算法和ABTM算法相比,改进算法明显提高了频繁项集的挖掘效率。     

Incremental sequence-based frequent query pattern mining from XML queries

Existing algorithms of mining frequent XML query patterns (XQPs) employ a candidate generate-and-test strategy. They involve expensive candidate enumeration and costly tree-containment checking. Further, most of existing methods compute the frequencies of candidate query patterns from scratch periodically by checking the entire transaction database, which consists of XQPs transferred from user query logs. However, it is not straightforward to maintain such discovered frequent patterns in real XML databases as there may be frequent updates that may not only invalidate some existing frequent query patterns but also generate some new frequent query patterns. Therefore, a drawback of existing methods is that they are rather inefficient for the evolution of transaction databases. To address above-mentioned problems, this paper proposes an efficient algorithm ESPRIT to mine frequent XQPs without costly tree-containment checking. ESPRIT transforms XML queries into sequences using a one-to-one mapping technique and mines the frequent sequences to generate frequent XQPs. We propose two efficient incremental algorithms, ESPRIT-i and ESPRIT-i ⁺, to incrementally mine frequent XQPs. We devise several novel optimization techniques of query rewriting, cache lookup, and cache replacement to improve the answerability and the hit rate of caching. We have implemented our algorithms and conducted a set of experimental studies on various datasets. The experimental results demonstrate that our algorithms achieve high efficiency and scalability and outperform state-of-the-art methods significantly.     

一种基于数据压缩的Apriori算法

随着物联网技术的飞速发展,数据采集手段迅速增加,对海量数据分析与处理的需求也愈加强烈。关联规则挖掘算法通过数据之间的关联分析,挖掘出数据之间的隐含关系,进而获得了大量应用。在众多的关联规则算法中,传统的Apriori算法虽然得到了大量应用,但是因为该算法产生大量的候选集,而且需要多次对数据库进行扫描,导致该算法的运行效率大大降低。为了克服Apriori算法的以上缺点,通过数据压缩的方法减少了数据库扫描次数的同时,对生成的候选集进行了多次验证,大大减少了无效候选集的数量。大量的数据挖掘实验证明提出的改进算法可以在正确挖掘数据集关联规则的同时,大大提高了算法的运行效率。     

基于改进FP-树的最大模式挖掘算法

频繁模式挖掘是数据挖掘领域中的一个非常重要的分支,但是由于其内在的计算复杂性,挖掘密集型数据的频繁模式完全集非常困难而且数量往往大得惊人,难以理解和应用。最大频繁模式(最大模式)压缩隐含了所有的频繁模式,存储所占用的空间远远小于完全集,因而最大模式挖掘具有十分重要的意义。该文改进了传统的FP-树结构并提出了一种有效的基于改进FP-树的最大模式挖掘算法IFP-M ax;通过引入后缀子树的概念,算法在挖掘过程中不用生成最大频繁模式候选集,从而大大提高了算法的时间效率和空间可伸缩性。实验表明,IFP-M ax的挖掘速度比M AFIA和GenM ax大约快一个数量级。     

Apriori算法的三种优化方法

通过对Apriori算法的思想和性能的分析,认为Apriori算法存在以下三点不足:(1)由K阶频繁集生成K+1阶候选频繁集时,在K+1阶候选频繁集中过滤掉非频繁集的策略值得进一步改进;(2)连接程序中相同的项目重复比较太多,因而其效率值得进一步改进;(3)在回扫数据库时有许多不必比较的项目或事务重复比较。根据上述三点不足,提出了相应的三种优化策略来优化Apriori算法,得到一效率较高的改进Apriori算法。     

关联规则挖掘中Apriori算法的研究与改进

经典的产生频繁项目集的Apriori算法存在多次扫描数据库可能产生大量候选及反复对候选项集和事务进行模式匹配的缺陷,导致了算法的效率较低。为此,对Apriori算法进行以下3方面的改进:改进由k阶频繁项集生成k+1阶候选频繁项集时的连接和剪枝策略;改进对事务的处理方式,减少Apriori算法中的模式匹配所需的时间开销;改进首次对数据库的处理方法,使得整个算法只扫描一次数据库,并由此提出了改进算法。实验结果表明,改进算法在性能上得到了明显提高。     

快速挖掘全局频繁项目集

分布式环境中，全局频繁项目集的挖掘是数据挖掘中最重要的研究课题之一．传统的全局频繁项目集挖掘算法采用Apriori算法框架，须多遍扫描数据库并产生大量的候选项目集，且通过传送局部频繁项目集求全局频繁项目集的网络通信代价高．为此，提出了一种分布数据库的全局频繁项目集快速挖掘算法——FMAGF.FMAGF算法采用传送条件频繁模式树或条件模式基来挖掘全局频繁项目集，可有效地减小网络通信量，提高全局频繁项目集挖掘效率．理论分析和实验结果表明提出的算法是有效可行的．     

Mining method of massive data of mobile library under information asymmetry facing large-scare database

The current method generates a large number of candidate sets when mining data in mobile libraries under asymmetric information, and the mining time and efficiency are poor. To this end, a new method for mobile library massive data mining based on improved Apriori algorithm is proposed to collect, clean and reduce massive data. Calculate the reader's interest distance by analyzing the borrowed historical data, and use the Apriori algorithm to find the association rules in the frequent itemsets of the data. In order to make up for the shortcomings of the current method, while filtering out infrequent candidate sets, the corresponding transaction set is also collaboratively filtered, which can reduce the amount of calculation and time consumption. Experimental results show that the proposed method can mine more valuable rules. The improved execution time is only 10 s, the CPU utilization exceeds 90%, and the acceleration ratio exceeds 1.81 s, which can better meet the needs of decision makers.