DBV-Miner: A Dynamic Bit-Vector approach for fast mining frequent closed itemsets

Frequent closed itemsets (FCI) play an important role in pruning redundant rules fast. Therefore, a lot of algorithms for mining FCI have been developed. Algorithms based on vertical data formats have some advantages in that they require scan databases once and compute the support of itemsets fast. Recent years, BitTable (Dong & Han, 2007) and IndexBitTable (Song, Yang, & Xu, 2008) approaches have been applied for mining frequent itemsets and results are significant. However, they always use a fixed size of Bit-Vector for each item (equal to number of transactions in a database). It leads to consume more memory for storage Bit-Vectors and the time for computing the intersection among Bit-Vectors. Besides, they only apply for mining frequent itemsets, algorithm for mining FCI based on BitTable is not proposed. This paper introduces a new method for mining FCI from transaction databases. Firstly, Dynamic Bit-Vector (DBV) approach will be presented and algorithms for fast computing the intersection between two DBVs are also proposed. Lookup table is used for fast computing the support (number of bits 1 in a DBV) of itemsets. Next, subsumption concept for memory and computing time saving will be discussed. Finally, an algorithm based on DBV and subsumption concept for mining frequent closed itemsets fast is proposed. We compare our method with CHARM, and recognize that the proposed algorithm is more efficient than CHARM in both the mining time and the memory usage.     

一种挖掘加权频繁项集的改进算法

分析了New-Apriori和MWFI（Mining Weighted Frequent Itemsets）算法之不足,提出了一种挖掘加权频繁项集的New-MWFI算法。该算法按属性的权值对事务进行分类,并依次求出每个类别内的加权频繁项集。由于每个类别内的频繁项集满足Apriori性质,因而可以利用Apriori算法或其他改进算法进行挖掘,从而克服了原来算法的不合理和效率低下的缺陷。实验表明该算法能更有效地从数据集中挖掘出加权频繁项集。     

基于二进制的频繁项集挖掘新算法

频繁项集挖掘是数据挖掘中的一个经典的问题。然而,大部分算法需要扫描数据库多次,算法效率比较低。该文提出了一个效率比较好的挖掘频繁项集的新算法,在这个算法中,所有的事务都是以二进制的形式表示,所以挖掘极大频繁项集的任务就变成了从二进制集中发现频繁模式。而且,这种算法只需要扫描原始数据库一次。最后,利用试验来证明这种算法的效率和优势。     

基于DDMINER分布式数据库系统中频繁项目集的更新

给出了一种分布式数据挖掘系统的体系结构DDMINER，对分布式数据库系统中频繁项目集的更新问题进行探讨，既考虑了数据库中事务增加的情况，又考虑了事务删除的情况；提出了一种基于DDMINER的局部频繁项目集的更新算法ULF和全局频繁项目集的更新算法UGF．该算法能够产生较少数量的候选频繁项目集，在求解全局频繁项目集过程中，传送候选局部频繁项目集支持数的通信量为O(n)；将文章提出的算法用Java语言加以实现，并对算法性能进行了研究；实验结果表明这些算法是正确、可行的，并且具有较高的效率．     

基于二进制的频繁项集挖掘新算法

频繁项集挖掘是数据挖掘中的一个经典的问题。然而,大部分算法需要扫描数据库多次,算法效率比较低。该文提出了一个效率比较好的挖掘频繁项集的新算法,在这个算法中,所有的事务都是以二进制的形式表示,所以挖掘极大频繁项集的任务就变成了从二进制集中发现频繁模式。而且,这种算法只需要扫描原始数据库一次。最后,利用试验来证明这种算法的效率和优势。     

一种基于单事务项集组合的频繁项集挖掘算法

Apriori是挖掘频繁项集的基本算法,目前该算法及其优化变种都没有解决候选项及重复扫描事务数据库的问题.文章通过对Apriori及其优化算法的深入探究,提出了一种基于单事务组合项集的挖掘算法,该算法在一个事务内部对"数据项"进行组合,在事务数据库中对所有相同"项集"进行计数.不经过迭代过程,不产生候选项集,所有频繁项集的挖掘过程只需对事务数据库一次扫描,提高了频繁项集挖掘效率.     

基于WNegNodeset结构的加权频繁项集挖掘算法

针对基于WN-list 加权频繁项集挖掘算法（NFWI）中挖掘加权频繁项集（FWI）效率低的问题,提出了一种基于WNegNodeset结构的加权频繁项集挖掘算法（NegNFWI）。该算法首先采用了新的数据结构WNegNodeset,它是NegNodeset的扩展,该数据结构采用了一种新的基于集合位图表示的位图加权树（BMW-tree）节点编码模型,通过按位运算符快速提取WNegNodeset的节点集,避免了大量的交集运算;其次采用了差集策略快速计算项集的加权支持度,从而减少了计算量;最后通过仿真实验验证了算法的有效性和可行性。     

基于链表数组的最大频繁项集挖掘算法

挖掘密集型数据集的全部频繁项集代价高昂,针对该问题,提出一种数据结构链表数组和基于链表数组的最大频繁项集快速生成算法。该方法使用链表数组为每个项目建立事务链表,并且链表的创建过程只需扫描数据库1次。使用深度优先搜索得到所有候选最大频繁项集,利用约束条件缩小搜索空间。使用标准数据集进行验证测试并与其他算法进行比较,实验结果表明,该算法具有较快的挖掘速度。     

An efficient algorithm for mining frequent weighted itemsets using interval word segments

Mining frequent weighted itemsets (FWIs) from weighted-item transaction databases has recently received research interest. In real-world applications, sparse weighted-item transaction databases (SWITDs) are common. For example, supermarkets have many items, but each transaction has a small number of items. In this paper, we propose an interval word segment (IWS) structure to store and process tidsets for enhancing the effectiveness of mining FWIs from SWITDs. The IWS structure allows the intersection of tidsets between two itemsets to be performed very fast. A map array is proposed for storing a 1-bit index for words. From the map array, 1-bits are mapped to create the tidset of an itemset for faster calculation of the weighted support of itemsets. Experimental results for a number of SWITDs show that the method based on IWS structure outperforms existing methods.     

快速挖掘全局最大频繁项目集

挖掘最大频繁项目集是多种数据挖掘应用中的关键问题.现行可用的最大频繁项目集挖掘算法大多基于单机环境,针对分布式环境下的全局最大频繁项目集挖掘尚不多见.若将基于单机环境的最大频繁项目集挖掘算法运用于分布式环境,或运用分布式环境下的全局频繁项目集挖掘算法来挖掘全局最大频繁项目集,均会产生大量的候选频繁项目集,且网络通信代价高.为此,提出了快速挖掘全局最大频繁项目集算法FMGMFI(fast mining global maximum frequent itemsets),该算法采用FP-tree存储结构,可方便地从各局部FP-tree的相关路径中得到项目集的频度,同时采用自顶向下和自底向上的双向搜索策略,可有效地降低网络通信代价.实验结果表明,FMGMF算法是有效、可行的.     

Efficient updating of discovered high-utility itemsets for transaction deletion in dynamic databases

Most algorithms related to association rule mining are designed to discover frequent itemsets from a binary database. Other factors such as profit, cost, or quantity are not concerned in binary databases. Utility mining was thus proposed to measure the utility values of purchased items for finding high-utility itemsets from a static database. In real-world applications, transactions are changed whether insertion or deletion in a dynamic database. An existing maintenance approach for handling high-utility itemsets in dynamic databases with transaction deletion must rescan the database when necessary. In this paper, an efficient algorithm, called PRE-HUI-DEL, for updating high-utility itemsets based on the pre-large concept for transaction deletion is proposed. The pre-large concept is used to partition transaction-weighted utilization itemsets into three sets with nine cases according to whether they have large (high), pre-large, or small transaction-weighted utilization in the original database and in the deleted transactions. Specific procedures are then applied to each case for maintaining and updating the discovered high-utility itemsets. Experimental results show that the proposed PRE-HUI-DEL algorithm outperforms a batch two-phase algorithm and a FUP2-based algorithm in maintaining high-utility itemsets.     

Incremental mining of weighted maximal frequent itemsets from dynamic databases

Frequent itemset mining allows us to find hidden, important information from large databases. Moreover, processing incremental databases in the itemset mining area has become more essential because a huge amount of data has been accumulated continually in a variety of application fields and users want to obtain mining results from such incremental data in more efficient ways. One of the major problems in incremental itemset mining is that the corresponding mining results can be very large-scale according to threshold settings and data volumes. In addition, it is considerably hard to analyze all of them and find meaningful information. Furthermore, not all of the mining results become actually important information. In this paper, to solve these problems, we propose an algorithm for mining weighted maximal frequent itemsets from incremental databases. By scanning a given incremental database only once, the proposed algorithm can not only conduct its mining operations suitable for the incremental environment but also extract a smaller number of important itemsets compared to previous approaches. The proposed method also has an effect on expert and intelligent systems since it can automatically provide more meaningful pattern results reflecting characteristics of given incremental databases and threshold settings, which can help users analyze the given data more easily. Our comprehensive experimental results show that the proposed algorithm is more efficient and scalable than previous state-of-the-art algorithms.     

分布式环境下约束性关联规则的快速挖掘

研究人员针对单机环境提出了约束性关联规则的挖掘算法,但它们不适用于分布式环境.为此本文讨论分布式环境下约束性关联规则的快速挖掘技术,提出一种基于分布式环境的约束性关联规则快速挖掘算法DCAR,其中包括局部约束性频繁项目集挖掘算法MLFC和全局约束性频繁项目集挖掘算法MGFC.该算法根据布尔约束条件产生向导集,采用一种新的候选项集生成函数Reorder-gen,该函数通过向导集高效地产生分布式环境中满足约束条件的、数量较少且完备的候选项集,并且求解全局约束性频繁项集过程中,传送局部候选项集支持数的通信量为O(n),从而提高了算法的挖掘效率.将本文提出的算法加以实现,实验结果表明DCAR算法高效可行,其效率大约是DMA-IC算法的2-3倍.     

Incrementally mining high utility patterns based on pre-large concept

In traditional association rule mining, most algorithms are designed to discover frequent itemsets from a binary database. Utility mining was thus proposed to measure the utility values of purchased items for revealing high utility itemsets from a quantitative database. In the past, a two-phase high utility mining algorithm was thus proposed for efficiently discovering high utility itemsets from a quantitative database. In dynamic data mining, transactions may be inserted, deleted, or modified from a database. In this case, a batch mining procedure must rescan the whole updated database to maintain the up-to-date information. Designing an efficient approach for handling dynamic databases is thus a critical research issue in utility mining. In this paper, an incremental mining algorithm is proposed for efficiently maintaining discovered high utility itemsets based on pre-large concepts. Itemsets are first partitioned into three parts according to whether they have large (high), pre-large, or small transaction-weighted utilization in the original database and in inserted transactions. Individual procedures are then executed for each part. Experimental results show that the proposed incremental high utility mining algorithm outperforms existing algorithms.     

频繁项集挖掘的研究进展及主流方法

关联分析作为数据挖掘的主要研究模块之一,主要用于发现隐藏在大型数据集中的强关联特征。而多数关联规则挖掘任务可分为频繁模式(频繁项集、频繁序列、频繁子图)的产生和规则的产生。前者发现数据集中满足最小支持度阈值的项集、序列与子图;后者从上一步发现的频繁模式中提取高置信度的规则。频繁项集挖掘是许多数据挖掘任务中的关键问题,也是关联规则挖掘算法的核心。十几年来,学者们致力于提高频繁项集的生成效率,从不同的角度进行改进以提高算法效率,大量的高效可伸缩性算法被提出。文中对频繁项集挖掘进行深入分析,对完全频繁项集、闭频繁项集、极大频繁项集的典型算法进行介绍和评述,最后对频繁项集挖掘算法的研究方向进行简要分析。     

Multipass Algorithms for Mining Association Rules in Text Databases

In this paper, we propose two new algorithms for mining association rules between words in text databases. The characteristics of text databases are quite different from those of retail transaction databases, and existing mining algorithms cannot handle text databases efficiently because of the large number of itemsets (i.e., words) that need to be counted. Two well-known mining algorithms, Apriori algorithm and Direct Hashing and Pruning (DHP) algorithm, are evaluated in the context of mining text databases, and are compared with the new proposed algorithms named Multipass-Apriori (M-Apriori) and Multipass-DHP (M-DHP). It has been shown that the proposed algorithms have better performance for large text databases. Received 12 November 1999 / Revised 27 September 2000 / Accepted in revised form 25 October 2000     

Weighted frequent itemset mining over uncertain databases

Frequent itemset mining (FIM) is a fundamental research topic, which consists of discovering useful and meaningful relationships between items in transaction databases. However, FIM suffers from two important limitations. First, it assumes that all items have the same importance. Second, it ignores the fact that data collected in a real-life environment is often inaccurate, imprecise, or incomplete. To address these issues and mine more useful and meaningful knowledge, the problems of weighted and uncertain itemset mining have been respectively proposed, where a user may respectively assign weights to items to specify their relative importance, and specify existential probabilities to represent uncertainty in transactions. However, no work has addressed both of these issues at the same time. In this paper, we address this important research problem by designing a new type of patterns named high expected weighted itemset (HEWI) and the HEWI-Uapriori algorithm to efficiently discover HEWIs. The HEWI-Uapriori finds HEWIs using an Apriori-like two-phase approach. The algorithm introduces a property named high upper-bound expected weighted downward closure (HUBEWDC) to early prune the search space and unpromising itemsets. Substantial experiments on real-life and synthetic datasets are conducted to evaluate the performance of the proposed algorithm in terms of runtime, memory consumption, and number of patterns found. Results show that the proposed algorithm has excellent performance and scalability compared with traditional methods for weighted-itemset mining and uncertain itemset mining.     

基于矩阵的加权关联规则挖掘算法

关联规则挖掘是数据挖掘中的一个重要模型。传统的关联规则挖掘算法需要多次扫描数据库,生成大量候选项集,并且把数据库中各个项目按平等一致的方法对待,算法复杂且与实际情况不符。为此提出一种基于矩阵的加权关联规则挖掘算法,它只需扫描一次数据库,不生成候选项目集,可以快速挖掘出频率小但重要性高的项目。     

基于约束的最大频繁项集挖掘算法

为了解决目前带约束的频繁项集挖掘算法在具有长模式的密集型数据库中挖掘的不足,提出了一种快速的基于约束的最大频繁项集挖掘算法。该算法在特定约束条件的基础上运用了深度优先策略和有效的剪枝方法快速挖掘最大频繁项集。实验结果表明了该算法是快速有效的。     

BitTableFI: An efficient mining frequent itemsets algorithm

Mining frequent itemsets in transaction databases, time-series databases and many other kinds of databases is an important task and has been studied popularly in data mining research. The problem of mining frequent itemsets can be solved by constructing a candidate set of itemsets first, and then, identifying those itemsets that meet the frequent itemset requirement within this candidate set. Most of the previous research mainly focuses on pruning to reduce the candidate itemsets amounts and the times of scanning databases. However, many algorithms adopt an Apriori-like candidate itemsets generation and support count approach that is the most time-wasted process. To address this issue, the paper proposes an effective algorithm named as BitTableFI. In the algorithm, a special data structure BitTable is used horizontally and vertically to compress database for quick candidate itemsets generation and support count, respectively. The algorithm can also be used in many Apriori-like algorithms to improve the performance. Experiments with both synthetic and real databases show that BitTableFI outperforms Apriori and CBAR which uses ClusterTable for quick support count.