应用于长频繁集挖掘的基于变动邻域搜索的遗传算法设计

提出了一种基于变动邻域搜索的长频繁集挖掘方法(VNS-GA),利用遗传算法的高效搜索性能快速挖掘最大频繁集。在遗传算法的适应度函数设计中,综合考虑项集支持度、长度以及项集支持度和邻域中心支持度的距离,算法一次运行可找出邻域内的最大频繁集,改变邻域中心即可找到我们需要的最大频繁集。算法有效性通过实验得到了验证,且实验表明该算法的时间复杂度与支持度阈值大小无关,因此对于长模式挖掘问题具有很高的效率。     

高置信度关联规则的挖掘

传统的关联规则和基于效用的关联规则,会忽略一些支持度或效用值不高、置信度（又称可信度）却非常高的规则,这些置信度很高的规则能帮助人们满足规避风险、提高成功率的期望。为挖掘这些低支持度（或效用值）、高置信度的规则,提出了HCARM算法。HCARM采用了划分的方法来处理大数据集,利用新的剪枝策略压缩搜索空间。同时,通过设定长度阈值minlen,使HCARM适合长模式挖掘。实验结果表明,该方法对高置信度长模式有效。     

高维大数据集中频繁闭合模式的挖掘

高维大数据集对现有的数据挖掘算法提出了挑战。该文把挖掘任务分解为挖掘频繁长模式与短模式2个子问题,提出一种在高维大数据集中挖掘长项集的算法,即inter-transaction。该算法利用了高维数据中长事务相交迅速变短的特性,通过事务的交集运算直接得到长闭合模式,同时采用新的减枝策略,优化了事务交集运算的方法。实验表明,该方法对高维大数据集非常有效。     

基于激励的关联规则的挖掘

基于支持度的关联规则挖掘算法无法找到那些非频繁但效用很高的项集,基于效用的关联规则会漏掉那些效用不高但发生比较频繁、支持度和效用值的积（激励）很大的项集。提出了基于激励的关联规则挖掘问题及一种自下而上的挖掘算法HM-miner。激励综合了支持度与效用的优点,能同时度量项集的统计重要性和语义重要性。HM-miner利用激励的上界特性进行减枝,能有效挖掘高激励项集。     

高激励项集的挖掘研究

基于支持度的关联规则只能找出所有的频繁集,无法找到那些非频繁但效用很高的项集;基于效用的关联规则致力于发现所有高效用项集,无法找到效用不高但支持度与效用的积很大的项集。为克服支持度与效用的不足,提出了一种新的项集重要性的度量方法（即激励）及一种自下而上的挖掘高激励项集的算法HM-Two-Phase-Miner。激励集成了支持度与效用的优点,能同时表达项集的语义特性与统计特性。HM-Two-Phase-Miner利用事务权重激励向下封闭特性进行减枝,有效提高了算法的性能。     

基于多效用阈值的分布式高效用序列模式挖掘

针对序列模式的高效用模式挖掘过程中搜索空间大、计算复杂度高的问题,提出一种基于多效用阈值的分布式高效用序列模式挖掘算法。采用数组结构保存模式的效用信息,解决效用矩阵导致的内存消耗大的缺点。设计1-项集与2-项集的深度剪枝策略,深入地缩小候选模式的搜索空间,减少搜索时间成本与缓存成本。提出挖掘算法的分布式实现方案,通过并行处理进一步降低模式挖掘的时间。基于中等规模与大规模的序列数据集分别进行实验,实验结果表明,该算法有效减少了候选模式的数量,降低了挖掘的时间成本与存储成本,对于大数据集表现出较好的可扩展能力与稳定性。     

多段支持度数据频繁模式关联规则挖掘仿真

针对传统数据关联挖掘过程只适用于单段数据集,导致内存负担重、挖掘频繁项集效率不高等问题,提出一种多段支持度数据频繁模式关联规则挖掘方法.运用多支持度算法对数据集逐步搜索,数据集按照数据项的MIS大小有序排列,采用最小值作为最小支持度,确保该算法的地推性.构建FP_ tree树,利用FP_tree算法对待选项实施剪枝,从而准确挖掘出频繁模式的关联规则.仿真结果证明,多段支持度数据频繁模式关联规则挖掘具有较好的性能,有效提高了关联规则的挖掘效率.     

多支持度下用户行为序列模式挖掘方法研究

针对现有用户行为序列模式挖掘方法的单一支持度局限性问题,提出一种基于前缀树结构的多支持度序列模式挖掘方法。设计一种多支持度条件下的前缀树结构MSLP-tree,并基于此结构提出一种序列模式增长算法MSLP-growth。通过考虑各数据项不同最小支持度,获取更精确的频繁序列模式,在确保挖掘结果的准确性和完整性的前提下,大大压缩搜索空间,缩短挖掘时间。实验结果表明,相较于MS-GSP算法,MSLP-growth算法具有更高的挖掘效率和可扩展性。     

一种基于升序FP-tree的频繁模式挖掘算法

本文提出了一种基于升序FP-tree的频繁模式挖掘算法,该算法按照支持度升序构造升序FP-tree,并通过在其中搜索扩展频繁集及归并子树来挖据频繁模式。实验表明,与FP-growth算法相比,算法的挖掘速度提高了将近2倍,此外新算法还具有比较好的伸缩性。     

改进的基于频繁模式树的最大频繁项集挖掘算法——FP-MFIA

针对最大频繁项目集挖掘算法(DMFIA)当候选项目集维数高而最大频繁项目集维数较低的情况下要产生大量的候选项目集的缺点,提出了一种改进的基于频繁模式树(FP-tree)结构的最大频繁项目集挖掘算法--FP-MFIA。该算法根据FP-tree的项目头表,采用自底向上的搜索策略逐层挖掘最大频繁项目集,从而加速每次对候选集计数的操作。在挖掘时根据每层的条件模式基产生维数较低的非频繁项目集,尽早对候选项目集进行剪枝和降维,可大量减少候选项目集的数量。同时在挖掘时充分利用最大频繁项集的性质,减少搜索空间。通过算法在不同支持度下挖掘时间的对比可知,算法FP-MFIA在最小支持度较低的情况下时间效率是DMFIA以及基于降维的最大频繁模式挖掘算法(BDRFI)的2倍以上,说明FP-MFIA在候选集维数较高的时候优势明显。     

An efficient algorithm for mining frequent inter-transaction patterns

In this paper, we propose an efficient method for mining all frequent inter-transaction patterns. The method consists of two phases. First, we devise two data structures: a dat-list, which stores the item information used to find frequent inter-transaction patterns; and an ITP-tree, which stores the discovered frequent inter-transaction patterns. In the second phase, we apply an algorithm, called ITP-Miner (Inter-Transaction Patterns Miner), to mine all frequent inter-transaction patterns. By using the ITP-tree, the algorithm requires only one database scan and can localize joining, pruning, and support counting to a small number of dat-lists. The experiment results show that the ITP-Miner algorithm outperforms the FITI (First Intra Then Inter) algorithm by one order of magnitude.     

Using a projection-based approach to mine frequent inter-transaction patterns

In this paper, we propose an algorithm called PITP-Miner that utilizes a projection based approach to mine frequent inter-transaction patterns efficiently. The algorithm only searches for local frequent items in a projected database that stores potential local inter-transaction items and partitions the database into a set of smaller databases recursively. In addition, two pruning strategies are designed to further condense the partitioned databases and thus accelerate the algorithm. Our experiment results demonstrate that the proposed PITP-Miner algorithm outperforms the ITP-Miner and FITI algorithms in most cases.     

Efficient mining of high utility pattern with considering of rarity and length

Techniques for mining rare patterns have been researched in the association rule mining area because traditional frequent pattern mining methods have to generate a large amount of unnecessary patterns in order to find rare patterns from large databases. One such technique, the multiple minimum support threshold framework was devised to extract rare patterns by using a different minimum item support threshold for each item in a database. Nevertheless, this framework cannot sufficiently reflect environments of the real world. The reason is that it does not consider weights of items, such as market prices of products and fatality rates of diseases, in its mining process. Therefore, an algorithm has been proposed to mine rare patterns with utilities exceeding a user-specified minimum utility by considering rarity and utility information of items. However, since this algorithm employs the concept of traditional high utility pattern mining, patterns’ lengths are not considered for determining utilities of the patterns. If the length of a pattern is sufficiently long, the pattern is more likely to have an enough utility to become a high utility pattern regardless of item utilities within the pattern. Therefore, the algorithm cannot guarantee that all items in a mined pattern have high utilities. In this paper, we propose a novel algorithm that effectively reduces such dependency of patterns on their lengths by considering their lengths in the mining process in order to mine more meaningful rare patterns compared to patterns mined by previous algorithms. Experimental results demonstrate that our algorithm extracts a lesser number of more meaningful patterns and consumes less computational resources compared to state-of-the-art algorithms.     

Indexed list-based high utility pattern mining with utility upper-bound reduction and pattern combination techniques

High utility pattern mining has been studied as an essential topic in the field of pattern mining in order to satisfy requirements of many real-world applications that need to process non-binary databases including item importance such as market analysis. In this paper, we propose an efficient algorithm with a novel indexed list-based data structure for mining high utility patterns. Previous approaches first generate an enormous number of candidate patterns on the basis of overestimation methods in their mining processes and then identify actual high utility patterns from the candidates through an additional database scan, which leads to high computational overheads. Although several list-based algorithms to discover high utility patterns without candidate generation have been suggested in recent years, they require a large number of comparison operations. Our method facilitates efficient mining of high utility patterns with the proposed indexed list by effectively reducing the total number of such operations. Moreover, we develop two techniques based on this novel data structure to more enhance mining performance of the proposed method. Experimental results on real and synthetic datasets show that the proposed algorithm mines high utility patterns more efficiently than the state-of-the-art algorithms.     

High utility pattern mining over data streams with sliding window technique

Processing changeable data streams in real time is one of the most important issues in the data mining field due to its broad applications such as retail market analysis, wireless sensor networks, and stock market prediction. In addition, it is an interesting and challenging problem to deal with the stream data since not only the data have unbounded, continuous, and high speed characteristics but also their environments have limited resources. High utility pattern mining, meanwhile, is one of the essential research topics in pattern mining to overcome major drawbacks of the traditional framework for frequent pattern mining that takes only binary databases and identical item importance into consideration. This approach conducts mining processes by reflecting characteristics of real world databases, non-binary quantities and relative importance of items. Although relevant algorithms were proposed for finding high utility patterns in stream environments, they suffer from a level-wise candidate generation-and-test and a large number of candidates by their overestimation techniques. As a result, they consume a huge amount of execution time, which is a significant performance issue since a rapid process is necessary in stream data analysis. In this paper, we propose an algorithm for mining high utility patterns from resource-limited environments through efficient processing of data streams in order to solve the problems of the overestimation-based methods. To improve mining performance with fewer candidates and search space than the previous ones, we develop two techniques for reducing overestimated utilities. Moreover, we suggest a tree-based data structure to maintain information of stream data and high utility patterns. The proposed tree is restructured by our updating method with decreased overestimation utilities to keep up-to-date stream information whenever the current window slides. Our approach also has an important effect on expert and intelligent systems in that it can provide users with more meaningful information than traditional analysis methods by reflecting the characteristics of real world non-binary databases in stream environments and emphasizing on recent data. Comprehensive experimental results show that our algorithm outperforms the existing sliding window-based one in terms of runtime efficiency and scalability.     

Sequential Pattern Mining in Multi-Databases via Multiple Alignment

To efficiently find global patterns from a multi-database, information in each local database must first be mined and summarized at the local level. Then only the summarized information is forwarded to the global mining process. However, conventional sequential pattern mining methods based on support cannot summarize the local information and is ineffective for global pattern mining from multiple data sources. In this paper, we present an alternative local mining approach for finding sequential patterns in the local databases of a multi-database. We propose the theme of approximate sequential pattern mining roughly defined as identifying patterns approximately shared by many sequences. Approximate sequential patterns can effectively summerize and represent the local databases by identifying the underlying trends in the data. We present a novel algorithm, ApproxMAP, to mine approximate sequential patterns, called consensus patterns, from large sequence databases in two steps. First, sequences are clustered by similarity. Then, consensus patterns are mined directly from each cluster through multiple alignment. We conduct an extensive and systematic performance study over synthetic and real data. The results demonstrate that ApproxMAP is effective and scalable in mining large sequences databases with long patterns. Hence, ApproxMAP can efficiently summarize a local database and reduce the cost for global mining. Furthremore, we present an elegant and uniform model to identify both high vote sequential patterns and exceptional sequential patterns from the collection of these consensus patterns from each local databases.     

多元时间序列中跨事务关联规则分析的高效处理算法

用挖掘跨事务关联规则的方法分析多元时间序列,可以找到序列中不同采样点观察值之间相互影响的关系。本文为实现这一目的,提出一种新的分析方法：ES—Apriori。此方法通过减少数据库扫描次数,优化内存分配,能够高效地分析多元时间序列之间的关联规则。试验表明,用此方法分析中国证券市场的股票时间序列非常有效。     

使用序列模式精简基挖掘序列模式

传统的序列模式挖掘方法在挖掘由短的频繁序列模式组成的数据库时有良好的性能．但在挖掘长的序列模式或支持度阈值很低时，这些方法可能遇到固有的困难，因为产生的频繁序列模式的数量经常太大．在许多情况下，用户可能只需要那些覆盖许多短模式的长模式．此外，在很多应用中，只要得到产生的频繁序列模式的近似支持度就已足够，而不需要它们的精确支持度．介绍了能将误差控制在确定范围内的频繁序列模式精简基的概念，并开发了一个挖掘这种序列模式精简基的算法．实验结果显示计算频繁序列模式精简基是很有前途的．