基于最大值约束的模糊关联规则挖掘

数据挖掘是从数据库中发现潜在有用知识或者感兴趣模式的过程。在数据挖掘领域中主要集中于单一支持度下的关联规则挖掘,在事务数据库中发现项目之间的关联性,而在实际应用中,项目可以有不同的最小支持度,不同的项目可能具有不同的标准去判断其重要性,因此提出一个在最大值支持度约束下,发现有用的模糊关联规则挖掘算法,在该约束下,利用逐层搜索的迭代方法发现频繁项目集,通过实例证明了该挖掘算法是易于理解和有意义的,具有很好的效率。     

特定数据最大频繁集挖掘算法

针对在某些限定项目数与交易长度数据的关联规则挖掘中FP-growth算法执行效率很低的问题,提出一种最大频繁模式挖掘算法,该算法引入与FP-tree结构类似的All-subset tree存储所有的最大频繁项目集,无需在扫描数据库前指定最小支持度,可以动态给定最小支持度而不用重新扫描数据库。实验结果表明,该算法在这些特定数据的挖掘中,与FP-growth相比明显提高了挖掘效率。     

Mining fuzzy temporal association rules by item lifespans

Data mining is the process of extracting desirable knowledge or interesting patterns from existing databases for specific purposes. In real-world applications, transactions may contain quantitative values and each item may have a lifespan from a temporal database. In this paper, we thus propose a data mining algorithm for deriving fuzzy temporal association rules. It first transforms each quantitative value into a fuzzy set using the given membership functions. Meanwhile, item lifespans are collected and recorded in a temporal information table through a transformation process. The algorithm then calculates the scalar cardinality of each linguistic term of each item. A mining process based on fuzzy counts and item lifespans is then performed to find fuzzy temporal association rules. Experiments are finally performed on two simulation datasets and the foodmart dataset to show the effectiveness and the efficiency of the proposed approach.     

基于粗糙集和单事务项组合的关联规则挖掘算法

Apriori算法必须反复地扫描数据库才能求出频繁项集,效率较低,且不支持更新挖掘。为了解决这些问题,提出了一种基于粗糙集、单事务项组合和集合运算的关联规则挖掘算法。本算法首先利用粗糙集进行属性约简,对新决策表中的每个事务进行“数据项”组合并标记地址,然后利用集合运算的方法计算支持度和置信度即可挖掘出有效规则。本算法只需要一次扫描数据库,同时有效地支持了关联规则的更新挖掘。应用实例和实验结果表明,本算法明显优于Apriori算法,是一种有效且快速的关联规则挖掘算法。     

基于多维时态关联规则的演化模糊推理预测算法

挖掘时态关联规则的目的是为了发现带有时态信息的项集之间有趣的关系.由于数据库经常动态更新,时态关联规则的挖掘也应该适应数据库的更新.然而,现有的大多数算法不仅需要重新挖掘更新的数据库,浪费了大量的时间和效率,而且不能利用已存在的规则定量地预测某些项的变化趋势.本文提出了一个基于多维时态关联规则的演化模糊推理预测建模算法（Evolving fuzzy inference model based on multidimensional temporal association rules,EFI-MTAR）,主要优势是构建了一种基于多维时态关联规则的模糊推理建模算法（Fuzzy inference modeling algorithm based on multidimensional temporal association rules,FI-MTAR）,实现了对时间序列的定量预测.此外,为了降低规则更新的代价和加快规则预测的速度,提出了概念漂移检测策略来处理时间序列数据以适应数据库的动态更新.实验结果表明了本文提出算法的有效性和准确性.     

SparkSql上自适应数据集的高效频繁集挖掘算法

针对基于Spark框架的关联规则算法存在I/O开销大、数据结构和挖掘频繁集方式单一、计算支持度的方式效率低等问题,提出基于SparkSql进行分布式编程的算法。将数据集加载到DataFrame,利用改进后的布隆过滤器高效存储频繁集挖掘过程中产生的项集,解决RDD内存资源和计算速度受限问题。基于先验定理对事务、项目和项集进行精简,同时提出用Sql语句对项集中项目对应事务集合求交集的方式计算项集支持度,提高计算支持度的效率。提出了两种迭代算法和自适应数据的选择条件,增强该算法对各种数据集的泛化性。进行多组实验,证明提出的算法总是自适应本次迭代数据的特点选择最优的迭代方法,同时具有较高并行算法性能,可以扩展到更大规模集群和数据;同基于Spark框架的关联规则算法YAFIM和R-Apriori进行对比,在每次迭代和总体运行计算效率上有更好的表现。     

ASTRA - A Novel interest measure for unearthing latent temporal associations and trends through extending basic gaussian membership function

Time profiled association mining is one of the important and challenging research problems that is relatively less addressed. Time profiled association mining has two main challenges that must be addressed. These include addressing i) dissimilarity measure that also holds monotonicity property and can efficiently prune itemset associations ii) approaches for estimating prevalence values of itemset associations over time. The pioneering research that addressed time profiled association mining is by J.S. Yoo using Euclidean distance. It is widely known fact that this distance measure suffers from high dimensionality. Given a time stamped transaction database, time profiled association mining refers to the discovery of underlying and hidden time profiled itemset associations whose true prevalence variations are similar as the user query sequence under subset constraints that include i) allowable dissimilarity value ii) a reference query time sequence iii) dissimilarity function that can find degree of similarity between a temporal itemset and reference. In this paper, we propose a novel dissimilarity measure whose design is a function of product based gaussian membership function through extending the similarity function proposed in our earlier research (G-Spamine). Our approach, MASTER (Mining of Similar Temporal Associations) which is primarily inspired from SPAMINE uses the dissimilarity measure proposed in this paper and support bound estimation approach proposed in our earlier research. Expression for computation of distance bounds of temporal patterns are designed considering the proposed measure and support estimation approach. Experiments are performed by considering naïve, sequential, Spamine and G-Spamine approaches under various test case considerations that study the scalability and computational performance of the proposed approach. Experimental results prove the scalability and efficiency of the proposed approach. The correctness and completeness of proposed approach is also proved analytically.

     

一种混合型的增量数据关联规则挖掘算法

在数据库中增加数据且调整最小支持度时,数据库中关联规则会发生变化,为从数据量和最小支持度同时发生变化的数据库中快速获取频繁项集,发现变化后的关联规则,通过对FIM和AIUA算法进行分析,提出一种结合两种算法优点的增量数据关联规则挖掘My_FIM_AIUA算法,该算法能减少数据库扫描次数,减少候选项集数量。通过实验表明My_FIM_AIUA算法能在数据量和最小支持度同时变化时快速找到频繁项集,提高挖掘增量数据关联规则的速度。     

DEMON: mining and monitoring evolving data

Data mining algorithms have been the focus of much research. In practice, the input data to a data mining process resides in a large data warehouse whose data is kept up-to-date through periodic or occasional addition and deletion of blocks of data. Most data mining algorithms have either assumed that the input data is static, or have been designed for arbitrary insertions and deletions of data records. We consider a dynamic environment that evolves through systematic addition or deletion of blocks of data. We introduce a new dimension, called the data span dimension, which allows user-defined selections of a temporal subset of the database. Taking this new degree of freedom into account, we describe efficient model maintenance algorithms for frequent item sets and clusters. We then describe a generic algorithm that takes any traditional incremental model maintenance algorithm and transforms it into an algorithm that allows restrictions on the data span dimension. We also develop an algorithm for automatically discovering a specific class of interesting block selection sequences. In a detailed experimental study, we examine the validity and performance of our ideas on synthetic and real datasets     

关联规则挖掘算法介绍

数据挖掘是一个多学科交叉融合而形成的新兴的学科,它利用各种分析工具在海量数据中发现模型和数据间的关系。而在大规模事务数据库中,挖掘关联规则是数据挖掘领域的一个非常重要的研究课题。文中介绍了关联规则挖掘的研究情况,描述了经典Apriori算法的实现,并对该算法进行了分析和评价,指出了其不足和原因。描述了FP树挖掘最大频繁项集的算法,通过实例对该算法进行了性能评估,并得到结论:数据库中潜在的最大频繁模式越多,运行时间越长。     

PARM—An Efficient Algorithm to Mine Association Rules From Spatial Data

Association rule mining, originally proposed for market basket data, has potential applications in many areas. Spatial data, such as remote sensed imagery (RSI) data, is one of the promising application areas. Extracting interesting patterns and rules from spatial data sets, composed of images and associated ground data, can be of importance in precision agriculture, resource discovery, and other areas. However, in most cases, the sizes of the spatial data sets are too large to be mined in a reasonable amount of time using existing algorithms. In this paper, we propose an efficient approach to derive association rules from spatial data using Peano Count Tree (P-tree) structure. P-tree structure provides a lossless and compressed representation of spatial data. Based on P-trees, an efficient association rule mining algorithm PARM with fast support calculation and significant pruning techniques is introduced to improve the efficiency of the rule mining process. The P-tree based Association Rule Mining (PARM) algorithm is implemented and compared with FP-growth and Apriori algorithms. Experimental results showed that our algorithm is superior for association rule mining on RSI spatial data.      

区间值属性数据集关联规则挖掘算法仿真

对区间值属性数据集进行挖掘,可以有效分析出数据之间的关系。针对现有数据挖掘方法未对大规模数据进行聚类,导致挖掘过程占据内存大,挖掘精度低的问题,提出了一种新的区间值属性数据集挖掘算法。对问题定义、数据准备、数据提取、模式预测和数据聚类等模块进行详细分析,完成区间值属性数据聚类。根据聚类结果,将区间值属性数据分成多个数据集,挑选出能够支持最小支持度的项目集,将这些项目集作为频繁项集,进而提取出数据集之间的关联规则,将关联规则融入数据计算步骤,完成数据挖掘。为验证算法效果,进行仿真,结果表明,相较于传统挖掘算法,所提挖掘算法占用容量更小,挖掘精度更高。     

一种基于倒排索引树的增量更新关联挖掘算法

增量更新关联规则挖掘主要解决事务数据库中交易记录不断更新和最小支持度发生变化时关联规则的维护问题。针对目前诸多增量更新关联规则挖掘算法存在效率低、计算成本高、规则难以维护等问题,提出一种基于倒排索引树的增量更新关联挖掘算法。该算法有效地将倒排索引技术与树型结构相结合,使得交易数据库中的数据不断更新和最小支持度随应用环境不同而不断改变时,以实现无需扫描原始交易数据库和不产生候选项集的情况下生成频繁项集。实验结果表明,该算法只需占用较小的存储空间、且检索项集的效率较高,能高效地解决增量更新关联规则难以维护的问题。     

一种新的关联规则增量式挖掘算法

针对数据库不断更新变化及现实生活中大多只对近期数据感兴趣的特点，该文提出了一种基于滑动窗口过滤器的关联规则增量式挖掘算法（SWFAI算法）。该算法通过分组及时舍弃挖掘过程中生成的非频繁项目集，有效降低主存压力，减少对数据库的扫描次数，能够对时变数据库进行高效地关联规则挖掘。通过实验证明了该算法能够有效地进行关联规则的挖掘，并在效率上有较大提高。     

Discovering multi-label temporal patterns in sequence databases

Sequential pattern mining is one of the most important data mining techniques. Previous research on mining sequential patterns discovered patterns from point-based event data, interval-based event data, and hybrid event data. In many real life applications, however, an event may involve many statuses; it might not occur only at one certain point in time or over a period of time. In this work, we propose a generalized representation of temporal events. We treat events as multi-label events with many statuses, and introduce an algorithm called MLTPM to discover multi-label temporal patterns from temporal databases. The experimental results show that the efficiency and scalability of the MLTPM algorithm are satisfactory. We also discuss interesting multi-label temporal patterns discovered when MLTPM was applied to historical Nasdaq data.     

关联规则挖掘中Apriori改进算法的研究

挖掘频繁项集是许多数据挖掘任务中的关键问题,也是关联规则挖掘算法的核心,提高频繁项集的生成效率一直是近几年数据挖掘领域研究的热点之一.在对关联规则挖 掘中基于Apriori算法的改进算法进行深入分析和研究后,本文根据Apriori算法的不足,提出了一种改进策略,从而得到一种优化的Apriori算法.最后,对频繁项集挖掘算法的发展方向进行了初步的探讨.     

Temporal data mining with up-to-date pattern trees

Mining interesting and useful frequent patterns from large databases attracts much attention in recent years. Among the mining approaches, finding temporal patterns and regularities is very important due to its practicality. In the past, Hong et al. proposed the up-to-date patterns, which were frequent within their up-to-date lifetime. Formally, an up-to-date pattern is a pair with the itemset and its valid corresponding lifetime in which the user-defined minimum support threshold must be satisfied. They also proposed an Apriori-like approach to find the up-to-date patterns. This paper thus proposes the up-to-date pattern tree (UDP tree) to keep the up-to-date 1-patterns in a tree structure for reducing database scan. It is similar to the FP-tree structure but more complex due to the requirement of up-to-date patterns. The UDP-growth mining approach is also designed to find the up-to-date patterns from the UDP tree. The experimental results show that the proposed approach has a better performance than the level-wise mining algorithm.     

基于最大频繁项目序列集挖掘ISS_DM算法的改进

阐述了挖掘最大频繁项目序列集ISS_DM算法,针对该算法不能有效地解决客户序列视图数据库的数据挖掘问题,结合序列模式提出了改进的ISS_DM算法,并进行了相应的验证.实践证明,改进后的算法同原算法相比,对相同的数据量进行挖掘,算法执行时间明显减少,效益较高.     

可信关联规则及其基于极大团的挖掘算法

目前的关联规则挖掘算法主要依靠基于支持度的剪切策略来减小组合搜索空间.如果挖掘潜在的令人感兴趣的低支持度模式,这种策略并非有效.为此,提出一种新的关联模式—可信关联规则(credible association rule,简称CAR),规则中每个项目的支持度处于同一数量级,规则的置信度直接反映其可信程度,从而可以不必再考虑传统的支持度.同时,提出MaxcliqueMining算法,该算法采用邻接矩阵产生2-项可信集,进而利用极大团思想产生所有可信关联规则提出并证明了几个相关命题以说明这种规则的特点及算法的可行性和有效性.在告警数据集及Pumsb数据集上的实验表明,该算法挖掘CAR具有较高的效率和准确性.     

Knowledge Discovery from Series of Interval Events

Knowledge discovery from data sets can be extensively automated by using data mining software tools. Techniques for mining series of interval events, however, have not been considered. Such time series are common in many applications. In this paper, we propose mining techniques to discover temporal containment relationships in such series. Specifically, an item A is said to contain an item B if an event of type B occurs during the time span of an event of type A, and this is a frequent relationship in the data set. Mining such relationships provides insight about temporal relationships among various items. We implement the technique and analyze trace data collected from a real database application. Experimental results indicate that the proposed mining technique can discover interesting results. We also introduce a quantization technique as a preprocessing step to generalize the method to all time series.