Efficient data mining for calling path patterns in GSM networks

In this paper, we explore a new data mining capability that involves mining calling path patterns in global system for mobile communication (GSM) networks. Our proposed method consists of two phases. First, we devise a data structure to convert the original calling paths in the log file into a frequent calling path graph. Second, we design an algorithm to mine the calling path patterns from the frequent calling path graph obtained. By using the frequent calling path graph to mine the calling path patterns, our proposed algorithm does not generate unnecessary candidate patterns and requires less database scans. If the corresponding calling path graph of the GSM network can be fitted in the main memory, our proposed algorithm scans the database only once. Otherwise, the cellular structure of the GSM network is divided into several partitions so that the corresponding calling path sub-graph of each partition can be fitted in the main memory. The number of database scans for this case is equal to the number of partitioned sub-graphs. Therefore, our proposed algorithm is more efficient than the PrefixSpan and a priori-like approaches. The experimental results show that our proposed algorithm outperforms the a priori-like and PrefixSpan approaches by several orders of magnitude.     

基于Bitmap的序列模式挖掘的改进算法

结合BBSP,提出了一种称做最终位置归纳序列模式挖掘(LPI-SPM)的新算法,该算法可以有效地从大型数据库中获取所有的频繁序列模式。该策略与以前工作的不同点在于:当判断一个序列是否是模式时,通过扫描数据库创建S-矩阵来实现(PrefixSpan)或者通过对候选项进行交运算(SPADE)或并运算(BBSP)统计其数量来实现。相反,在基于下列事实的基础上LPI-SPN会很容易实施这一过程,即若一个项的最终位置小于当前前缀位置,在相同的顾客序列中,该项就不会出现在当前前缀的后面。LPI-SPM在序列挖掘过程中可以大大缩减搜索空间,而且挖掘序列模式的效力可观。实验结果表明,在各种数据集合中LPI-SPM胜过BBSP三倍。     

Effective database transformation and efficient support computation for mining sequential patterns

In this paper, we propose a novel algorithm for mining frequent sequences from transaction databases. The transactions of the same customers form a set of customer sequences. A sequence (an ordered list of itemsets) is frequent if the number of customer sequences containing it satisfies the user-specified threshold. The 1-sequence is a special type of sequences because it consists of only a single itemset instead of an ordered list, while the k-sequence is a sequence composed of k itemsets. Compared with the cost of mining frequent k-sequences (k ≥ 2), the cost of mining frequent 1-sequences is negligible. We adopt a two-phase architecture to find the two types of frequent sequences separately in order that the discovery of frequent k-sequences can be well designed and optimized. For efficient frequent k-sequence mining, every frequent 1-sequence is encoded as a unique symbol and the database is transformed into one constituted by the symbols. We find that it is unnecessary to encode all the frequent 1-seqences, and make full use of the discovered frequent 1-sequences to transform the database into one with a smaller size. For every k ≥ 2, the customer sequences in the transformed database are scanned to find all the frequent k-sequences. We devise the compact representation for a customer sequence and elaborate the method to enumerate all distinct subsequences from a customer sequence without redundant scans. The soundness of the proposed approach is verified and a number of experiments are performed. The results show that our approach outperforms the previous works in both scalability and execution time.     

基于抽样技术的序列模式的维护

维护已发现的序列模式的方法主要有两种：一种是简单地利用已有的挖掘序列模式算法对更新后的整个数据库进行操作,这种方法涉及的数据库中的数据不仅有改变的部分而且有未改变的部分,而未改变的数据数量很大,当更新频率高时,代价是非常大的;另一种方法是根据库中记录数目改变的多少来决定何时对整个数据库进行操作,但是记录数目数据并不能代表序列模式化亦大,因此利用样品抽样的方法来评估序列模式改变的程度,并根据改变的程度决定何时对整个数据库进行操作来更新序列模式,从而较好地解决序列模式维护的问题,能高效地、准确地发现序列模式。     

An efficient algorithm for mining frequent inter-transaction patterns

In this paper, we propose an efficient method for mining all frequent inter-transaction patterns. The method consists of two phases. First, we devise two data structures: a dat-list, which stores the item information used to find frequent inter-transaction patterns; and an ITP-tree, which stores the discovered frequent inter-transaction patterns. In the second phase, we apply an algorithm, called ITP-Miner (Inter-Transaction Patterns Miner), to mine all frequent inter-transaction patterns. By using the ITP-tree, the algorithm requires only one database scan and can localize joining, pruning, and support counting to a small number of dat-lists. The experiment results show that the ITP-Miner algorithm outperforms the FITI (First Intra Then Inter) algorithm by one order of magnitude.     

基于压缩FP-树和数组技术的频繁模式挖掘算法

FP-growth算法是目前较高效的频繁模式挖掘算法之一.它只需扫描数据库两次,而且不需要产生和测试候选集,避免了这些费时的工作,因此该算法具有较高的效率.然而,FP-growth算法需要递归地生成大量的条件FP-树,这耗费了大量的存储空间和时间.综合已有的几项优势技术,提出了一种频繁模式挖掘算法CFPmine. 一是采用了基于压缩FP-树的约束子树的挖掘方法,避免在挖掘过程中生成条件FP-树,减少内存占用;二是采用基于数组的技术,减少FP-树的遍历时间,提高算法的效率.另外,在算法中还实现了统一的内存管理.实验结果表明,CFPmine是一个高效的频繁模式挖掘算法,其性能优于Apriori,Eclat和FP-growth算法,而需要的内存却少于FP-growth算法.     

基于Aproiri算法的频繁项集挖掘优化方法

为了进一步降低扫描数据库的次数和减轻内存负担,从而更好地提高挖掘频繁项集的效率,一种基于Apriori的优化算法（M-Apriori）被提出. 该方法通过构建频繁状态矩阵来存放项集的频繁状态,构建事务布尔矩阵来存放事务与项集的关系,此算法只需在初始化阶段扫描一次数据库产生初始的频繁状态矩阵和事务布尔矩阵,并在此基础上直接递推产生所有的频繁项集. 实验证明,与Apriori算法相比,M-Apriori算法具有更好的性能与效率.     

基于矩阵约简的Apriori算法改进

 Apriori算法在搜索频繁项集过程中,通常需要对数据库进行多次的重复扫描和产生大量无用的候选集,针对此问题提出一种基于矩阵约简的Apriori改进算法。该算法只需扫描一次数据库,将数据库信息转换成布尔矩阵,根据频繁k-项集的性质推出的结论来约简数据结构,有效地降低无效候选项集的生成规模。通过对已有算法的对比,验证该算法能有效地提高挖掘频繁项集的效     

面向审计的严格约束的序列挖掘算法

网络安全审计数据具有很强的时间特征。提出了面向审计基于SPAD算法的严格约束的序列挖掘快速算法(Sequence mIning with Strict Constraints，SISC)，它充分利用了序列数据的时间和属性相关的特征指导挖掘，并使用严格的属性模式裁减概念等价类，提高了规则的有用度。最后在真实的审计数据集上的试验结果表明， SISC的效率优于SPADE，尤其当项的个数远大于属性的个数的时候。     

A framework for mining interesting high utility patterns with a strong frequency affinity

High utility pattern (HUP) mining is one of the most important research issues in data mining. Although HUP mining extracts important knowledge from databases, it requires long calculations and multiple database scans. Therefore, HUP mining is often unsuitable for real-time data processing schemes such as data streams. Furthermore, many HUPs may be unimportant due to the poor correlations among the items inside of them. Hence,the fast discovery of fewer but more important HUPs would be very useful in many practical domains. In this paper, we propose a novel framework to introduce a very useful measure, called frequency affinity, among the items in a HUP and the concept of interesting HUP with a strong frequency affinity for the fast discovery of more applicable knowledge. Moreover, we propose a new tree structure, utility tree based on frequency affinity (UTFA), and a novel algorithm, high utility interesting pattern mining (HUIPM), for single-pass mining of HUIPs from a database. Our approach mines fewer but more valuable HUPs, significantly reduces the overall runtime of existing HUP mining algorithms and is applicable to real-time data processing. Extensive performance analyses show that the proposed HUIPM algorithm is very efficient and scalable for interesting HUP mining with a strong frequency affinity.