Characterizing network traffic behaviour using granule‐based association rule mining

Association rule mining is one important technique to characterize the behaviour of network traffic. However, mining association rules from network traffic data still have three obstacles such as efficiency, huge number of results and insufficiency to represent the behaviour of network traffic. Aiming to tackle these issues, this paper presents a granule‐based association rule mining approach, called association hierarchy mining. The proposed approach adopts top‐down rule mining strategy to directly generate interesting rules according to subjectively specified rule template hierarchies, which improves the efficiency of rule generation and subjectively filters user uninterested rules. The approach also proposes to prune a new type of redundant rules defined by this research to reduce the number of rules. Finally, the approach introduces the concept of diversity, aiming to select the interesting rules for better interpreting the behaviour of network traffic. The experiments performed on the MAWI network traffic traces show the efficiency and effectiveness of the proposed approach. Copyright © 2016 John Wiley & Sons, Ltd.     

关联规则挖掘技术在Web预取中的应用

随着WWW上信息的爆炸性的增长，在如此大量的数据中发现有用的信息变得困难。因此，采用数据挖掘技术从WWW上提取出有价值的知识，提高WWW的服务效率，是具有十分重要的现实意义和广泛的应用前景。本文提出了一种利用关联规则解决网络拥塞问题的方法，即通过对服务器上的日志进行挖掘，得到用户访问序列的关联规则．根据这些规则和用户目前访问网页，把用户后继最有可能访问的网页预先发送给用户，来提高用户浏览速度。     

Spatiotemporal Pattern Mining Technique for Location‐Based Service System

In this paper, we offer a new technique to discover frequent spatiotemporal patterns from a moving object database. Though the search space for spatiotemporal knowledge is extremely challenging, imposing spatial and timing constraints on moving sequences makes the computation feasible. The proposed technique includes two algorithms, AllMOP and MaxMOP, to find all frequent patterns and maximal patterns, respectively. In addition, to support the service provider in sending information to a user in a push‐driven manner, we propose a rule‐based location prediction technique to predict the future location of the user. The idea is to employ the algorithm AllMOP to discover the frequent movement patterns in the user's historical movements, from which frequent movement rules are generated. These rules are then used to estimate the future location of the user. The performance is assessed with respect to precision and recall. The proposed techniques could be quite efficiently applied in a location‐based service (LBS) system in which diverse types of data are integrated to support a variety of LBSs.     

基于项权值变化的完全加权正负关联规则挖掘

本文提出一种基于项权值变化的完全加权正负关联规则挖掘算法,解决了基于项权值变化的负模式挖掘问题.该算法考虑项权值依赖于事务记录的特点,采用新的项集剪枝方法和模式评价框架,通过项集的项内权值比和维数比的简单计算和比较,挖掘有效的完全加权正负关联规则.实验结果表明,与现有无加权正负关联规则挖掘算法比较,本文算法能避免无效的模式出现,其挖掘时间和候选项集数量明显减少,减幅最大分别可达94.09%和88.16%.     

基于频繁模式图的多维关联规则挖掘算法研究

关联规则挖掘是数据挖掘领域中重要的研究分支,频繁项集或频繁谓词集的计算是其中的关键问题.本文针对包括多值属性的关系数据库,以多维关联规则挖掘为目标,研究频繁谓词集的计算方法,提出了MPG算法及IMPG增量算法.MPG算法通过构建频繁模式图MP-graph,按照深度优先搜索方法,动态挖掘频繁谓词集,只需扫描数据库一次.此外,该方法至多增加一次数据库扫描,就能扩展为IMPG算法,进行增量关联规则挖掘.文章分析了算法时间和空间性能,用实验说明了算法的有效性.     

基于改进Apriori算法的气象数据质量控制研究

针对传统气象数据质量控制算法存在的不足,首先提出将Apriori关联规则挖掘算法用于气象数据中,通过Apriori算法挖掘出关联规则;其次分析了Apriori算法存在的不足,提出了一种改进的MC_Apriori算法,通过真实数据仿真表明,新算法在时间性能上更加优越;最后,在原数据的基础上植入部分错误数据,通过与规则库中的关联规则进行规则匹配,找出错误数据率达到93.3%。     

Mining Frequent Sets Using Fuzzy Multiple-Level Association Rules

At present, most of the association rules algorithms are based on the Boolean attribute and single-level association rules mining. But data of the real world has various types, the multi-level and quantitative attributes are got more and more attention. And the most important step is to mine frequent sets. In this paper, we propose an algorithm that is called fuzzy multiple-level association (FMA) rules to mine frequent sets. It is based on the improved Eclat algorithm that is different to many researchers’ proposed algorithms that used the Apriori algorithm. We analyze quantitative data’s frequent sets by using the fuzzy theory, dividing the hierarchy of concept and softening the boundary of attributes’ values and frequency. In this paper, we use the vertical-style data and the improved Eclat algorithm to describe the proposed method, we use this algorithm to analyze the data of Beijing logistics route. Experiments show that the algorithm has a good performance, it has better effectiveness and high efficiency.     

基于数据挖掘技术的字典生成方法

随着信息技术和数据技术的飞速发展,从大量的数据中获取有用的信息和知识变得越来越重要。聚类分析和关联规则挖掘是数据挖掘中的核心技术。文中基于数据挖掘技术和口令因子的概念,提出一种崭新的用户口令字典生成方法,通过对原始的用户口令进行数据挖掘,分析在用户口令中所隐含的口令因子关联规则,将所获得的口令因子关联规则成功地应用在用户口令字典生成中,并在WinRAR文件破解中取得了良好的效果。     

基于用户行为的网络流量模型及自相似性分析

本文在到达网络的用户数服从Ｐｏｉｓｓｏｎ分布这一基本假设基础上,对用户行为做了统计描述,得到一新的网络流量模型。分析表明,该模型不仅可以产生与网络数据相类似的流量特征,而且具有较好的自相似性,其自相似性变化规律也与结果吻合。     

基于改进关联规则的网络入侵检测方法的研究

研究关联规则的高效挖掘算法对于提高入侵检测的准确性和时效性具有非常重要的意义.针对现行的入侵检测方法建立的正常模式和异常模式不够准确、完善,容易造成误警或漏警的问题,本文将改进后的关联规则挖掘算法-XARM和关联规则增量更新算法-SFUP应用于网络入侵检测,提出了新的入侵检测方法,该方法通过挖掘训练审计数据中的频繁项集建立系统和用户的正常行为模型以及入侵行为模型.     

基于源码模式挖掘的软件辅助开发技术研究

由于软件代码的复杂性,对于不了解框架的新手,很难利用开源社区中的代码来开发软件。因此,利用数据挖掘技术挖掘现有代码中的编程模式成为研究热点。文中介绍了频繁项挖掘Apriori算法,并提出了基于源码模式的软件辅助开发方法。它能够根据用户输入的关键字来智能匹配类库中的特定父类,挖掘基于此父类的编程模式,给出优先要重写的方法以及关联规则。实验结果表明,新手可以利用这些编码建议,快速学习一个新的框架,提高开发效率。     

基于项权值排序挖掘的跨语言查询扩展

为了改善自然语言处理应用中长期存在的主题漂移和词不匹配问题,本文首先提出一种加权项集支持度计算方法和基于项权值排序的剪枝方法,给出面向查询扩展的基于项权值排序的加权关联规则挖掘算法,讨论关联规则混合扩展、后件扩展和前件扩展模型,最后提出基于项权值排序挖掘的跨语言查询扩展算法.该算法采用新的支持度和剪枝策略挖掘加权关联规则,根据扩展模型从规则中提取高质量扩展词实现跨语言查询扩展.实验结果表明,与现有基于加权关联规则挖掘的跨语言扩展算法比较,本文扩展算法能有效遏制查询主题漂移和词不匹配问题,可用于各种语言的信息检索以改善检索性能,扩展模型中后件扩展获得最优检索性能,混合扩展的检索性能不如后件扩展和前件扩展,支持度对后件扩展更有效,置信度更有利于提升前件扩展和混合扩展的检索性能.本文挖掘方法可用于文本挖掘、商务数据挖掘和推荐系统以提高其挖掘性能.     

一种无冗余的Web日志挖掘算法

Web日志挖掘是Web数据挖掘的一个重要研究领域。Web日志挖掘通过发现Web日志中用户的访问规律和模式,可以提取出其中潜在的规律和信息,人们对这个领域的研究也日益重视。然而,传统的基于关联规则的Web日志挖掘算法都是基于所有关联规则的。这种方式往往挖掘产生大量的候选规则,而且存在大量冗余的规则。提出了一种新的无冗余的Web日志挖掘算法,该算法通过引入频繁闭项集合最小关联规则的概念,从而解决了以往基于所有关联规则挖掘算法中出现的上述问题。     

Mining Association Rules Based on Deep Pruning Strategies

Today mobile network and various smart devices flourish rapidly. Data collected from the mobile devices and network can bring us huge opportunities to understand some significant characteristics of the users which traditional data cannot. Association rules mining is an extremely important topic in data mining that can make the utmost value of massive data effectively. Apriori algorithm and the improved Apriori ones based on Boolean matrix are the representative ones in association rules mining. Nevertheless, these solutions have their problems. In this paper, we have proposed an algorithm called MAR-DPS, which has some deep pruning strategies containing three methods to compress the size of frequent itemsets and reduce the joining numbers in generating new frequent itemsets. It can also select the appropriate method to generate frequent 2-itemsets when facing different data sets. Extensive experimental results on three different data sets have demonstrated that our MAR-DPS can perform much better than other tested algorithms.     

Recognizing and characterizing dynamics of cellular devices in cellular data network through massive data analysis

The user clients for accessing Internet are increasingly shifting from desktop computers to cellular devices. To be competitive in the rapidly changing market, operators, Internet service providers and application developers are required to have the capability of recognizing the models of cellular devices and understanding the traffic dynamics of cellular data network. In this paper, we propose a novel Jaccard measurement‐based method to recognize cellular device models from network traffic data. This method is implemented as a scalable paralleled MapReduce program and achieves a high accuracy, 91.5%, in the evaluation with 2.9 billion traffic records collected from the real network. Based on the recognition results, we conduct a comprehensive study of three characteristics of network traffic from device model perspective, the network access time, the traffic volume, and the diurnal patterns. The analysis results show that the distribution of network access time can be modeled by a two‐component Gaussian mixture model, and the distribution of traffic volumes is highly skewed and follows the power law. In addition, seven distinct diurnal patterns of cellular device usage are identified by applying unsupervised clustering algorithm on the collected massive traffic data. Copyright © 2014 John Wiley & Sons, Ltd.     

一个保护私有信息的布尔关联规则挖掘算法

本文基于随机响应技术,提出了一种在保护隐私的关联规则挖掘中对数据进行伪装的方法;设计了在伪装的数据集上进行挖掘的算法;分析了算法的效率.实验结果表明,该算法在伪装的数据集上挖掘出的规则与原始规则相比,相对误差不超过2％,并给出了使得相对误差最小时相关参数的取值.     

一种安全事件模糊关联规则挖掘算法

随着信息技术和数据库技术的飞速发展,从大量的数据中获取有用的信息和知识变得越来越重要。模糊关联规则挖掘是数据挖掘中针对数量型属性关联规则发现的一种有效方法。提出了一种基于矩阵的模糊关联规则挖掘算法,并将其应用于网络安全事件关联分析中,通过对DARPA标准数据集的分析,得出了预期数量的关联规则,并成功验证了某些攻击场景,该模糊关联规则挖掘算法取得了较好的实验结果。     

电磁频谱数据的关联规则挖掘

为更合理利用频谱资源以及更好地评估各类电磁环境,本文提出一种基于关联规则挖掘的频谱数据挖掘方案.该方案首先基于一般挖掘流程获取频谱数据中的有用信息,包括异常信息、底噪信息、占用度信息和预定时间功率信息等;再将频谱信息作为关联分析对象,通过构建关联库,构建模糊集,基于模糊关联规则挖掘算法对频谱信息进行系统性的分析.本文对传统的算子选择策略加以改进,使用大尺度参数改进模糊隶属函数.通过实测数据集的验证分析,实验结果表明,频谱信息的强关联规则能反映各种信息之间隐含的关联性以及各种信息出现的频次;基于频谱信息的关联规则挖掘能有效地简化频谱挖掘工作,通过各种信息的关联性可以通过分析一部分频谱信息而得到另外的频谱信息.频谱信息的关联规则可以用于进行电磁无线电环境的评估,选择合适的频谱信息该方案可以应用于各类电磁环境的评估.     

一种新的面向普通用户的多值属性关联规则可视化挖掘方法

针对传统关联规则可视化挖掘方法不利于处理多值属性数据、缺乏展现数据间的频繁模式和关联模式以及效率低下等问题,提出了基于KAF因子和CHF因子的Apriori改进算法进行多值属性关联规则挖掘,实现了一种新的基于概念格的多值属性关联规则可视化方法.运用概念格理论对多值属性数据进行了重新定义和分类,建立了较为完整的挖掘过程参数调整策略,方便用户选择关键属性值进行规则挖掘分析,提高了算法运行速度和挖掘效率.以概念格结构将多值数据组织起来,实现了对频繁项集的可视化展示,以及关联规则的多模式可视化展示.实验结果表明,改进后的挖掘算法具有更好的性能,所提出的可视化形式和已有成果相比具有良好的展现效果.     

一种新的分布式DDoS攻击防御体系

Internet技术的发展和应用,给人们的生产和生活带来了很多便捷,但随之出现的网络安全问题,也成为日益严重的社会问题。针对网络中存在的DDoS攻击进行研究,以分布式并行系统的思想为基础,建立了一种新型DDoS攻击的安全防御体系。该体系通过不同组件间的相互协调、合作,实现了对DDoS攻击的分析及其防御。在对DDoS的攻击流量进行分析的过程中,以数据挖掘的模糊关联规则的方法进行分析,并实现了对攻击源的定位,有效地避免了攻击造成进一步的危害。