基于MapReduce的新会话识别方法

Web日志预处理因其输出结果的重要性而受到越来越多的重视,同时Hadoop对海量数据的分布式处理也得到广泛研究和应用,因此使用MapReduce进行Web日志预处理成为一种必然的发展趋势。为了提高会话识别结果的准确率,在分析会话识别算法研究现状的基础上,提出一种基于网络拓扑结构和动态阈值相结合的新会话识别方法并讨论其优势所在,接着用MapReduce模型实现新方法的分布式处理,最后通过对比实验分析验证MapReduce模型实现新算法的高效性和高精确度。

A new session identification method based on MapReduce

Web log preprocessing attracts more and more attention due to the importance of its output result. Meanwhile distributed processing of massive data based on Hadoop is being widely studied and applied, so Web log preprocessing with MapReduce becomes an inevitable development trend. In order to improve the accuracy of session identification results, we propose a new method to identify user session based on network topology and dynamic threshold. The current research state is analyzed and the advantages of this method are also discussed. Then, the MapReduce model is used to implement the distributed processing of the new method. Experimental results demonstrate high efficiency and high accuracy of the proposed method.