基于Jaccard的移动终端自动识别并行算法及其MapReduce实现

The ability of accurate and scalable mobile device recognition is critically important for mobile network operators and ISPs to understand their customers’ behaviours and enhance their user experience. In this paper, we propose a novel method for mobile device model recognition by using statistical infor-mation derived from large amounts of mobile network traffic data. Specifically, we create a Jaccard-based coefficient measure method to identify a proper keyword representing each mobile device model from massive unstruc-tured textual HTTP access logs. To handle the large amount of traffic data generated from large mobile networks, this method is designed as a set of parallel algorithms, and is imple-mented through the MapReduce framework which is a distributed parallel programming model with proven low-cost and high-efficiency features. Evaluations using real data sets show that our method can accurately recognise mobile client models while meeting the scalability and pro-ducer-independency requirements of large mobile network operators. Results show that a 91.5% accuracy rate is achieved for rec-ognising mobile client models from 2 billion records, which is dramatically higher than existing solutions.     

Research on data load balancing technology of massive storage systems for wearable devices

Because of the limited memory of the increasing amount of information in current wearable devices, the processing capacity of the servers in the storage system can not keep up with the speed of information growth, resulting in low load balancing, long load balancing time and data processing delay. Therefore, a data load balancing technology is applied to the massive storage systems of wearable devices in this paper. We first analyze the object-oriented load balancing method, and formally describe the dynamic load balancing issues, taking the load balancing as a mapping problem. Then, the task of assigning each data node and the request of the corresponding data node’s actual processing capacity are completed. Different data is allocated to the corresponding data storage node to complete the calculation of the comprehensive weight of the data storage node. According to the load information of each data storage node collected by the scheduler in the storage system, the load weight of the current data storage node is calculated and distributed. The data load balancing of the massive storage system for wearable devices is realized. The experimental results show that the average time of load balancing using this method is 1.75 ?h, which is much lower than the traditional methods. The results show the data load balancing technology of the massive storage system of wearable devices has the advantages of short data load balancing time, high load balancing, strong data processing capability, short processing time and obvious application.     

聚类分析的并行化实现技术研究

文中基于使传统聚类算法能够满足当前大数据分析的对计算效率的需求,采用将传统聚类算法分布式化的方法提高传统聚类算法效率。结合近年来广泛使用的Map Reduce分布式处理模型,对K-means、PAM、CLARA 3种算法进行了分布式化实验,并从数据规模和节点数量两个方面考察、讨论了一些影响并行算法性能的因素。实验分析表明,该方法能够有效地将聚类方法并行化,并可以应用在分布式系统当中。     

基于云计算的海量网络流量数据分析处理研究

随着互联网技术的迅速发展,在我们的生活中网络已经成为我们不可以缺少的重要的组成部分.网络流量监控技术是对网络中海量的流量数据进行分析的重要工具和技术.基于云计算的网络流量监控技术可以对网络流量数据和用户的特征进行更好的分析,可以对用户的上网行为进行深入的挖掘,更好的为用户推荐喜欢的网络内容.本文结合互联网的发展现状对海量网络流量数据分析技术进行了分析,在基于云计算的基础上提出了海量网络流量数据分析的几个关键性技术,对基于云计算的海量网络流量数据分析技术进行了分析和研究.     

大数据分析技术的海量网络流量建模与预测分析

为了有效避免网络拥堵,文中提出基于大数据分析技术的海量网络流量建模与预测分析方法,准确预测网络流量数据.首先利用经验模态分解处理网络流量数据,获取模式函数分量,分析网络流量的短相关性,然后通过确定数据平稳性、计算平稳时间网络流量序列的自相关系数和偏自相关系数,建立网络流量的预测模型,最后进行了网络流量的仿真实验,实验结...     

基于大数据技术的电信网络告警关联分析设计与实现

本文研究将网络告警数据进行合理有效的存储和筛选之后,通过分布式内存计算,将同一时间窗口内同时出现的告警进行采集,计算出告警的共现的频率大小,根据共现频率的高低判断潜在告警之间的关联关系,提供给相关网络专业做出进一步确认。     

Performance of cellular packet CDMA in an integrated voice/data network

This paper presents a simulation-based study of cellular packet CDMA systems operating in an integrated voice/data traffic scenario. Spread-spectrum CDMA provides a suitable framework for resource-shared packet transport capable of combining isochronous (voice, ISDN) and bursty data services. In this work, a general network model for cellular packet CDMA with mixed voice/data traffic is described and used to evaluate the capacity/performance impact of several key system parameters. First, the effect of spreading factor (N) and forward error correction (FEC) rate are studied, confirming earlier work indicating a weak dependence onN and a well-defined optimum code rate in the range of 0.5–0.7 (with BCH coding). Next, the effect of propagation loss coefficient () on network capacity is investigated over a range of possible assumptions for, including both constant and distance-dependent models. The results show that system capacity depends strongly on, varying by as much as a factor of 2 over the range of parameters considered. For a given distance-dependent assumption, performance results are also obtained for different cell sizes in order to understand the overall spatial reuse efficiency achievable in different cellular and microcellular scenarios. This is followed by an investigation of traffic source model effects: first the capacity improvement from voice activity detection VAD) is presented, showing the expected 21 gains. Results for varying proportions of voice and data traffic intensities indicate that the operating efficiency does not change significantly as the proportion of bursty data relative to voice is varied.     

基于数据挖掘的高校学生网络行为分析与管理研究

随着移动互联网和高校校园网的日趋完善,大学生已成为网络社会最主要的社会群体之一。网络已成为大学生校园生活和日常生活不可缺少的重要工具之一,对高校学生网络行为进行分析、管理和引导具有重要的作用和深远的意义。文章重点对网络访问数据预处理、网络访问数据清洗、网页分类等核心环节进行了阐述,并构建了网络行为分析与管理系统,为高校网络部门优化校园网络服务、保障网络安全提供了参考。     

基于数据挖掘的增值业务网管告警关联分析模型

随着增值业务的种类越来越多、规模越来越大,仅仅通过领域专家获得告警关联知识已经无法满足网络维护的需要.因此数据挖掘技术在告警关联分析中的应用应运而生,它在解决告警关联分析问题时首先通过大量历史告警的一些统计规律来发现告警关联规则,然后根据规则分析和预测网络中可能出现的故障.本文介绍了适用于增值业务网管系统的告警关联规则...     

数据挖掘技术在网络故障管理系统中的应用

阐述了计算机网络管理的需求和目标，对网络管理系统进行了综述。并对数据挖掘技术在网络故障管理系统中的应用进行了分析与研究。     

网络智能分析系统的设计与实现

网络数据是一座金矿,尤其是基于网络相关的信令类数据,包含了用户的各种行为特征和信息。本文详细描述了基于网络信令数据的分析平台搭建,重点针对大数据处理,数据存储和分析,以及智能化应用几个层面来分别阐述和介绍设计思路和实现原理,为面向客户和业务的网络智能分析系统建设提供参考。     

基于数据融合的蜂窝无线定位算法研究

本文从数理统计角度,利用TOA和TDOA测量数据并结合扇区信息进行数据融合,然后通过定义可信度函数构造了一种基于移动台位置的动态定位算法,从而实现蜂窝网络定位。最后仿真结果表明这种融合算法在保证决策可靠的前提下,有效提高了决策输出的定位精度。     

Performance analysis of cellular CDMA high‐speed data services

This paper investigates the forward‐link peak and average data rates, throughput, and coverage of a cellular CDMA system for delivering high‐speed wireless data services. The analysis takes into account major aspects commonly found in the forward data channel and applies the generalized Shannon capacity formula for multi‐element antenna (MEA) systems. The study focuses on the physical layer and is flexible for various propagation environments, antenna configurations, multicode allocations, user distributions, and cell site configurations. Numerical results for various multicode allocations are presented for a system model with two‐tier interfering cells operating under a frequency selective slow fading channel with propagation environments specified in the Recommendation ITU‐R M.1225. Copyright © 2006 John Wiley & Sons, Ltd.     

Spectral and energy efficient cognitive radio‐aided heterogeneous cellular network with uplink power adaptation

In future heterogeneous cellular networks, cognitive radio compatible with device to device communication technique can be an aid to further enhance system spectral and energy efficiency. The unlicensed smart devices (SDs) are allowed to detect the available licensed spectrum and utilise the spectrum resource which is detected as not being used by the licensed users. In this work, we propose such a system and provide comprehensive analysis of the effect of selection of SDs' frame structure on the energy efficiency, throughput and interference. Moreover, uplink power control strategy is also considered where the licensed users and SDs adapt the transmit power based on the distance from their reference receivers. The optimal frame structure with power control is investigated under high‐signal‐to‐noise ratio (SNR) and low‐SNR network environments. The impact of power control and optimal sensing time and frame length, on the achievable energy efficiency, throughput and interference are illustrated and analysed by simulation results. It has been also shown that the optimal sensing time and frame length which maximizes the energy efficiency of SDs strictly depends on the power control factor employed in the underlying network such that the considered power control strategy may decrease the energy efficiency of SDs under very low‐SNR regime. Copyright © 2016 John Wiley & Sons, Ltd.     

海量冗余数据干扰下的网络数据捕获和分析系统研究

传统的网络信号捕获方法通过滑动相关法分析本地样本信号与接收网络信号的关联性,实现信号捕获,但当网络中存在海量冗余数据干扰时,占空较小,容易在无信号时间区间形成大量的噪声,导致网络信号误捕获的问题。因此,提出基于Winpcap网络和kd-treed模型的数据捕获方法,设计网络数据捕获和分析系统,捕获系统包括用于捕获和过滤数据的内核模块、用于数据变换和协议分析的用户分析模块并分析系统实现数据捕获的功能模块。通过Winpcap底层网络分析体系结构,实现网络数据包的捕获,采用kd-treed模型过滤海量数据干扰,提高数据捕获的精度。实验结果表明,所设计系统稳定性高,数据捕获精度高。     

大数据环境下的网络舆情管理系统设计

随着网络的快速发展,越来越多的人们通过网络发表个人观点及看法,网络舆情成为社会舆情中的重点对象和主要方式.本文通过对大数据环境下网络舆情及其特点的阐述、分析,结合数据挖掘、文本情感分析等技术,初步构建出了网络舆情管理系统的模型.     

基于电信大数据的5G网络海量用户复访行为预测模型

5G网络中的用户会产生大量的访问数据，导致用户复访行为难以精准预测，因此提出基于电信大数据的5G网络海量用户复访行为预测模型。从电信大数据中提取用户上网历史行为特征数据，构建数据集。引入多阶加权马尔可夫链模型，通过计算各阶自相关系数，得到模型权重值，计算模型的统计量。经过分析后得到各阶步长的马尔可夫氏链一步转移概率矩阵，从而实现对5G网络海量用户复访行为的精准预测。实验结果表明，该模型拥有最低的均值误差和标准差，以及最高的精度、查全率、查准率、F1指标，可证明该方法在预测用户复访行为方面有着非常明显的优势。     

校园网论坛评论的人类动力学分析研究

基于校园网论坛评论的数据统计,从群体层面和个体层面深入挖掘学生用户在校园网论坛上活动行为的时间间隔分布的统计特性,以及给出学生群体兴趣度消失模型的实证研究.研究发现,论坛整体评论时间间隔以及用户个体活动时间隔服从幂律分布,学生群体兴趣度消失模型中,学生群体对于某个话题的兴趣逐渐消失,评论的时间间隔服从幂率分布,且对于比较敏感的话题,会存在兴趣再次上升的情况.     

Energy efficiency analysis of heterogeneous cellular network based on Thomas cluster process

In the existing heterogeneous cellular networks (HCN),the rapid increasing of small cells caused a problem that the hotspots where users were clustered generated a huge amount of energy consumption.To improve the energy efficiency of the networks,the spatial model was researched.The users in hotspots in the HCN were modeled as the Thomas cluster process,combined with the max mean bias received signal power (max-BRP) strategy to achieve base station load balancing,and the user’s association probability,coverage probability and the network’s energy efficiency were derived using tool from stochastic geometry.The simulation shows that the derived formulas are accurate,and the network’s energy efficiency can be greatly improved by setting appropriate bias factors.     

网元性能及业务数据特征精准分析方法

传统的网络性能分析以及业务数据分析,都是通过网管KPI 确定的,网络指标出现异常则需要根据优化人员经验进行问题定位。对问题定位的效率取决于技术人员的水平,同时网络业务数据特征的分析粒度也很粗,不能精细化、全面化、多样化地分析用户特征。随着统一DPI系统的引入,展示了用户会话过程中涉及的最细粒度的业务状态以及网络信道质量。为了更加准确地反映小区网络质量和用户真实业务体验,所研究的网元性能及业务数据分析方法采用数据切片和关联的方式,更加精细化地将体现网络质量的控制面信令XDR（X data recording,呼叫数据记录的扩展）数据和用户面业务XDR数据关联在一起分析。实现分应用、分业务拓展的用户感知分析,明确质差用户或场景等情况下XDR的数据特征,从而提升网络优化效率、提高用户感知。