| 基于电信运营商固网DPI系统的大数据清洗方案探讨 |
| |
| 引用本文： | 肖明坤,王吉顺.基于电信运营商固网DPI系统的大数据清洗方案探讨[J].电信工程技术与标准化,2015(11). |
| |
| 作者姓名： | 肖明坤 王吉顺 |
| |
| 作者单位： | 江苏省邮电规划设计院有限责任公司,江苏省邮电规划设计院有限责任公司 |
| |
| 摘 要： | 随着大数据时代的到来,传统电信运营商也加入到大数据业务开展的庞大阵营中来。固网DPI系统采集到的HTTP等信息为运营商开展对内、对外的大数据业务提供了重要数据支撑。 由于DPI采集信息中含有大量的垃圾数据,在使用数据前对其进行清洗是非常必要的。文章提出一种针对电信运营商固网HTTP信息的清洗方案,经过现网试点部署验证可到80%以上的清洗率,大大节省了存储空间和网络传输带宽,对运营商开展固网大数据业务具有重要的借鉴意义。
|
| 关 键 词： | 大数据 数据清洗 DPI系统 |
| 收稿时间： | 2015/9/13 0:00:00 |
| 修稿时间： | 2015/9/13 0:00:00 |
A Study of Big Data Cleaning Solution based on DPI System of Operators |
| |
| xiaomingkun and wangjishun.A Study of Big Data Cleaning Solution based on DPI System of Operators[J].Telecom Engineering Technics and Standardization,2015(11). |
| |
| Authors: | xiaomingkun and wangjishun |
| |
| Affiliation: | Jiangsu Posts&Telecommunications Planning and Designning Institute CO.LTD,Jiangsu Posts&Telecommunications Planning and Designning Institute CO.LTD |
| |
| Abstract: | The triditional operators is starting their big data business while the big data is becoming the major trend. Information such as HTTP gained by DPI System provides significant support for the internal and external big data bussiness. Data cleaning becomes a critical step before calculation as large quantity of useless data exists in the raw data provided by DPI. This paper provides a data clean solution based on DPI raw data provided by operators. The solution is verified that rate of data cleaning can be up to 80% while big data business is not impacted by deploying on real network. The solution is valuable when operators deploy big data business since much bandwith and storage can be saved. |
| |
| Keywords: | Big data Data cleaning DPI (Deep Packet Inspection)System |
|
| 点击此处可从《电信工程技术与标准化》浏览原始摘要信息 |
|
点击此处可从《电信工程技术与标准化》下载全文 |
