油气长输管道大数据整合技术方法研究

随着管道数据种类的增加和数据量的增大,管道大数据应运而生,同时也为管道数据的存储、管理等带来了巨大的挑战。以全生命周期管道数据为整合范围,采用闭合式的数据采集流程完成了管道大数据的采集整理。以APDM为基础的包含逻辑和物理结构的管道数据模型可完成对大数据的存储,并以各管节信息为基础数据,将采集的管道大数据按照空间位置关系进行匹配,使各类数据能够与环焊缝信息一一对应。利用多维统计分析、数据挖掘等算法实现对管道完整性大数据的相关性分析,并且完成管道大数据的整合对齐。通过对管道大数据采集、存储、处理、对齐、优化等过程的研究,确定了大数据的整合方法,能够很好地将全生命周期的管道数据进行处理,这对油气长输管道公司管道业务管理、决策分析及风险评估有着重要意义。

Research on Integration Technology Method of Oil and Gas Long Distance Pipeline Big Data

With the increase of pipeline data types and the increase of data volume,pipeline big data is born at the right moment,however,it also brings huge challenges to the storage and management of pipeline data.With the life cycle pipeline data as the integration scope,the closed data collection process is used to complete the collection and collation of pipeline big data.Based on APDM,a pipeline data model including logic and physical structure completes the storage of big data.Based on the information of each section,the collected big data of pipeline is matched according to the spatial position relationship,so that the data can correspond to the ring weld information one-to-one.The multi-dimensional statistical analysis,data mining and other algorithms are used to realize the correlation analysis of pipeline integrity big data,and the integration alignment of pipeline big data is completed.Through the research on the process of pipeline big data collection,storage,processing,alignment,optimization,etc.,the integration method of big data is determined,which can process the pipeline data of whole life cycle very well.It is of great significance to pipeline business management,decision analysis and risk assessment of oil and gas pipeline companies.