按列存储的配电网监测数据包区间编码正规化压缩处理

针对不断增长的配电网按行存储数据的模式造成数据体量持续增大、压缩率难以提高的问题,利用按列存储的数据包架构和扩展区间编码的正规化处理技巧,提出了一种大体量配电网监测信息的压缩处理新方法。以列数据包为底层存储、知识网格为组织架构,将区间编码和正规化处理融入底层监测数据包的并行压缩处理进程中。以北京某动车段的配电网调度监测工程数据为算例,进行列数据包区间编码的正规化压缩测试。仿真结果表明:扩展区间编码的正规化处理技术可使监测信息压缩率提高至94%,获得了比已有云计算集群更好的压缩效果,通过动态调整映射区间,即可方便实现在无穷区间中对海量化监测信息编码压缩的工程应用需求。

Range encoding normalization compression processing for distribution network monitoring data pack with column storage

Aiming at the continually increasing large data sets and the difficulty of improving compression ratio caused by the increasing data storage mode by rows in distribution network, a new compression method for large distribution network monitoring data is proposed based on the data pack structure of column storage and the normalization processing of extended range encoding. Taking column data pack as underlying storage and knowledge grid as organization structure, the range encoding and normalization are integrated into the parallel compression process of the underlying monitoring data pack. Taking the distribution network monitoring data of Beijing train set depot section as an example, the column data pack range encoding normalization compression test is carried out. Simulative results show that the normalization of extension range encoding technology can increase the compression ratio of monitoring information to 94%,the compression effect is better than the existing cloud computing cluster, and the engineering application requirements of encoding the massive monitoring information in the infinite interval can be realized by dynamically adjusting the mapping range.