空间科学卫星数据快速处理方法

针对卫星获取的大规模数据进行快速数据处理一直是空间信息处理系统建设中的关键。面对空间科学卫星全天候观测、探测载荷类型多、处理算法多样带来的数据处理难题，现有基于CCSDS标准格式的数据分析方法，难以满足目前在轨的多颗空间科学卫星数据处理系统在正确性和时效性方面的要求。针对空间科学卫星探测数据处理特点，提出了一种空间科学数据快速处理方法，设计两层联合索引结构，将空间科学大数据处理问题转化为索引表和源包数据单元的处理问题，提高了数据处理效率；采用科学工作流技术设计了数据驱动和业务驱动协同的处理框架，支持多样化的空间科学卫星数据处理流程，各类载荷数据处理任务并行调度。实验结果表明，这种方法处理速度可扩展，内存使用较少，已应用于空间科学卫星地面系统中，取得了良好的效果。

A rapid data processing methodfor space science satellites

Rapid data processing for large scale data acquired by satellites has always been the key to build spatial information processing systems. Faced with space science satellite problems such as all weather observation, multiple types of detection loads, and various processing algorithms, the existing data analysis methods based on CCSDS standard format are difficult to satisfy the current requirements of data processing systems for the on orbit space science satellites in correctness and timeliness. According to the observing data characteristics of the space science satellites, the paper proposes a fast data processing method for space science, designs the two layer joint index structure that transforms the processing problem from the space science large data into the index tables and the source packet data units, and improves the efficiency of data processing. The paper designs a processing framework using scientific workflow technique, which supports the cooperation between the data driven processing and the business driven processing of space science satellite data, and also supports various data processing workflows and the parallel scheduling of numerous tasks for different satellite payload types. The experimental results show that this method has good scalability and less memory usage, which has been well applied to the spatial information processing system of space science satellites.