基于流式计算的空间科学卫星数据实时处理

针对空间科学卫星探测数据的实时处理要求越来越高的问题,提出一种基于流计算框架的空间科学卫星数据实时处理方法。首先,根据空间科学卫星数据处理特点对数据流进行抽象分析;然后,对各处理单元的输入输出数据结构进行重新定义;最后,基于流计算框架Storm设计数据流处理并行结构,以适应大规模数据并行处理和分布式计算的要求。对应用该方法开发的空间科学卫星数据处理系统进行测试分析,测试结果显示,在相同条件下数据处理时间比原有系统缩短了一半;数据局部性策略比轮询策略具有更高的吞吐率,数据元组吞吐率平均提高29%。可见采用流式计算框架能够大幅缩短数据处理延迟,提高空间科学卫星数据处理系统的实时性。     

多重索引加时间戳校正的科学卫星源包排序法

卫星载荷数据源包的排序还原是地面数据处理流程中的重要环节,其正确性和完备性直接影响基于数据开展的科学研究.在对国际现行CCSDS标准下的卫星数据组织和星地数传机制可能给地面数据处理带来的数据异常情形与源包排序挑战进行剖析的基础上,提出了"多重索引+时间戳校正"的源包排序方法,不仅能够解决由于空间科学卫星数据采集频率极快...     

基于LabVIEW的卫星有效载荷遥测数据解析处理方法

针对有效载荷遥测数据解析处理难以适应不同卫星载荷的地面测试通用性的问题，提出了一种基于LabVIEW的遥测数据解析处理方法。该方法将底层硬件控制及顶层数据分析功能独立，构建一个UDP网络，以JSON数据作为各独立控制系统间交换接口。遥测解析系统接收底层RS422通信系统送来的JSON格式原始遥测数据包，根据可设置的配置文件完成识别遥测包类型、提取预设原始参量、解析参数、存储显示数据，向顶层控制系统转发JSON格式解析后数据。所设计的测试系统经2个卫星载荷的实际工程应用和3个卫星载荷的桌面仿真测试，实现了不同载荷遥测数据解析通用性，可为快速搭建地面测试系统提供设计参考。     

分布式计算系统中的算法集成方法

算法集成是研制建设综合性数据处理分析系统的一项重要内容.本文基于临近空间科学实验支持系统研制实践,针对面向科学研究需求的数据处理分析系统的算法集成问题,设计了一种大型分布式系统中集成多种类、多版本、多功能算法的方法,通过构建算法插件,满足了此类科学实验支持系统并行开发、插件化调用、灵活扩展、更新迭代的要求.采用该方法集成的系统部署在ZStack私有云平台上,通过对临近空间科学实验中多种大气参数、原位探测参数、生态参数的计算分析实验验证了该方法的有效性.     

一种高效的船舶活动热点海域探测算法

针对传统数据处理工具处理海洋船舶位置大数据效率低、空间特征不敏感的问题,引入Spatial Hadoop空间大数据处理平台,提出一种基于Spatial Hadoop的船舶活动热点海域探测算法。该算法结合空间索引和MapReduce分布式处理框架实现了基于区域统计的热点海域探测方法。从理论上分析,该方法能够高效地探测船舶活动热点海域。利用真实AIS数据进行实验,并从不同海域和算法运行效率两个方面对实验结果进行了对比分析,结果表明该算法具有有效的探测结果和较高的处理效率,验证了理论分析的正确性。     

可移动式卫星地面数据加解密处理系统设计

针对现有卫星地面数据加解密处理系统在灵活性、适应性、稳定性、集成性、兼容性等方面不能较好地兼顾等问题,结合数据处理的任务需求和作业条件,提出并设计一种可移动式的、能够同时处理多颗卫星多路数据的新型通用化、集成化数据处理系统。系统硬件选用高配置加固计算机,软件开发基于多线程并行计算和模块化设计思想,采用C++的Boost库编写。经验收测试,系统能灵活适应固定站和机动站的作业环境,性能满足规定指标要求,圆满完成了多颗卫星型号的多路数据处理任务。     

云计算智能电网大数据驱动的方法研究

针对智能电网数据繁多、用户应用困难的技术问题,提出了智能电网大数据驱动方法,并设计出基于云计算软件平台管理智能电网大数据的系统。通过采用数据挖掘算法对智能电网大数据样本训练、学习,建立多个数据处理模型。根据决策树算法能够实现智能电网数据不同属性数据的分类,使得用户快速从浩瀚的云端数据库中获取目标数据,并将处理后的数据通过区块链系统保存,进而实现数据的永久性存储。不同用户通过签发密钥实现数据的共享,使得数据应用和管理更为便捷有效。本方案为智能电网大数据处理与应用提供了技术参考。     

一种微纳卫星集成化载荷数据处理单元设计

为满足微纳卫星多载荷数据采集处理的需求,设计了一种集成化载荷数据处理单元,该单元作为载荷与微纳卫星进行数据交换的枢纽,具有多路载荷数字量和高精度模拟量采集通道,且具备接收注入数据、进行指令解码、载荷模式控制功能.通过双控制总线冗余、故障总线动态切换,提高了通信可靠性.该单元资源需求低,集成度高,具有一定的通用性.     

基于Kubernetes的卫星遥感数据容器云平台

随着卫星遥感行业进入大数据时代,传统的数据处理平台日渐难以满足卫星遥感数据快速积累的发展需求,同时也对系统在动态调度资源、易维护、弹性扩展、复杂度可控和灵活部署等方面提出了更高的要求。针对上述问题,文章引入容器技术及Kubernetes容器集群管理系统,设计实现了基于Kubernetes的开源容器云平台,将单个业务系统拆分成多个独立运行在相互隔离容器中的服务,实现了应用服务容器的调度和管理、快速部署和迁移,并以GF7高分卫星数据为例进行验证,验证了容器集群能够有效处理卫星遥感数据,提高了卫星遥感数据处理平台的资源利用率和运维效率;文末结合卫星遥感行业的现状及特点,就容器化应用的前景及面临的困难进行了展望分析。     

基于GBDT的卫星工程参数异常检测

卫星及其载荷的在轨运行异常诊断是卫星高效安全运行的重要支持,发展智能、高效的卫星异常检测方法,是卫星地面系统的研究焦点之一.在我国空间科学先导专项系列卫星任务的应用背景下,根据空间科学卫星的数据特性与异常形态,基于梯度提升决策树(gradient boosting decision tree,GBDT)原理构建卫星工程...     

基于流式计算的遥感卫星数据快视处理方法

随着高分辨率遥感卫星数据获取能力和地面数传接收能力的提高，现有遥感卫星快视处理系统的处理负载增大，实时性要求越来越难以满足。针对这些问题，采用流式计算思想提出了一种新的遥感卫星数据快视处理系统设计方法。在分析遥感卫星数据快视处理数据流特点的基础上，应用Storm框架对现有系统进行并行优化，设计遥感数据流处理任务拓扑结构，同时利用消息队列中间件Kafka改进处理单元间数据交换和数据缓存方式。实验表明，该系统在数据吞吐率和可靠性方面测试效果良好。     

卫星遥感及图像处理平台发展

航天科技是国家综合国力和科技实力的重要体现,而卫星遥感则是航天科技转化为生产力最直接、最现实的途径之一。遥感数据获取与分发、数据处理与信息提取是卫星遥感应用的两个基本步骤。随着国家民用空间基础设施规划中的遥感卫星体系稳步推进,以及商业卫星遥感的蓬勃发展,我国的卫星遥感数据获取能力呈现质量齐升之势。但同时,作为卫星遥感应用的基础设施和关键工具,遥感图像处理系统平台逐渐成为制约自主卫星数据应用和空间信息业务发展的重要因素之一。本文围绕卫星遥感对地观测主题,从卫星遥感数据获取能力、卫星遥感数据处理系统平台两方面,对国内外现状进行综述,在此基础上分析了卫星遥感的发展趋势。     

Analysing fundamental properties of marker-based vision system designs

This paper investigates fundamental properties of marker-based vision (MBV) systems. We present a theoretical analysis of the performance of basic tag designs which is extended through simulation to investigate the effects of different processing algorithms. Real-world data are processed and related to the simulated results. Image processing is performed using Cantag, an open-source software toolkit for building marker-based vision (MBV) systems that can identify and accurately locate printed markers in three dimensions. Cantag supports multiple fiducial shapes, payload types, data sizes and image processing algorithms in one framework. This paper explores the design space of tags within the Cantag system, and describes the design parameters and performance characteristics which an application writer can use to select the best tag system for any given scenario.     

基于分级存储的石油勘探地震处理应用研究

石油勘探中“两宽一高”采集技术的深入应用，对地震资料并行处理能力提出了更高的要求，针对存储对地震资料处理应用的影响进行研究。为提高各类应用的运算效率，设计并实现了基于SSD、集群NAS多级分层存储地震资料处理方法，针对高IOPS应用、大数据块并行应用进行算法流程级设计。通过DD与IOZONE基准测试、两种应用实际运行效果，验证了设计方案具备在地震资料处理中应用效率与性价比上的优势，适用于勘探数据中心地震资料处理，具有一定的普适性。     

A review of regional science applications of satellite remote sensing in urban settings

This paper reviews the potential applications of satellite remote sensing to regional science research in urban settings. Regional science is the study of social problems that have a spatial dimension. The availability of satellite remote sensing data has increased significantly in the last two decades, and these data constitute a useful data source for mapping the composition of urban settings and analyzing changes over time. The increasing spatial resolution of commercial satellite imagery has influenced the emergence of new research and applications of regional science in urban settlements because it is now possible to identify individual objects of the urban fabric. The most common applications found in the literature are the detection of urban deprivation hot spots, quality of life index assessment, urban growth analysis, house value estimation, urban population estimation and urban social vulnerability assessment. The satellite remote sensing imagery used in these applications has medium, high or very high spatial resolution, such as images from Landsat MSS, Landsat TM and ETM+, SPOT, ASTER, IRS, Ikonos and QuickBird. Consistent relationships between socio-economic variables derived from censuses and field surveys and proxy variables of vegetation coverage measured from satellite remote sensing data have been found in several cities in the US. Different approaches and techniques have been applied successfully around the world, but local research is always needed to account for the unique elements of each place. Spectral mixture analysis, object-oriented classifications and image texture measures are some of the techniques of image processing that have been implemented with good results. Many regional scientists remain skeptical that satellite remote sensing will produce useful information for their work. More local research is needed to demonstrate the real potential and utility of satellite remote sensing for regional science in urban environments.     

基于LEO的骨干接入空间信息网络与用频策略研究

空间信息网络利用各类空间平台来实现空间信息的实时获取、传输和处理,打破了当前多个独立系统之间资源不能共享的壁垒。立足我国空间基础设施种类繁杂、不成体系、缺少统一的通信标准、综合服务能力差、覆盖范围小的建设现状,坚持“既着眼于现有系统,又兼顾未来发展”的建设原则,在空间信息网络建设的初级阶段,设计了基于双层LEO的空间信息网络的体系架构,即将较高轨道的LEO卫星作为骨干核心网来负责骨干传输功能;将较低轨道的LEO卫星作为热点接入层来负责地面和空间业务节点的接入。同时,为未来空间信息业务提供可扩展的体系特征,譬如未来增加对GEO骨干节点的拓扑关系的考虑,从而解决空间信息网络的轨道资源问题。最后,针对空间信息网络频率资源受限的问题,提出了频率获取策略。     

A robust framework for real-time distributed processing of satellite data

It is estimated that future satellite instruments such as the Advanced Baseline Imager (ABI) and the Hyperspectral Environmental Suite (HES) on the GOES-R series of satellites will provide raw data volume of about 1.5 Terabyte per day. Due to the high data rate, satellite ground data processing will require considerable computing power to process data in real-time. Cluster technologies employing a multi-processor system present the only current economically viable option. To sustain high levels of system reliability and operability in a cluster-oriented operational environment, a fault-tolerant data processing framework is proposed to provide a platform for encapsulating science algorithms for satellite data processing. The science algorithms together with the framework are hosted on a Linux cluster.In this paper we present an architectural model and a system prototype for providing performance, reliability, and scalability of candidate hardware and software for a satellite data processing system. Furthermore, benchmarking results are presented for a selected number of science algorithms for the Geosynchronous Imaging Fourier Transform Spectrometer (GIFTS) instrument showing that considerable performance can be gained without sacrificing the reliability and high availability constraints imposed on the operational cluster system.     

An overview of small satellites in remote sensing*

This article gives a global overview of some aspects of small satellite developments since the launch of Sputnik‐1 50 years ago. These developments are offering new opportunities for remote sensing.

The earliest satellites were small but, as time went on, the satellites that were flown were developed to serve several different projects and they became larger and more expensive and took a long time to design, build and be launched. One of the extreme examples was Envisat. For these large satellites compromises often had to be made between different objectives and different instruments. A failure of the whole system meant the death of many different projects.

The future is likely to see more small satellites, each of which is dedicated to a particular mission objective and carries a single instrument. Through this approach more and more countries around the world are becoming involved in Earth observation from space, not just in using the data from the major established systems but also in constructing their own systems.

There were some small, low‐cost satellites in the early days, but they were overlooked or considered toys by the space community. The first microsatellites were built by enthusiasts of the amateur radio community and launched in the early 1960s. The invention/introduction of the microprocessor in the 1970s represented a quantum jump for the onboard capabilities of a spacecraft. This technology introduction represented a prime catalyst in the development of microsatellites since it enabled small physical structures in support of sophisticated data handling applications. The engineering of microsatellites, which emerged in the early 1980s, took a radical change of approach from the custom design of traditional spacecraft, namely a design‐to‐capability scheme to achieve cost reductions by focusing on available, and existing technologies using a general purpose bus and ‘off‐the‐shelf’ components and instruments.

The new approach of small satellite design was pioneered by Surrey Satellite Technology Ltd (SSTL) of Surrey University, UK. SSTL's lead has now been followed by various companies and space agencies throughout the world. A key feature of this work is the development of microsatellite technology transfer programmes, providing partnerships and on‐the‐job training of engineers and scientists of foreign national organizations in cooperative programmes – in particular to those who were not in a position to start or afford their own space projects – to participate in the development of their own microsatellites.

In addition to discussing these developments, this article also covers small satellite classification, small satellite initiatives in the USA, small satellite development in the rest of the world, some aspects of the technology and applications of small satellites, and small satellites developed by universities, particularly the CubeSats programme.

Today, small satellites are changing the economics of space. These spacecraft embrace cutting edge Commercial Off‐The‐Shelf (COTS) technology, permitting novel and less‐expensive ways to perform meaningful observation missions, although there are various technical challenges. There are several synthetic aperture radar (SAR) and hyperspectral imaging missions on small satellites in operation and in planning.     

Elevation modelling from satellite visible and infrared (VIR) data

Since the early emergence of Earth observation satellites, researchers have investigated different methods of extracting three-dimensional information using satellite data. Apart from a few early stereo images by hand-held photographs acquired during the Gemini and Apollo missions, the first experiments to extract three-dimensional data using stereo viewing from space began with the Earth Terrain Camera flown onboard SkyLab in 1973/74. Since this time, various analogue or digital sensors in the visible spectrum have flown to provide researchers and geoscientists with spatial data to extract and interpret three-dimensional information of the Earth's surface. Although clinometry techniques can be applied with the optical sensor images, stereo viewing of images was and still is the most common method used by the mapping, photogrammetry and remote sensing communities for elevation modelling. The paper will review clinometry and stereoscopy and their applicability to the different satellite sensors (space photographs and scanners). Their performances to extract absolute or relative elevation from various research and commercial organizations are addressed. The respective advantages, difficulties and constraints of the sensors are discussed, as well as the methods and the technologies used for extracting elevation data in an operational context.