A Visual Global Positioning System for Unmanned Aerial Vehicles Used in Photogrammetric Applications

The combination of photogrammetric aerial and terrestrial recording methods can provide new opportunities for photogrammetric applications. A UAV (Unmanned Aerial Vehicle), in our case a helicopter system, can cover both the aerial and quasi-terrestrial image acquisition methods. A UAV can be equipped with an on-board high resolution camera and a priori knowledge of the operating area where to perform photogrammetric tasks. In this general scenario our paper proposes vision-based techniques for localizing a UAV. Only natural landmarks provided by a feature tracking algorithm will be considered, without the help of visual beacons or landmarks with known positions. The novel idea is to perform global localization, position tracking and localization failure recovery (kidnapping) based only on visual matching between current view and available georeferenced satellite images. The matching is based on SIFT features and the system estimates the position of the UAV and its altitude on the base of the reference image. The vision system replaces the GPS signal combining position information from visual odometry and georeferenced imagery. Georeferenced satellite or aerial images must be available on-board beforehand or downloaded during the flight. The growing availability of high resolution satellite images (e.g., provided by Google Earth or other local information sources) makes this topic very interesting and timely. Experiments with both synthetic (i.e., taken from satellites or datasets and pre elaborated) and real world images have been performed to test the accuracy and the robustness of our method. Results show sufficient performance if compared with common GPS systems and give a good performance also in the altitude estimation, even if in this last case there are only preliminary results.     

AUTOKITE

An experimental kite-plane capable of autonomous aerial imaging is introduced as a viable low-cost small-scale civilian UAV imaging platform ideal for field use. The AUTOKITE fulfills a need currently unmet by other fully automated Unmanned Aerial Vehicles (UAVs), resulting from ease of operation, extended flight time, and overall reliability. The AUTOKITE is outfitted with an off-the-shelf autopilot system, and has demonstrated fully autonomous flight in field deployments while collecting high-resolution (~12 cm/pixel) images. The AUTOKITE has been used to map regions historically prone to earthquakes along the Southern San Andreas Fault in California. Comparative image methods enabled by photogrammetric software, like Agisoft’s PhotoScan, are then used to discern Structure-from-Motion (SfM) from a multitude of aerial images taken by AUTOKITE (Fonstad, Earth Surf. Process. Landf. 38:421–430, 2013). Processing SfM data from overlapping images results in the creation of Digital Elevation Models (DEMs) and Orthophotos for geographic areas of interest. In addition to sample data sets illustrating the SfM process, The AUTOKITE is compared with three alternative UAV systems, and payload integration/automation details are discussed.     

Fusion of Data and Knowledge for Safe UAV Landing

Autonomous Unmanned Aerial Vehicles (UAVs) have the potential to significantly improve current working practices for a variety of applications including aerial surveillance and search-and-rescue. However before UAVs can be widely integrated into civilian airspace there are a number of technical challenges which must be overcome including provision of an autonomous method of landing which would be executed in the event of an emergency. A fundamental component of autonomous landing is safe landing zone detection of which terrain classification is a major constituent. Presented in this paper is an extension of the Multi-Modal Expectation Maximization algorithm which combines data in the form of multiple images of the same scene, with knowledge in the form of historic training data and Ordnance Survey map information to compute updated class parameters. These updated parameters are subsequently used to classify the terrain of an area based on the pixel data contained within the images. An image''s contribution to the classification of an area is then apportioned according to its coverage of that area. Preliminary results are presented based on aerial imagery of the Antrim Plateau region in Northern Ireland which indicates potential in the approach used.     

Local radiometric correction of digital aerial photographs for multi source forest inventory

Digitized colour infrared images are a potential information source for multi-source forest inventory applications and particularly for estimating the forest characteristics of relatively small areas. The main problem in computer-aided image analysis of aerial photographs is the presence of bidirectional reflectance, which causes the spectral values of the image pixels to depend on their location in the image. Because of this, and the lack of operational methods for radiometric correction, the full potential of aerial photographs has not been utilised. This paper presents a local radiometric correction method that can be used for reducing the effect of bidirectional reflectance. The method employs satellite images that are less affected by the bidirectional reflectance for the local adjustment of the registered pixel values of aerial photographs. Because of the different spatial resolution of the aerial and satellite imagery, the local correction coefficients are computed for units that are larger than a single aerial photo pixel. In this way, the general level of brightness of the correction units can be determined on the basis of the satellite imagery while retaining the finer spatial resolution of the original aerial photo. The main advantage of the suggested method is that it does not require complex mathematical models for simulating the effect of bidirectional reflectance, neither does it require a priori knowledge of the actual forest attributes in the inventory area, but relies only on the image data.     

Timely autonomous identification of UAV safe landing zones

For many applications such as environmental monitoring in the aftermath of a natural disaster and mountain search-and-rescue, swarms of autonomous Unmanned Aerial Vehicles (UAVs) have the potential to provide a highly versatile and often relatively inexpensive sensing platform. Their ability to operate as an ‘eye-in-the-sky’, processing and relaying real-time colour imagery and other sensor readings facilitate the removal of humans from situations which may be considered dull, dangerous or dirty. However, as with manned aircraft they are likely to encounter errors, the most serious of which may require the UAV to land as quickly and safely as possible. Within this paper we therefore present novel work on autonomously identifying Safe Landing Zones (SLZs) which can be utilised upon occurrence of a safety critical event. Safe Landing Zones are detected and subsequently assigned a safety score either solely using multichannel aerial imagery or, whenever practicable by fusing knowledge in the form of Ordnance Survey (OS) map data with such imagery. Given the real-time nature of the problem we subsequently model two SLZ detection options one of which utilises knowledge enabling the UAV to choose an optimal, viable solution. Results are presented based on colour aerial imagery captured during manned flight demonstrating practical potential in the methods discussed.     

UAVs surpassing satellites and aircraft in remote sensing over China

ABSTRACT

Many users are now showing strong interest in UAV RS (Unmanned Aerial Vehicle Remote Sensing) due to its easy accessibility. UAVs have become popular platforms for remote sensing data acquisition. In a number of practical and time constrained circumstances, UAV RS data have been widely used as a substitute for traditional satellite remote sensing data. However, airspace-related regulations are far behind the rapid growth in the number of UAVs and their wide applications. Much effort of the network-based UAV RS have been made by the UAV RS group of the Institute of Geographic Sciences and Natural Resources Research, Chinese Academy of Sciences (IGSNRR, CAS), which proposed the concept of UAV RS data carrier. UAV RS data carrier refers to UAV RS data platform with task planning, data storage, image processing, product generation and output products for various UAVs. An ongoing effort to create a nationwide UAV RS network in addition to an existing ground observational network is being carried out in China.     

Mapping and Tracking

This article presents the implementation of decentralized data fusion (DDF) and cooperative control algorithms on an unmanned aerial system (UAS).We conduct a number of demonstrations with a pair of unmanned aerial vehicles (UAVs) performing an information-gathering mission, and we show that significant benefits can be achieved by enabling cooperation through the sharing of information between members of the team. The objective is to utilize the UAV team to estimate the position and velocity of a number of ground-based features. The UAVs are given some prior knowledge of the feature states and are required to gather further information above a predefined threshold. This situation models a scenario where initial information is made available from an external source (e.g., a high-flying UAV or satellite imagery), which then prompts the start of the feature-localizationmission.     

Performance of different spectral and textural aerial photograph features in multi-source forest inventory

Most multi-source forest inventory (MSFI) applications have thus far been based on the use of medium resolution satellite imagery, such as Landsat TM. The high plot and stand level estimation errors of these applications have, however, restricted their use in forest management planning. One reason suggested for the high estimation errors has been the coarse spatial resolution of the imagery employed. Therefore, very high spatial resolution (VHR) imagery sources provide interesting data for stand-level inventory applications. However, digital interpretation of VHR imagery, such as aerial photographs, is more complicated than the use of traditional satellite imagery. Pixel-by-pixel analysis is not applicable to VHR imagery because a single pixel is small in relation to the object of interest, i.e. a forest stand, and therefore it does not adequately represent the spectral properties of a stand. Additionally in aerial photographs, the spectral properties of the objects are dependent on their location in the image. Therefore, MSFI applications based on aerial imagery must employ features that are less sensitive to their location in the image and that have been derived using the spatial neighborhood of each pixel, e.g. a square-shaped window of pixels. In this experiment several spectral and textural features were extracted from color-infrared aerial photographs and employed in estimation of forest attributes. The features were extracted from original, normalized difference vegetation index and channel ratio images. The correlations between the extracted image features and forest attributes measured from sample plots were examined. Additionally, the spectral and textural features were used for estimating the forest attributes of sample plots, applying the k nearest neighbor estimation method. The results show that several spectral and textural image features that are moderately or well correlated with the forest attributes. Furthermore, the accuracy of forest attribute estimation can be significantly improved by a careful selection of image features.     

斑块状植被遥感检测研究进展

斑块状植被是世界上干旱—半干旱区常见的景观类型,对于它们的形成、结构和演替研究能够提高人们对干旱—半干旱地区生态系统动态及其重要的生态水文过程的理解,具有重要的理论研究意义和应用价值.传统的基于地面调查和长期定位观测的方法观测范围有限,已无法满足目前区域斑块状植被分布及其空间格局特征研究的需要.利用遥感技术快速重复获取...     

A Survey and Categorization of Small Low-Cost Unmanned Aerial Vehicle System Identification

Remote sensing has traditionally be done with satellites and manned aircraft. While these methods can yield useful scientific data, satellites and manned aircraft have limitations in data frequency, process time, and real time re-tasking. Small low-cost unmanned aerial vehicles (UAVs) can bridge the gap for personal remote sensing for scientific data. Precision aerial imagery and sensor data requires an accurate dynamics model of the vehicle for controller development. One method of developing a dynamics model is system identification (system ID). The purpose of this paper is to provide a survey and categorization of current methods and applications of system ID for small low-cost UAVs. This paper also provides background information on the process of system ID with in-depth discussion on practical implementation for UAVs. This survey divides the summaries of system ID research into five UAV groups: helicopter, fixed-wing, multirotor, flapping-wing, and lighter-than-air. The research literature is tabulated into five corresponding UAV groups for further research.     

Integration of Corona and Landsat Thematic Mapper data for bedrock geological studies in the high Arctic

The Severnaya Zemlya Archipelago near the continental edge in the Russian high Arctic is one of few land areas along the Eurasian Arctic margin. It is of particular interest for investigating the Arctic's tectonic history. This study focuses on the Palaeozoic bedrock of October Revolution Island. In the Russian high Arctic detailed topographic maps and aerial photography often are not available. The potential of low-cost satellite imagery as a substitute is shown in this study. High-resolution Corona KH-4A panchromatic satellite imagery and Landsat Thematic Mapper (TM) multispectral data have been integrated. In combination with field investigations in key areas, these data provide the basis for new interpretations of the geology. Corona images were digitized and georeferenced to provide a basis for conventional and digital geological mapping. Merging Corona and Landsat TM data resulted in a high-resolution multispectral image of enhanced interpretability. Lithological contacts have been traced, supported by a bedrock image extracted from the Landsat TM data. Stereoscopic coverage of the Corona KH-4A photographic sensor allowed a structural interpretation. All results were integrated into a geological interpretation of southern October Revolution Island which provides an encouraging platform for further work in the high Arctic.     

Well site extraction from Landsat-5 TM imagery using an object- and pixel-based image analysis method

Well sites, including both well pads and exploratory core holes, are small polygonal landscape disturbance features approximately one half to one hectare (0.5–1 ha) in area, resulting from oil and gas exploration activities. Automatic extraction and monitoring of such small features using remote-sensing technology at regional scales has always been desirable for wildlife habitat monitoring and environmental planning and modelling. Due to the vast disturbances of well sites in a province like Alberta, Canada, high-resolution imagery is not practical for well site extraction. For operational purposes, mid-resolution and cost-effective satellite imagery such as Landsat is the choice. However, automatic well site extraction using mid-resolution satellite imagery is a challenging task. Wells are typically less than three pixels in width and length in a Landsat multispectral image. Furthermore, the spectral contrast between the well site pixels and the surrounding areas is low due to vegetation regrowth and the spectral complexity of the surrounding environment. This article presents a novel methodology for automatic extraction of well sites from Landsat-5 TM imagery. The method combines both pixel- and object-based image analyses and contains three major steps: geometric enhancement, segmentation, and well site extraction. The method was applied to Landsat-5 TM images acquired over Fort McMurray, Alberta, Canada. For accuracy assessment, four regions of interest were selected and the results of the proposed automatic method were evaluated against visual inspection of the Landsat-8 pan-sharpened image. The method results in a total average correctness, completeness, and quality measures of about 80, 96, and 77%, respectively over the four sites. In addition, the method is very fast as an entire Landsat scene is processed in less than 10 minutes. The method is an operational approach for automatic detection of well sites over the entire province and can dramatically reduce the labour cost of manual digitization for monitoring and updating well site maps.     

Spatial accuracy of orthorectified IKONOS imagery and historical aerial photographs across five sites in China

High‐resolution (?1?m) satellite imagery and archival World War II era (WW2) aerial photographs are currently available to support high‐resolution long‐term change measurements at sites across China. A major limitation to these measurements is the spatial accuracy with which this imagery can be orthorectified and co‐registered. We orthorectified IKONOS 1?m resolution GEO‐format imagery and WW2 aerial photographs across five 100?km² rural sites in China with terrain ranging from flat to hilly to mountainous. Ground control points (GCPs) were collected uniformly across 100?km² IKONOS scenes using a differential Global Positioning Systems (GPS) field campaign. WW2 aerial photos were co‐registered to orthorectified IKONOS imagery using bundle block adjustment and rational function models. GCP precision, terrain relief and the number and distribution of GCPs significantly influenced image orthorectification accuracy. Root mean square errors (RMSEs) at GCPs for IKONOS imagery were <2.0?m (0.9–2.0?m) for all sites except the most heterogeneous site (Sichuan Province, 2.6?m), meeting 1:12?000 to 1:4800 US National Map Accuracy Standards and equalling IKONOS Precision and Pro format accuracy standards. RMSEs for WW2 aerial photos ranged from 0.2 to 3.5?m at GCPs and from 4.4 to 6.2?m at independent checkpoints (ICPs), meeting minimum requirements for high‐resolution change detection.     

Visual 3-D SLAM from UAVs

The aim of the paper is to present, test and discuss the implementation of Visual SLAM techniques to images taken from Unmanned Aerial Vehicles (UAVs) outdoors, in partially structured environments. Every issue of the whole process is discussed in order to obtain more accurate localization and mapping from UAVs flights. Firstly, the issues related to the visual features of objects in the scene, their distance to the UAV, and the related image acquisition system and their calibration are evaluated for improving the whole process. Other important, considered issues are related to the image processing techniques, such as interest point detection, the matching procedure and the scaling factor. The whole system has been tested using the COLIBRI mini UAV in partially structured environments. The results that have been obtained for localization, tested against the GPS information of the flights, show that Visual SLAM delivers reliable localization and mapping that makes it suitable for some outdoors applications when flying UAVs.     

无人机航拍图像语义分割研究综述

随着无人机技术的快速发展,无人机在研究领域和工业应用方面受到了广泛的关注。图像和视频是无人机感知周围环境的重要途径。图像语义分割是计算机视觉领域的研究热点,在无人驾驶、智能机器人等场景中应用广泛。无人机航拍图像语义分割是在无人机航拍图像的基础上,运用语义分割技术使无人机获得场景目标智能感知能力。介绍了语义分割技术和无人机的应用发展、相关无人机航拍数据集、无人机航拍图像特点和常用语义分割评价指标。针对无人机航拍的特点介绍了相关语义分割方法,包括小目标、模型实时性和多尺度整合等方面。综述无人机语义分割相关应用,包括线检测、农业和建筑物提取等方向,并分析无人机语义分割未来发展趋势和挑战。     

Radiometric homogenisation of aerial images by calibrating with satellite data

The use of very high resolution (VHR) aerial imagery for quantitative remote sensing has been limited by unwanted radiometric variation over temporal and spatial extents. In this paper we propose a simple yet effective technique for the radiometric homogenisation of the digital numbers of aerial images. The technique requires a collocated and concurrent, well-calibrated satellite image as surface reflectance reference to which the aerial images are calibrated. The bands of the reference satellite sensor should be spectrally similar to those of the aerial sensor. Using radiative transfer theory, we show that a spatially varying local linear model can be used to approximate the relationship between the surface reflectance of the reference image and the digital numbers of the aerial images. The model parameters for each satellite pixel location are estimated using least squares regression inside a small sliding window. The technique was applied to a set of aerial images captured over multiple days with an Intergraph Digital Mapping Camera (DMC) system. A near-concurrent Moderate Resolution Imaging Spectroradiometer (MODIS) nadir bidirectional reflectance distribution function (BRDF) adjusted reflectance image was used as the reflectance reference dataset. The resulting DMC mosaic was compared to a near-concurrent Satellite Pour l’Observation de la Terre (SPOT) 5 reflectance image of a portion of the same area, omitting the blue channel from the DMC mosaic due to its absence in the SPOT 5 data. The mean absolute reflectance difference was found to be 3.43% and the mean coefficient of determination (R²) over the bands was 0.84. The technique allows the production of seamless mosaics corrected for coarse scale atmospheric and BRDF effects and does not require the manual acquisition (or provision) of ground reflectance references. The accuracy of corrections is limited by the resolution of the reference image, which is generally significantly coarser than VHR imagery. The method cannot correct for small scale BRDF or other variations not captured at the reference resolution. Nevertheless, results show a significant improvement in homogeneity and correlation with SPOT 5 reflectance.     

Disaster Monitoring of Satellite Image Processing Using Progressive Image Classification

The analysis of remote sensing image areas is needed for climate detection and management, especially for monitoring flood disasters in critical environments and applications. Satellites are mostly used to detect disasters on Earth, and they have advantages in capturing Earth images. Using the control technique, Earth images can be used to obtain detailed terrain information. Since the acquisition of satellite and aerial imagery, this system has been able to detect floods, and with increasing convenience, flood detection has become more desirable in the last few years. In this paper, a Big Data Set-based Progressive Image Classification Algorithm (PICA) system is introduced to implement an image processing technique, detect disasters, and determine results with the help of the PICA, which allows disaster analysis to be extracted more effectively. The PICA is essential to overcoming strong shadows, for proper access to disaster characteristics to false positives by operators, and to false predictions that affect the impact of the disaster. The PICA creates tailoring and adjustments obtained from satellite images before training and post-disaster aerial image data patches. Two types of proposed PICA systems detect disasters faster and more accurately (95.6%).     

TrackerBots: Autonomous unmanned aerial vehicle for real‐time localization and tracking of multiple radio‐tagged animals

Autonomous aerial robots provide new possibilities to study the habitats and behaviors of endangered species through the efficient gathering of location information at temporal and spatial granularities not possible with traditional manual survey methods. We present a novel autonomous aerial vehicle system—TrackerBots—to track and localize multiple radio‐tagged animals. The simplicity of measuring the received signal strength indicator (RSSI) values of very high frequency (VHF) radio‐collars commonly used in the field is exploited to realize a low‐cost and lightweight tracking platform suitable for integration with unmanned aerial vehicles (UAVs). Due to uncertainty and the nonlinearity of the system based on RSSI measurements, our tracking and planning approaches integrate a particle filter for tracking and localizing and a partially observable Markov decision process for dynamic path planning. This approach allows autonomous navigation of a UAV in a direction of maximum information gain to locate multiple mobile animals and reduce exploration time and, consequently, conserve on‐board battery power. We also employ the concept of search termination criteria to maximize the number of located animals within power constraints of the aerial system. We validated our real‐time and online approach through both extensive simulations and field experiments with five VHF radio‐tags on a grassland plain.     

Robotic technologies for solar‐powered UAVs: Fully autonomous updraft‐aware aerial sensing for multiday search‐and‐rescue missions

Large‐scale aerial sensing missions can greatly benefit from the perpetual endurance capability provided by high‐performance low‐altitude solar‐powered unmanned aerial vehicles (UAVs). However, today these UAVs suffer from small payload capacity, low energetic margins, and high operational complexity. To tackle these problems, this paper presents four individual technical contributions and integrates them into an existing solar‐powered UAV system: First, a lightweight and power‐efficient day/night‐capable sensing system is discussed. Second, means to optimize the UAV platform to the specific payload and to thereby achieve sufficient energetic margins for day/night flight with payload are presented. Third, existing autonomous launch and landing functionality is extended for solar‐powered UAVs. Fourth, as a main contribution an extended Kalman filter (EKF)‐based autonomous thermal updraft tracking framework is developed. Its novelty is that it allows the end‐to‐end integration of the thermal‐induced roll moment into the estimation process. It is assessed against unscented Kalman filter and particle filter methods in simulation and implemented on the aircraft's low‐power autopilot. The complete system is verified during a 26 h search‐and‐rescue aerial sensing mock‐up mission that represents the first‐ever fully autonomous perpetual endurance flight of a small solar‐powered UAV with a day/night‐capable sensing payload. It also represents the first time that solar‐electric propulsion and autonomous thermal updraft tracking are combined in flight. In contrast to previous work that has focused on the energetic feasibility of perpetual flight, the individual technical contributions of this paper are considered core functionality to guarantee ease‐of‐use, effectivity, and reliability in future multiday aerial sensing operations with small solar‐powered UAVs.