Under canopy light detection and ranging‐based autonomous navigation

This paper describes a light detection and ranging (LiDAR)‐based autonomous navigation system for an ultralightweight ground robot in agricultural fields. The system is designed for reliable navigation under cluttered canopies using only a 2D Hokuyo UTM‐30LX LiDAR sensor as the single source for perception. Its purpose is to ensure that the robot can navigate through rows of crops without damaging the plants in narrow row‐based and high‐leaf‐cover semistructured crop plantations, such as corn (Zea mays) and sorghum ( Sorghum bicolor). The key contribution of our work is a LiDAR‐based navigation algorithm capable of rejecting outlying measurements in the point cloud due to plants in adjacent rows, low‐hanging leaf cover or weeds. The algorithm addresses this challenge using a set of heuristics that are designed to filter out outlying measurements in a computationally efficient manner, and linear least squares are applied to estimate within‐row distance using the filtered data. Moreover, a crucial step is the estimate validation, which is achieved through a heuristic that grades and validates the fitted row‐lines based on current and previous information. The proposed LiDAR‐based perception subsystem has been extensively tested in production/breeding corn and sorghum fields. In such variety of highly cluttered real field environments, the robot logged more than 6 km of autonomous run in straight rows. These results demonstrate highly promising advances to LiDAR‐based navigation in realistic field environments for small under‐canopy robots.     

Registration of three‐dimensional scanning LiDAR sensors: An evaluation of model‐based and model‐free methods

Registration, also know as extrinsic calibration, is the process of determining the position and orientation of a sensor relative to a known frame of reference. For ranging sensors such as light detection and ranging (LiDAR) used in field robotic applications, the quality of the registration determines the utility of the range measurements. This paper makes two contributions. The first is the introduction of a new method, termed maximum sum of evidence (MSoE) for registering three‐dimensional LiDAR sensors to moving platforms. This method is shown to produce more accurate registration solutions than two leading methods for these sensors, the adaptive structure registration filter (ASRF) and Rényi quadratic entropy (RQE). The second contribution of the paper is to study the accuracy of the MSoE registration against these two other approaches. One of these, like the MSoE, requires a truth model of the environment. The other, a model‐free method, seeks the registration that minimizes the RQE of a compound point cloud. The main finding of this investigation is that while the model‐based methods prove more accurate than the model‐free approach, the results of all three methods are fit for their intended field robotic applications. This leads us to conclude that registration based on RQE is preferable in many, if not all, field robotic applications for reasons of convenience, since a truth model of the environment is not required.     

Investigating RaDAR-LiDAR synergy in a North Carolina pine forest

A low frequency (80-120 MHz) VHF RaDAR, BioSAR, specifically designed for forest biomass estimation and a profiling LiDAR, PALS, were flown over loblolly pine plantations in the southeastern United States. LiDAR-only, RaDAR-only, and joint LiDAR-RaDAR linear models were developed to determine if returns from two sensors could be used to estimate pine biomass more accurately and precisely than returns from either sensor alone. The best five-variable RaDAR model explained 81.8% (R²) of the stem green biomass variability, with a regression RMSE of 57.5 t/ha. The best one-variable LiDAR model explained 93.3% of the biomass variation (RMSE = 33.9 t/ha). Combining the RaDAR normalized volumetric returns with the profiling LiDAR ranging measurements did little to improve the best LiDAR-only model. The best LiDAR-RaDAR model explained 93.8% of the biomass variation (RSME = 32.7 t/ha). Cross-validation and training/test validation procedures demonstrated (1) that all models are unbiased and (2) the increased precision of the LiDAR-only and LiDAR-RaDAR models. The results of this investigation and a companion study indicate that there is little to be gained combining VHF-RaDAR volumetric returns and profiling LiDAR ranging measurements in pine forests; a LiDAR ranging system is sufficient for accurate, precise biomass estimation.     

Multimodal fusion for autonomous navigation via deep reinforcement learning with sparse rewards and hindsight experience replay

The multimodal perception of intelligent robots is essential for achieving collision-free and efficient navigation. Autonomous navigation is enormously challenging when perception is acquired using only vision or LiDAR sensor data due to the lack of complementary information from different sensors. This paper proposes a simple yet efficient deep reinforcement learning (DRL) with sparse rewards and hindsight experience replay (HER) to achieve multimodal navigation. By adopting the depth images and pseudo-LiDAR data generated by an RGB-D camera as input, a multimodal fusion scheme is used to enhance the perception of the surrounding environment compared to using a single sensor. To alleviate the misleading way for the agent to navigate with dense rewards, the sparse rewards are intended to identify its tasks. Additionally, the HER technique is introduced to address the sparse reward navigation issue for accelerating optimal policy learning. The results show that the proposed model achieves state-of-the-art performance in terms of success, crash, and timeout rates, as well as generalization capability.     

Exploring LiDAR-RaDAR synergy—predicting aboveground biomass in a southwestern ponderosa pine forest using LiDAR, SAR and InSAR

Scanning Light Detecting and Ranging (LiDAR), Synthetic Aperture Radar (SAR) and Interferometric SAR (InSAR) were analyzed to determine (1) which of the three sensor systems most accurately predicted forest biomass, and (2) if LiDAR and SAR/InSAR data sets, jointly considered, produced more accurate, precise results relative to those same data sets considered separately. LiDAR ranging measurements, VHF-SAR cross-sectional returns, and X- and P-band cross-sectional returns and interferometric ranges were regressed with ground-estimated (from dbh) forest biomass in ponderosa pine forests in the southwestern United States. All models were cross-validated. Results indicated that the average canopy height measured by the scanning LiDAR produced the best predictive equation. The simple linear LiDAR equation explained 83% of the biomass variability (n = 52 plots) with a cross-validated root mean square error of 26.0 t/ha. Additional LiDAR metrics were not significant to the model. The GeoSAR P-band (λ = 86 cm) cross-sectional return and the GeoSAR/InSAR canopy height (X-P) captured 30% of the forest biomass variation with an average predictive error of 52.5 t/ha. A second RaDAR-FOPEN collected VHF (λ ∼ 7.8 m) and cross-polarized P-band (λ = 88 cm) cross-sectional returns, none of which proved useful for forest biomass estimation (cross-validated R² = 0.09, RMSE = 63.7 t/ha). Joint consideration of LiDAR and RaDAR measurements produced a statistically significant, albeit small improvement in biomass estimation precision. The cross-validated R² increased from 83% to 84% and the prediction error decreased from 26.0 t/ha to 24.9 t/ha when the GeoSAR X-P interferometric height is considered along with the average LiDAR canopy height. Inclusion of a third LiDAR metric, the 60th decile height, further increased the R² to 85% and decreased the RMSE to 24.1 t/ha. On this 11 km² ponderosa pine study area, LiDAR data proved most useful for predicting forest biomass. RaDAR ranging measurements did not improve the LiDAR estimates.     

An evidence‐based approach to object pose estimation from LiDAR measurements in challenging environments

This paper addresses the problem of estimating object pose from high‐density LiDAR measurements in unpredictable field robotic environments. Point‐cloud measurements collected in such environments do not lend themselves to providing an initial estimate or systematic segmentation of the point‐cloud. A novel approach is presented that evaluates measurements individually for the evidence they provide to a collection of pose hypotheses. A maximum evidence strategy is constructed that is based in the idea that the most likely pose must be that which is most consistent with the observed LiDAR range measurements. This evidence‐based approach is shown to handle the diversity of range measurements without an initial estimate or segmentation. The method is robust to dust. The approach is demonstrated by two pose estimation problems associated with the automation of a large mining excavator.     

Cataglyphis: An autonomous sample return rover

This paper presents the design of Cataglyphis, a research rover that won the NASA Sample Return Robot Centennial Challenge in 2015. During the challenge, Cataglyphis was the only robot that was able to autonomously find, retrieve, and return multiple types of samples in a large natural environment without using Earth‐specific sensors such as GPS and magnetic compasses. It navigates through a fusion of measurements collected from inertial sensors, wheel encoders, a nodding Lidar, a set of ranging radios, a camera on a panning platform, and a sun sensor. In addition to visual detection of a homing beacon, computer vision algorithms provide the sample detection, identification, and localization capabilities, with low false positive and false negative rates demonstrated during the competition. The mission planning and control software enables robot behaviors, determines sequences of actions, and helps the robot to recover from various failure conditions. A compliant, under‐actuated manipulator conforms to the natural terrain before picking up samples of various size, weight, and shape.     

Track Compensation Algorithm Using Free Space Information with Occupancy Grid Map

Over the past few years, numerous technologies have emerged to enable safe and convenient driving. However, there still exist various problems autonomous vehicles should overcome. Precise detection and perception of surrounding environments are the essential foundations to overcome them. Consequently, many sensor fusion algorithms have been developed to handle more complex situations, with sensor manufacturers also making strenuous efforts to enhance sensor performance. Although Light Detection And Ranging(LiDAR) sensor generally outperforms other sensor types, they remain prohibitively expensive from car manufacturing companies perspective. Therefore, camera and radar sensors have been enhanced, and are starting to provide free space information, similar to LiDAR sensor data and somewhat different from target information they have previously provided. The aim of this paper was to utilize the free space information to improve track information for vehicles. We employ the probability model with two occupancy grid map (OGM) types, which are Bayesian theory and Dempster-Shafer theory based OGMs, to classify free space information states and to efficiently handle free space information. Final output from the proposed algorithm is the target vehicle’s compensated track. Experimental results verify superior performance compared with non-compensated algorithms.

     

SaccadeFork: A lightweight multi-sensor fusion-based target detector

Commercialization of self-driving applications requires precision and reliability of the perception system due to the highly dynamic and complex road environment. Early perception systems either rely on the camera or on LiDAR for moving obstacle detection. With the development of vehicular sensors and deep learning technologies, the multi-view and sensor fusion based convolutional neural network (CNN) model for detection tasks has become a popular research area. In this paper, we present a novel multi-sensor fusion-based CNN model–SaccadeFork–that integrates the image and upsampled LiDAR point clouds as the input. SaccadeFork includes two modules: (1) a lightweight backbone that consists of hourglass convolution feature extraction module and a parallel dilation convolution module for adaptation of the system to different target sizes; (2) an anchor-based detection head. The model also considers deployment of resource-limited edge devices in the vehicle. Two refinement strategies, i.e., Mixup and Swish activation function are also adopted to improve the model. Comparison with a series of latest models on public dataset of KITTI shows that SaccadeFork can achieve the optimal detection accuracy on vehicles and pedestrians under different scenarios. The final model is also deployed and tested on a local dataset collected based on edge devices and low-cost sensor solutions, and the results show that the model can achieve real-time efficiency and high detection accuracy.     

Assessing forest structural and physiological information content of multi-spectral LiDAR waveforms by radiative transfer modelling

The concept for a multi-spectral, full-waveform canopy LiDAR instrument was tested by simulating return waveforms using a model providing ecological sound tree structure (TREEGROW) and a model of leaf optical properties (PROSPECT). The proposed instrument will take measurements at four different wavelengths, which were chosen according to physiological processes altering leaf reflectance and transmittance. The modelling was used to assess both the structural and physiological information content such an instrument could provide, especially whether the normally structure-dominated return waveform would pick up small changes in reflectance at the leaf level. Multi-spectral waveforms were simulated for models of single Scots pine trees of different ages and at different stages of the growing season, including chlorophyll concentration induced changes in leaf optical properties. It was shown that the LiDAR waveforms would not only capture the tree height information, but would also pick up the seasonal and vertical variation of NDVI computed from two of the four MSCL wavelengths inside the tree canopy. The instrument concept was further tested in a simulation of a virtual forest stand constructed of 74 trees of different ages according to measurements taken on a field site being 20 by 20 meter in size. A total of 1521 NDVI profiles were computed and mean NDVI corrected backscatter was compared to the actual canopy profile of the virtual stand. The profiles picked up the seasonal variation of chlorophyll within the canopy, while the return of ground remained unchanged from June to September. Thus, it was shown that a MSCL instrument would be able to separately pick up the physiology of canopy and understorey and/or soil. It was found that occlusion would mask the lower parts of the canopy volume within the stand and the seasonal variation of this occlusion effect was quantified, being larger in September, when the absorption of canopy elements is higher. In addition, it could be demonstrated that a new multi-wavelength LiDAR predictor variable was able to significantly improve the retrieval accuracy of photosynthetically active biomass opposed to using a single-wavelength LiDAR alone.     

A Computational Geometry Approach for Localization and Tracking in GPS‐denied Environments*

Localization and tracking of vehicles is still an important issue in GPS‐denied environments (both indoors and outdoors), where accurate motion is required. In this work, a localization system based on the random disposition of LiDAR sensors (which share a partially common field of view) and on the use of the Hausdorff distance is addressed. The proposed system uses the Hausdorff distance to estimate both the position of the LiDAR sensors and the pose of the vehicle as it drives within the environment. Our approach is not restricted to the number of LiDAR sensors (the estimation procedure is asynchronous), the number of vehicles (it is a multidimensional approach), or the nature of the environment. However, it is implemented in open spaces, limited by the range of the LiDAR sensors and the geometry of the vehicle. An empirical analysis of the presented approach is also included here, showing that the error in the localization estimation remains bounded in approximately 50 cm. Real‐time experimentation as validation of the proposed localization and tracking techniques as well as the pros and cons of our proposal are also shown in this work.     

A multistage database of field measurements and synoptic remotely sensed data to support model validation and testing in Earth observation

This paper presents a novel database of ground and remotely sensed data from the United Kingdom, which is uniquely suited to scaling-up multispectral measurements from a single plot to the scale of satellite sensor observations. Multiple aircraft and satellite sensors were involved, and most of the data were acquired on a single day in June 2006, providing a synoptic view which, at its largest extent, covered most of southern England and Wales. Three airborne imaging spectrometers were involved (Specim AISA Eagle, Itres CASI-2 and -3) and three satellite sensors (UK-DMC, PROBA/CHRIS, and SPOT HRG), complemented with airborne LiDAR, multispectral survey cameras, and ground measurements (land cover, LAI, reflectance factors, and atmospheric measurements). In this paper the NCAVEO Field Campaign (NFC) database is described and an example of its use to produce a high spatial resolution leaf area index map for the validation of medium-resolution products (MODIS, VEGETATION, and MERIS) is presented.     

Development of a simulation model to predict LiDAR interception in forested environments

Airborne scanning LiDAR systems are used to predict a range of forest attributes. However, the accuracy with which this can be achieved is highly dependent on the sensor configuration and the structural characteristics of the forest examined. As a result, there is a need to understand laser light interactions with forest canopies so that LiDAR sensor configurations can be optimised to assess particular forest types. Such optimisation will not only ensure the targeted forest attributes can be accurately and consistently quantified, but may also minimise the cost of data acquisition and indicate when a survey configuration will not deliver information needs.In this paper, we detail the development and application of a model to simulate laser interactions within forested environments. The developed model, known as the LiDAR Interception and Tree Environment (LITE) model, utilises a range of structural configurations to simulate trees with variable heights, crown dimensions and foliage clumping. We developed and validated the LITE model using field data obtained from three forested sites covering a range of structural classes. Model simulations were then compared to coincident airborne LiDAR data collected over the same sites. Results indicate that the LITE model can be used to produce comparable estimates of maximum height of trees within plots (differences < 2.42 m), mean heights of first return data (differences < 2.27 m), and canopy height percentiles (r² = 0.94, p < 0.001) when compared to airborne LiDAR data. In addition, the distribution of airborne LiDAR hits through the canopy profile was closely matched by model predictions across the range of sites. Importantly, this demonstrates that the structural differences between forest stands can be characterised by LITE. Models that are capable of interpreting the response of small-footprint LiDAR waveforms can facilitate algorithm development, the generation of corrections for actual LiDAR data, and the optimisation of sensor configurations for differing forest types, benefiting a range of experimental and commercial LiDAR applications. As a result, we also performed a scenario analysis to demonstrate how differences in forest structure, terrain, and sensor configuration can influence the interception of LiDAR beams.     

High‐ambient‐contrast augmented reality with a tunable transmittance liquid crystal film and a functional reflective polarizer

We have proposed a compact, yet high ambient contrast ratio augmented reality (AR) system by incorporating a tunable transmittance liquid crystal (LC) cell and a thin functional reflective polarizer. The broadband polarization‐independent guest–host LC cell can change the transmittance from ~73% to ~26% with merely 8 V. Its response time (~50 ms) is at least 10× faster than that of photochromic materials used in commercial transition glasses. Combining the LC cell with a light sensor, the tunable transmittance LC cell can efficiently improve the ambient contrast ratio of the AR system under different lighting conditions. Meanwhile, the functional reflective polarizer works similarly to a polarizing beam splitter, except that it is much more compact and lighter weight. With some modification, we also designed a functional reflective polarizer to help people with color vision deficiency.     

A complex study of Etna's volcanic plume from ground‐based,in situ and space‐borne observations

Two periods of transboundary transport of volcanic aerosols and debris following recent eruptions of Mount Etna, Italy, were examined using ground‐based and satellite spectrophotometric measurements together with Light Detection And Ranging (LiDAR) and aerosol filter observations in Athens and Thessaloniki, Greece. Independent columnar SO₂ measurements from ground and space identified peaks at Greek sites after the volcanic eruptions. LiDAR measurements of the aerosol extinction at Thessaloniki and Athens performed in July 2001 have shown the height of the volcanic plume to be about 3.5 km asl and the optical thickness of the dust layer to be of the order of 3×10^?3 at 532 nm. Strong ozone depletion observed at the volcano plume level by using ozonesonde ascents may be attributed to the in‐plume processes that generate reactive halogens, which in turn destroy ozone. The chemical and elemental composition of aerosol samples, taken at the Earth's surface, was analysed and confirmed the volcanic origin of the dust.     

Assessment of forest structure with airborne LiDAR and the effects of platform altitude

Airborne scanning LiDAR is a spatial technology increasingly used for forestry and environmental applications. However, the accuracy and coverage of LiDAR observations is highly dependent on both the extrinsic specifications of the LiDAR survey as well as the intrinsic effects such as the underlying forest structure. Extrinsic parameters which are set as part of the LiDAR survey include platform altitude, scan angle (half max. angle off nadir), and beam cross sectional diameter at the reflecting surface (referred to as footprint size). In this paper we investigate the effect of a number of these extrinsic parameters, including three different platform altitudes (1000, 2000, and 3000 m), two scan angles at 1000 m (10° and 15° half max. angle off nadir), and three footprint sizes (0.2, 0.4, and 0.6 m). The comparison was undertaken in eucalypt forests at three sites, varying in vegetation structure and topography within the Wedding Bells State Forest, Coffs Harbour, Australia. Results at the plot scale (40 × 90 m areas) indicate that tree heights computed from the 1000 m LiDAR data set (10° half max. angle off nadir) are well correlated with maximum plot heights (difference < 3 m) and field measured canopy volume (r² > 0.75, p < 0.001). Using normalised canopy height profiles (CHP) derived for sites, from data recorded at each altitude, we observed no significant difference between the relative distribution of LiDAR returns, indicating that platform altitude and footprint size have not had a major influence on CHP estimation. Interestingly, comparisons of first and last returns for individual pulses at increasing altitudes identified progressively fewer discrete first/last pulse combinations with more than 70% of pulses recorded as a single return at the highest altitude (3000 m). A possible hypothesis is that greater platform altitude and footprint size reduces the intensity of laser beam incident on a given surface area thus decreasing the probability of recording a last return above the noise threshold. Furthermore, tree scale analysis found a positive relationship between platform altitude and the underestimation of crown area and crown volume. The implications of this work for forest management are: (i) platform altitudes as high as 3000 m can be used to quantify the vertical distribution of phyto-elements, (ii) higher platform altitudes record a lower proportion of first/last return combinations that will further reduce the number of points available for forest structural assessment and development of digital elevation models, and (iii) for discrete LiDAR data, increasing platform altitude will record a lower frequency of returns per crown, resulting in larger underestimates of individual tree crown area and volume if standard algorithms are applied.     

基于双多线激光雷达的非结构化环境负障碍感知技术

负障碍感知是非结构化环境下的难点问题,本文针对该问题提出一种新的基于双多线激光雷达（Light detection and ranging,LiDAR）的感知方法.采用分布嵌入式架构对双激光雷达数据进行同步采集与实时处理,将雷达点云映射到多尺度栅格,统计栅格的点云密度与相对高度等特征并标记,从点云数据提取负障碍几何特征,通过将栅格的统计特征与负障碍的几何特征做多特征关联找到关键特征点对,将特征点对聚类并过滤,识别出负障碍.方法不受地面平整度影响,已成功应用在无人驾驶车上.使用表明该方法具有较高的实时性和可靠性,在非结构化环境下具有良好的感知效果.     

室内环境下UWB与LiDAR融合定位算法研究

当前全球导航卫星系统与激光雷达的数据融合被广泛应用于无人驾驶车辆的定位系统中,但在室内环境下由于卫星信号的丢失导致定位精度低甚至无法定位。为此提出一种基于超宽带（Ultra-Wideband,UWB）与激光雷达（Light Detection and Ranging,LiDAR）的融合定位算法。该算法以粒子滤波为基础,对两个传感器的定位数据进行互补融合解算。利用UWB实时定位数据通过提供起始粒子范围的方式来提高LiDAR的定位速率。通过求解LiDAR定位信息与粒子之间的几何距离来更新粒子的权重,从而弥补UWB的非视距误差。搭建一个室内测试场景,并将融合定位算法在智能小车平台上进行验证。实验结果表明,该方法优于UWB或LiDAR单一传感器的定位方案,而且在UWB视距受阻或LiDAR匹配失效的情况下,车辆仍能够获得良好的定位精度和定位实时性。     

Multispectral and LiDAR data fusion for fuel type mapping using Support Vector Machine and decision rules

This paper presents a method for mapping fuel types using LiDAR and multispectral data. A two-phase classification method is proposed to discriminate the fuel classes of the Prometheus classification system, which is adapted to the ecological characteristics of the European Mediterranean basin. The first step mapped the main fuel groups, namely grass, shrub and tree, as well as non-fuel classes. This phase was carried out using a Support Vector Machine (SVM) classification combining LiDAR and multispectral data. The overall accuracy of this classification was 92.8% with a kappa coefficient of 0.9. The second phase of the proposed method focused on discriminating additional fuel categories based on vertical information provided by the LiDAR measurements. Decision rules were applied to the output of the SVM classification based on the mean height of LiDAR returns and the vertical distribution of fuels, described by the relative LiDAR point density in different height intervals. The final fuel type classification yielded an overall accuracy of 88.24% with a kappa coefficient of 0.86. Some confusion was observed between fuel types 7 (dense tree cover presenting vertical continuity with understory vegetation) and 5 (trees with less than 30% of shrub cover) in some areas covered by Holm oak, which showed low LiDAR pulses penetration so that the understory vegetation was not correctly sampled.     

Outdoor field performances of insect‐based visual motion sensors

Considerable attention has been paid during the past decade to navigation systems based on the use of visual optic flow cues. Optic flow‐‐based visuomotor control systems have been implemented on an increasingly large number of sighted autonomous robots designed to travel under specific lighting conditions. Many algorithms based on conventional cameras or custom‐made sensors are being used nowadays to process visual motion. In this paper, we focus on the reliability of our optical sensors, which can be used to measure the local one‐dimensional angular speed of robots flying outdoors over a visual scene in terms of their accuracy, range, refresh rate, and sensitivity to illuminance variations. We have designed, constructed, and characterized two miniature custom‐made visual motion sensors: (i) the APIS (adaptive pixels for insect‐based sensors)‐based local motion sensor involving the use of an array custom‐made in Very‐Large‐Scale Integration (VLSI) technology, which is equipped with Delbrück‐type autoadaptive pixels, and (ii) the LSC‐based (LSC is a component purchased from iC‐Haus) local motion sensor involving the use of off‐the‐shelf linearly amplified photosensors, which is equipped with an onchip preamplification circuit. By combining these photodetectors with a low‐cost optical assembly and a bioinspired visual processing algorithm, highly effective miniature sensors were obtained for measuring the visual angular speed in field experiments. The present study focused on the static characteristics and the dynamic responses of these local motion sensors over a wide range of illuminance values, ranging from 50 to 10,000 lux both indoors and outdoors. Although outdoor experiments are of great interest to equip micro‐air vehicles with visual motion sensors, we also performed indoor experiments as a comparison. The LSC‐based visual motion sensor was found to be more accurate in a narrow, 1.5‐decade illuminance range, whereas the APIS‐based visual motion sensor was more robust to illuminance changes in a larger, 3‐decade range. The method presented in this study provides a new benchmark test for thoroughly characterizing visual motion and optic flow sensors designed to operate outdoors under various lighting conditions, in unknown environments where future micro‐aerial vehicles will be able to navigate safely. © 2011 Wiley Periodicals, Inc.