Localization using GPS and VISION aided INS with an image database and a network of a ground-based reference station in outdoor environments

GPS/INS integrated systems do not guarantee robustness and accuracy of localization, because GPS has vulnerability to external disturbances. However, the overall performance and reliability of the system can be significantly improved by fusing multiple sensors with a different operating principle. In outdoor environments where GPS may be blocked, there are many features compared to the open space and these features can provide much information for UGV localization. Thus, this paper proposes an improved localization algorithm based on the hierarchical federation of three measurement layers, i.e., GPS, INS, and visual localization, to overcome the shortcomings of GPS/INS integrated systems. The proposed algorithm automatically switches the operation modes according to GPS status and a network of a ground-based reference station. A vocabulary tree with SURF is used in the visual localization method. In the data fusion of visual localization and INS, an asynchronous and time-delayed data fusion algorithm is presented because visual localization is always time-delayed compared with INS. By using DGPS to obtain the reference position under the dynamic conditions of the reference station, the restrictions of the conventional DGPS are overcome and all UGVs within WiBro communication range of the reference station can accurately estimate the position with a common GPS. The experiment results with a predefined path demonstrate enhancement of the robustness and accuracy of localization in outdoor environments.     

Temporal Range Registration for Unmanned Ground and Aerial Vehicles

An iterative temporal registration algorithm is presented in this article for registering 3D range images obtained from unmanned ground and aerial vehicles traversing unstructured environments. We are primarily motivated by the development of 3D registration algorithms to overcome both the unavailability and unreliability of Global Positioning System (GPS) within required accuracy bounds for Unmanned Ground Vehicle (UGV) navigation. After suitable modifications to the well-known Iterative Closest Point (ICP) algorithm, the modified algorithm is shown to be robust to outliers and false matches during the registration of successive range images obtained from a scanning LAser Detection And Ranging (LADAR) rangefinder on the UGV. Towards registering LADAR images from the UGV with those from an Unmanned Aerial Vehicle (UAV) that flies over the terrain being traversed, we then propose a hybrid registration approach. In this approach to air to ground registration to estimate and update the position of the UGV, we register range data from two LADARs by combining a feature-based method with the aforementioned modified ICP algorithm. Registration of range data guarantees an estimate of the vehicle's position even when only one of the vehicles has GPS information. Temporal range registration enables position information to be continually maintained even when both vehicles can no longer maintain GPS contact. We present results of the registration algorithm in rugged terrain and urban environments using real field data acquired from two different LADARs on the UGV. ★Commercial equipment and materials are identified in this article in order to adequately specify certain procedures. Such identification does not imply recommendation or endorsement by the National Institute of Standards and Technology, nor does it imply that the materials or equipment identified are necessarily the best available for the purpose.     

Localization of an unmanned ground vehicle based on hybrid 3D registration of 360 degree range data and DSM

A computer vision technique to identify the location of an outdoor unmanned ground vehicle (UGV) is presented. The proposed technique is based on hybrid 3D registration of 360 degree laser range data to a digital surface model (DSM). Range frames obtained from 48 laser detectors are aligned with the reference coordinate system of the DSM. Three novel approaches are proposed for accurate and fast 3D registration of range data and the DSM. First, a two-step hybrid 3D registration technique is proposed. A pair-wise registration step of two consecutive range frames is followed by a refinement step using a layered DSM. Second, a fast projection-based pair-wise registration is proposed by employing rasterized 360 degree range frames. Third, a high elevation DSM is divided into several elevation layers and correspondence search is done near the vehicle’s current elevation. This reduces the number of matching outliers and facilitates fast localization. Experimental results show that the proposed approaches yield better performance in 3D localization compared to conventional 3D registration techniques. Error analysis on five outdoor paths is presented with respect to ground truth.     

基于ARToolKit的地下管网增强现实系统研究

增强现实是把计算机生成的虚拟图像或其它信息叠加到用户所看到的真实世界中的一种技术,是虚拟现实领域的一个难点热点。ARToolKit为增强现实技术的应用提供了一种方便快捷的开发工具,但其不足之处在于它是基于计算机视觉的注册方式,不适合户外增强现实系统的开发,需要做进一步的改进。论文利用ARToolKit在室内实现了管网三维可视化的增强显示,取得了满意的效果;根据户外的实际,提出了以全球定位系统载波相位差分(GPSRTK)和惯性导航系统(INS)相结合进行精确定位、以电子罗盘和倾角计相结合测定视线方向的混合注册方法;对利用ARToolKit建立户外地下管网增强现实系统的框架及功能作了进一步的探讨。     

Robust Navigation and Seamless Localization for Carlike Robots in Indoor‐outdoor Environments

This paper describes a navigation and seamless localization system that permits carlike robots to move safely in heterogeneous scenarios within indoor and outdoor environments. The robot localization integrates different sensor (GPS, odometry, laser rangefinders) information depending on the kind of area (indoors, outdoors, and areas between) or on the sensor uncertainty in such a way that there are no discontinuities in the localization, and a bounded uncertainty is constantly maintained. Transitions through indoor and outdoor environments are thoroughly considered to assure a smooth change in‐between. The paper addresses a navigation technique that combines two well‐known obstacle avoidance techniques, namely the nearness diagram and the dynamic window approaches, exploiting the advantages and properties of both, and integrating the seamless localization technique. The navigation technique is developed for carlike robots by considering their shape and kinodynamic constraints, and the restrictions imposed by the environment. Forward‐backward maneuvers are also integrated in the method, allowing difficult situations in dense scenarios to be managed. The whole system has been tested in simulations and experiments in real large‐scale scenarios.     

Benchmarking urban six‐degree‐of‐freedom simultaneous localization and mapping

Quite a number of approaches for solving the simultaneous localization and mapping (SLAM) problem exist by now. Some of them have recently been extended to mapping environments with six‐degree‐of‐freedom poses, yielding 6D SLAM approaches. To demonstrate the capabilities of the respective algorithms, it is common practice to present generated maps and successful loop closings in large outdoor environments. Unfortunately, it is nontrivial to compare different 6D SLAM approaches objectively, because ground truth data about the outdoor environments used for demonstration are typically unavailable. We present a novel benchmarking method for generating the ground truth data based on reference maps. The method is then demonstrated by comparing the absolute performance of some previously existing 6D SLAM algorithms that build a large urban outdoor map. © 2008 Wiley Periodicals, Inc.     

基于cubature Kalman filter的INS/GPS组合导航滤波算法

INS/GPS组合导航系统的本质是非线性的,为改善非线性下INS/GPS组合导航精度,提出将一种新的非线性滤波cubature Kalman filter（CKF）应用于INS/GPS组合导航中.为此,建立了基于平台失准角的非线性状态模型和以速度误差及位置误差描述的观测模型,分析了CKF滤波原理,设计了INS/GPS组合滤波器,对组合导航非线性模型进行了仿真.仿真结果显示,相对于扩展卡尔曼滤波（EKF）,CKF降低了姿态、位置和速度估计误差,CKF更适合于处理组合导航的状态估计问题.     

Localizing Ground Penetrating RADAR: A Step Toward Robust Autonomous Ground Vehicle Localization

Autonomous ground vehicles navigating on road networks require robust and accurate localization over long‐term operation and in a wide range of adverse weather and environmental conditions. GPS/INS (inertial navigation system) solutions, which are insufficient alone to maintain a vehicle within a lane, can fail because of significant radio frequency noise or jamming, tall buildings, trees, and other blockage or multipath scenarios. LIDAR and camera map‐based vehicle localization can fail when optical features become obscured, such as with snow or dust, or with changes to gravel or dirt road surfaces. Localizing ground penetrating radar (LGPR) is a new mode of a priori map‐based vehicle localization designed to complement existing approaches with a low sensitivity to failure modes of LIDAR, camera, and GPS/INS sensors due to its low‐frequency RF energy, which couples deep into the ground. Most subsurface features detected are inherently stable over time. Significant research, discussed herein, remains to prove general utility. We have developed a novel low‐profile ultra‐low power LGPR system and demonstrated real‐time operation underneath a passenger vehicle. A correlation maximizing optimization technique was developed to allow real‐time localization at 126 Hz. Here we present the detailed design and results from highway testing, which uses a simple heuristic for fusing LGPR estimates with a GPS/INS system. Cross‐track localization accuracies of 4.3 cm RMS relative to a “truth” RTK GPS/INS unit at speeds up to 100 km/h (60 mph) are demonstrated. These results, if generalizable, introduce a widely scalable real‐time localization method with cross‐track accuracy as good as or better than current localization methods.     

GPS‐denied Indoor and Outdoor Monocular Vision Aided Navigation and Control of Unmanned Aircraft

GPS‐denied closed‐loop autonomous control of unstable Unmanned Aerial Vehicles (UAVs) such as rotorcraft using information from a monocular camera has been an open problem. Most proposed Vision aided Inertial Navigation Systems (V‐INSs) have been too computationally intensive or do not have sufficient integrity for closed‐loop flight. We provide an affirmative answer to the question of whether V‐INSs can be used to sustain prolonged real‐world GPS‐denied flight by presenting a V‐INS that is validated through autonomous flight‐tests over prolonged closed‐loop dynamic operation in both indoor and outdoor GPS‐denied environments with two rotorcraft unmanned aircraft systems (UASs). The architecture efficiently combines visual feature information from a monocular camera with measurements from inertial sensors. Inertial measurements are used to predict frame‐to‐frame transition of online selected feature locations, and the difference between predicted and observed feature locations is used to bind in real‐time the inertial measurement unit drift, estimate its bias, and account for initial misalignment errors. A novel algorithm to manage a library of features online is presented that can add or remove features based on a measure of relative confidence in each feature location. The resulting V‐INS is sufficiently efficient and reliable to enable real‐time implementation on resource‐constrained aerial vehicles. The presented algorithms are validated on multiple platforms in real‐world conditions: through a 16‐min flight test, including an autonomous landing, of a 66 kg rotorcraft UAV operating in an unconctrolled outdoor environment without using GPS and through a Micro‐UAV operating in a cluttered, unmapped, and gusty indoor environment. © 2013 Wiley Periodicals, Inc.     

Continuous‐Time Three‐Dimensional Mapping for Micro Aerial Vehicles with a Passively Actuated Rotating Laser Scanner

The ability to generate accurate and detailed three‐dimensional (3D) maps of a scene from a mobile platform is an essential technology for a wide variety of applications from robotic navigation to geological surveying. In many instances, the best vantage point is from above, and as a result, there is a growing demand for low‐altitude mapping solutions from micro aerial vehicles such as small quadcopters. Existing lidar‐based 3D airborne mapping solutions rely on GPS/INS solutions for positioning, or focus on producing relatively low‐fidelity or locally focused maps for the purposes of autonomous navigation. We have developed a general‐purpose airborne 3D mapping system capable of continuously scanning the environment during flight to produce accurate and dense point clouds without the need for a separate positioning system. A key feature of the system is a novel passively driven mechanism to rotate a lightweight 2D laser scanner using the rotor downdraft from a quadcopter. The data generated from the spinning laser is input into a continuous‐time simultaneous localization and mapping (SLAM) solution to produce an accurate 6 degree‐of‐freedom trajectory estimate and a 3D point cloud map. Extensive results are presented illustrating the versatility of the platform in a variety of environments including forests, caves, mines, heritage sites, and industrial facilities. Comparison with conventional surveying methods and equipment demonstrates the high accuracy and precision of the proposed solution.     

Robust outdoor visual localization using a three‐dimensional‐edge map

Visual localization systems that are practical for autonomous vehicles in outdoor industrial applications must perform reliably in a wide range of conditions. Changing outdoor conditions cause difficulty by drastically altering the information available in the camera images. To confront the problem, we have developed a visual localization system that uses a surveyed three‐dimensional (3D)‐edge map of permanent structures in the environment. The map has the invariant properties necessary to achieve long‐term robust operation. Previous 3D‐edge map localization systems usually maintain a single pose hypothesis, making it difficult to initialize without an accurate prior pose estimate and also making them susceptible to misalignment with unmapped edges detected in the camera image. A multihypothesis particle filter is employed here to perform the initialization procedure with significant uncertainty in the vehicle's initial pose. A novel observation function for the particle filter is developed and evaluated against two existing functions. The new function is shown to further improve the abilities of the particle filter to converge given a very coarse estimate of the vehicle's initial pose. An intelligent exposure control algorithm is also developed that improves the quality of the pertinent information in the image. Results gathered over an entire sunny day and also during rainy weather illustrate that the localization system can operate in a wide range of outdoor conditions. The conclusion is that an invariant map, a robust multihypothesis localization algorithm, and an intelligent exposure control algorithm all combine to enable reliable visual localization through challenging outdoor conditions. © 2009 Wiley Periodicals, Inc.     

Planar Segment Based Three‐dimensional Point Cloud Registration in Outdoor Environments

We present an odometry‐free three‐dimensional (3D) point cloud registration strategy for outdoor environments based on area attributed planar patches. The approach is split into three steps. The first step is to segment each point cloud into planar segments, utilizing a cached‐octree region growing algorithm, which does not require the 2.5D image‐like structure of organized point clouds. The second step is to calculate the area of each segment based on small local faces inspired by the idea of surface integrals. The third step is to find segment correspondences between overlapping point clouds using a search algorithm, and compute the transformation from determined correspondences. The transformation is searched globally so as to maximize a spherical correlation‐like metric by enumerating solutions derived from potential segment correspondences. The novelty of this step is that only the area and plane parameters of each segment are employed, and no prior pose estimation from other sensors is required. Four datasets have been used to evaluate the proposed approach, three of which are publicly available and one that stems from our custom‐built platform. Based on these datasets, the following evaluations have been done: segmentation speed benchmarking, segment area calculation accuracy and speed benchmarking, processing data acquired by scanners with different fields of view, comparison with the iterative closest point algorithm, robustness with respect to occlusions and partial observations, and registration accuracy compared to ground truth. Experimental results confirm that the approach offers an alternative to state‐of‐the‐art algorithms in plane‐rich environments.     

Three‐dimensional localization for the MagneBike inspection robot

The MagneBike inspection robot is a climbing robot equipped with magnetic wheels. The robot is designed to drive on three‐dimensional (3D) complexly shaped pipe structures; therefore it is necessary to provide 3D visualization tools for the user, who remotely controls the robot out of sight. The localization system is required to provide a 3D map of the unknown environment and the 3D location of the robot in the environment's map. The localization strategy proposed in this paper consists of combining 3D odometry with 3D scan registration. The odometry model is based on wheel encoders and a three‐axis accelerometer. Odometry enables the tracking of the robot trajectory between consecutive 3D scans and is used as a prior for the scan matching algorithm. The 3D scan registration facilitates the construction of a 3D map of the environment and refines the robot position computed with odometry. This paper describes in detail the implementation of the localization concept. It presents the lightweight, small‐sized 3D range finder that has been developed for the MagneBike. It also proposes an innovative 3D odometry model that estimates the local surface curvature to compensate for the absence of angular velocity inputs. The different tools are characterized in detail based on laboratory and field experiments. They show that the localization concepts reliably track the robot moving in the specific application environment. We also describe various techniques to optimize the 3D scanning process, which is time consuming, and to compensate for the identified limitations. These techniques are useful inputs for the future automatization of the robot's control and optimization of its localization process. © 2010 Wiley Periodicals, Inc.     

On Feature Matching and Image Registration for Two‐dimensional Forward‐scan Sonar Imaging

The computer processing of forward‐look sonar video imagery enables significant capabilities in a wide variety of underwater operations within turbid environments. Accurate automated registration of sonar video images to complement measurements from traditional positioning devices can be instrumental in the detection, localization, and tracking of distinct scene targets, building feature maps, change detection, as well as improving precision in the positioning of unmanned submarines. This work offers a novel solution for the registration of two‐dimensional (2‐D) forward‐look sonar images recorded from a mobile platform, by optimization over the sonar 3‐D motion parameters. It incorporates the detection of key features and landmarks, and effectively represents them with Gaussian maps. Improved performance is demonstrated with respect to the state‐of‐the‐art approach utilizing 2‐D similarity transformation, based on experiments with real data.     

Bag‐of‐words‐driven,single‐camera simultaneous localization and mapping

This paper describes BoWSLAM, a scheme for a robot to reliably navigate and map previously unknown environments, in real time, using monocular vision alone. BoWSLAM can navigate challenging dynamic and self‐similar environments and can recover from gross errors. Key innovations allowing this include new uses for the bag‐of‐words image representation; this is used to select the best set of frames from which to reconstruct positions and to give efficient wide‐baseline correspondences between many pairs of frames, providing multiple position hypotheses. A graph‐based representation of these position hypotheses enables the modeling and optimization of errors in scale in a dual graph and the selection of only reliable position estimates in the presence of gross outliers. BoWSLAM is demonstrated mapping a 25‐min, 2.5‐km trajectory through a challenging and dynamic outdoor environment without any other sensor input, considerably farther than previous single‐camera simultaneous localization and mapping (SLAM) schemes. © 2010 Wiley Periodicals, Inc.     

RANGE–Robust autonomous navigation in GPS‐denied environments

This paper addresses the problem of autonomous navigation of a micro air vehicle (MAV) in GPS‐denied environments. We present experimental validation and analysis for our system that enables a quadrotor helicopter, equipped with a laser range finder sensor, to autonomously explore and map unstructured and unknown environments. The key challenge for enabling GPS‐denied flight of a MAV is that the system must be able to estimate its position and velocity by sensing unknown environmental structure with sufficient accuracy and low enough latency to stably control the vehicle. Our solution overcomes this challenge in the face of MAV payload limitations imposed on sensing, computational, and communication resources. We first analyze the requirements to achieve fully autonomous quadrotor helicopter flight in GPS‐denied areas, highlighting the differences between ground and air robots that make it difficult to use algorithms developed for ground robots. We report on experiments that validate our solutions to key challenges, namely a multilevel sensing and control hierarchy that incorporates a high‐speed laser scan‐matching algorithm, data fusion filter, high‐level simultaneous localization and mapping, and a goal‐directed exploration module. These experiments illustrate the quadrotor helicopter's ability to accurately and autonomously navigate in a number of large‐scale unknown environments, both indoors and in the urban canyon. The system was further validated in the field by our winning entry in the 2009 International Aerial Robotics Competition, which required the quadrotor to autonomously enter a hazardous unknown environment through a window, explore the indoor structure without GPS, and search for a visual target. © 2011 Wiley Periodicals, Inc.     

Multistage bayesian autonomy for high‐precision operation in a large field

This paper presents a generalized multistage bayesian framework to enable an autonomous robot to complete high‐precision operations on a static target in a large field. The proposed framework consists of two multistage approaches, capable of dealing with the complexity of high‐precision operation in a large field to detect and localize the target. In the multistage localization, locations of the robot and the target are estimated sequentially when the target is far away from the robot, whereas these locations are estimated simultaneously when the target is close. A level of confidence (LOC) for each detection criterion of a sensor and the associated probability of detection (POD) of the sensor are defined to make the target detectable with different LOCs at varying distances. Differential entropies of the robot and target are used as a precision metric for evaluating the performance of the proposed approach. The proposed multistage observation and localization approaches were applied to scenarios using an unmanned ground vehicle (UGV) and an unmanned aerial vehicle (UAV). Results with the UGV in simulated environments and then real environments show the effectiveness of the proposed approaches to real‐world problems. A successful demonstration using the UAV is also presented.     

6D scan registration using depth-interpolated local image features

This paper describes a novel registration approach that is based on a combination of visual and 3D range information. To identify correspondences, local visual features obtained from images of a standard color camera are compared and the depth of matching features (and their position covariance) is determined from the range measurements of a 3D laser scanner. The matched depth-interpolated image features allow one to apply registration with known correspondences. We compare several ICP variants in this paper and suggest an extension that considers the spatial distance between matching features to eliminate false correspondences. Experimental results are presented in both outdoor and indoor environments. In addition to pair-wise registration, we also propose a global registration method that registers all scan poses simultaneously.     

室外崎岖地形下基于视差图的无人自主车障碍物识别

对于地面无人车和室外非结构化环境, 本文介绍了我们开发的基于立体视觉的障碍物快速识别系统. 为了使无人地面车适应于较复杂的地形, 根据V视差图, 我们提出了一种新的地面主视差图的估计方法. 通过地面主视差图和局部的三维重建, 本文给出了一种由粗到精的障碍物识别与定位方法. 在我们的无人地面车平台上, 我们对这一障碍物自动识别系统进行了相应的实际试验. 其试验结果验证了该系统的有效性.     

Energy‐efficient Path Planning for Solar‐powered Mobile Robots*

We explore the problem of energy‐efficient, time‐constrained path planning of a solar‐powered robot embedded in a terrestrial environment. Because of the effects of changing weather conditions, as well as sensing concerns in complex environments, a new method for solar power prediction is desirable. We present a method that uses Gaussian Process regression to build a solar map in a data‐driven fashion. Using this map and an empirical model for energy consumption, we perform dynamic programming to find energy‐minimal paths. We validate our map construction and path‐planning algorithms with outdoor experiments, and we perform simulations on our solar maps to further determine the limits of our approach. Our results show that we can effectively construct a solar map using only a simple current measurement circuit and basic GPS localization, and this solar map can be used for energy‐efficient navigation. This establishes informed solar harvesting as a viable option for extending system lifetime even in complex environments with low‐cost commercial solar panels.