Air‐ground Matching: Appearance‐based GPS‐denied Urban Localization of Micro Aerial Vehicles

In this paper, we address the problem of globally localizing and tracking the pose of a camera‐equipped micro aerial vehicle (MAV) flying in urban streets at low altitudes without GPS. An image‐based global positioning system is introduced to localize the MAV with respect to the surrounding buildings. We propose a novel air‐ground image‐matching algorithm to search the airborne image of the MAV within a ground‐level, geotagged image database. Based on the detected matching image features, we infer the global position of the MAV by back‐projecting the corresponding image points onto a cadastral three‐dimensional city model. Furthermore, we describe an algorithm to track the position of the flying vehicle over several frames and to correct the accumulated drift of the visual odometry whenever a good match is detected between the airborne and the ground‐level images. The proposed approach is tested on a 2 km trajectory with a small quadrocopter flying in the streets of Zurich. Our vision‐based global localization can robustly handle extreme changes in viewpoint, illumination, perceptual aliasing, and over‐season variations, thus outperforming conventional visual place‐recognition approaches. The dataset is made publicly available to the research community. To the best of our knowledge, this is the first work that studies and demonstrates global localization and position tracking of a drone in urban streets with a single onboard camera.     

RANGE–Robust autonomous navigation in GPS‐denied environments

This paper addresses the problem of autonomous navigation of a micro air vehicle (MAV) in GPS‐denied environments. We present experimental validation and analysis for our system that enables a quadrotor helicopter, equipped with a laser range finder sensor, to autonomously explore and map unstructured and unknown environments. The key challenge for enabling GPS‐denied flight of a MAV is that the system must be able to estimate its position and velocity by sensing unknown environmental structure with sufficient accuracy and low enough latency to stably control the vehicle. Our solution overcomes this challenge in the face of MAV payload limitations imposed on sensing, computational, and communication resources. We first analyze the requirements to achieve fully autonomous quadrotor helicopter flight in GPS‐denied areas, highlighting the differences between ground and air robots that make it difficult to use algorithms developed for ground robots. We report on experiments that validate our solutions to key challenges, namely a multilevel sensing and control hierarchy that incorporates a high‐speed laser scan‐matching algorithm, data fusion filter, high‐level simultaneous localization and mapping, and a goal‐directed exploration module. These experiments illustrate the quadrotor helicopter's ability to accurately and autonomously navigate in a number of large‐scale unknown environments, both indoors and in the urban canyon. The system was further validated in the field by our winning entry in the 2009 International Aerial Robotics Competition, which required the quadrotor to autonomously enter a hazardous unknown environment through a window, explore the indoor structure without GPS, and search for a visual target. © 2011 Wiley Periodicals, Inc.     

GPS‐denied Indoor and Outdoor Monocular Vision Aided Navigation and Control of Unmanned Aircraft

GPS‐denied closed‐loop autonomous control of unstable Unmanned Aerial Vehicles (UAVs) such as rotorcraft using information from a monocular camera has been an open problem. Most proposed Vision aided Inertial Navigation Systems (V‐INSs) have been too computationally intensive or do not have sufficient integrity for closed‐loop flight. We provide an affirmative answer to the question of whether V‐INSs can be used to sustain prolonged real‐world GPS‐denied flight by presenting a V‐INS that is validated through autonomous flight‐tests over prolonged closed‐loop dynamic operation in both indoor and outdoor GPS‐denied environments with two rotorcraft unmanned aircraft systems (UASs). The architecture efficiently combines visual feature information from a monocular camera with measurements from inertial sensors. Inertial measurements are used to predict frame‐to‐frame transition of online selected feature locations, and the difference between predicted and observed feature locations is used to bind in real‐time the inertial measurement unit drift, estimate its bias, and account for initial misalignment errors. A novel algorithm to manage a library of features online is presented that can add or remove features based on a measure of relative confidence in each feature location. The resulting V‐INS is sufficiently efficient and reliable to enable real‐time implementation on resource‐constrained aerial vehicles. The presented algorithms are validated on multiple platforms in real‐world conditions: through a 16‐min flight test, including an autonomous landing, of a 66 kg rotorcraft UAV operating in an unconctrolled outdoor environment without using GPS and through a Micro‐UAV operating in a cluttered, unmapped, and gusty indoor environment. © 2013 Wiley Periodicals, Inc.     

Current-Aided Multiple-AUV Cooperative Localization and Target Tracking in Anchor-Free Environments

In anchor-free environments, where no devices with known positions are available, the error growth of autonomous underwater vehicle (AUV) localization and target tracking is unbounded due to the lack of references and the accumulated errors in inertial measurements. This paper aims to improve the localization and tracking accuracy by involving current information as extra references. We first integrate current measurements and maps with belief propagation and design a distributed current-aided message-passing scheme that theoretically solves the localization and tracking problems. Based on this scheme, we propose particle-based cooperative localization and target tracking algorithms, named CaCL and CaTT, respectively. In AUV localization, CaCL uses the current measurements to correct the predicted and transmitted position information and alleviates the impact of the accumulated errors in inertial measurements. With target tracking, the current maps are applied in CaTT to modify the position prediction of the target which is calculated through historical estimates. The effectiveness and robustness of the proposed methods are validated through various simulations by comparisons with alternative methods under different trajectories and current conditions.      

Vision‐based Localization and Robot‐centric Mapping in Riverine Environments

This paper presents a vision‐based localization and mapping algorithm developed for an unmanned aerial vehicle (UAV) that can operate in a riverine environment. Our algorithm estimates the three‐dimensional positions of point features along a river and the pose of the UAV. By detecting features surrounding a river and the corresponding reflections on the water's surface, we can exploit multiple‐view geometry to enhance the observability of the estimation system. We use a robot‐centric mapping framework to further improve the observability of the estimation system while reducing the computational burden. We analyze the performance of the proposed algorithm with numerical simulations and demonstrate its effectiveness through experiments with data from Crystal Lake Park in Urbana, Illinois. We also draw a comparison to existing approaches. Our experimental platform is equipped with a lightweight monocular camera, an inertial measurement unit, a magnetometer, an altimeter, and an onboard computer. To our knowledge, this is the first result that exploits the reflections of features in a riverine environment for localization and mapping.     

Terrain‐inclination‐based Three‐dimensional Localization for Mobile Robots in Outdoor Environments

A new terrain‐inclination‐based localization technique is proposed in this paper to enable a robot to identify its three‐dimensional location relative to measurable terrain inclinations. Given a topographical map and a planned path, a robot‐terrain‐inclination model (RTI model) is extracted along the path on the terrain upon which the robot is operating. A particle filter is then used to fuse the measurement data with the robot motion based on the extracted RTI model for either a three‐wheeled or a four‐wheeled mobile robot. Experiments were carried out in four outdoor scenarios: one short path with different initial conditions and map resolution, another short path with different surface roughness and sensor accuracy, and two long paths with different types of rigid terrains and multiple loops. Experimental results show that the proposed method could achieve good localization performance on inclined outdoor terrains.     

Robust Navigation and Seamless Localization for Carlike Robots in Indoor‐outdoor Environments

This paper describes a navigation and seamless localization system that permits carlike robots to move safely in heterogeneous scenarios within indoor and outdoor environments. The robot localization integrates different sensor (GPS, odometry, laser rangefinders) information depending on the kind of area (indoors, outdoors, and areas between) or on the sensor uncertainty in such a way that there are no discontinuities in the localization, and a bounded uncertainty is constantly maintained. Transitions through indoor and outdoor environments are thoroughly considered to assure a smooth change in‐between. The paper addresses a navigation technique that combines two well‐known obstacle avoidance techniques, namely the nearness diagram and the dynamic window approaches, exploiting the advantages and properties of both, and integrating the seamless localization technique. The navigation technique is developed for carlike robots by considering their shape and kinodynamic constraints, and the restrictions imposed by the environment. Forward‐backward maneuvers are also integrated in the method, allowing difficult situations in dense scenarios to be managed. The whole system has been tested in simulations and experiments in real large‐scale scenarios.     

Four‐Dimensional Geometry Lens: A Novel Volumetric Magnification Approach

We present a novel methodology that utilizes four‐dimensional (4D) space deformation to simulate a magnification lens on versatile volume datasets and textured solid models. Compared with other magnification methods (e.g. geometric optics, mesh editing), 4D differential geometry theory and its practices are much more flexible and powerful for preserving shape features (i.e. minimizing angle distortion), and easier to adapt to versatile solid models. The primary advantage of 4D space lies at the following fact: we can now easily magnify the volume of regions of interest (ROIs) from the additional dimension, while keeping the rest region unchanged. To achieve this primary goal, we first embed a 3D volumetric input into 4D space and magnify ROIs in the fourth dimension. Then we flatten the 4D shape back into 3D space to accommodate other typical applications in the real 3D world. In order to enforce distortion minimization, in both steps we devise the high‐dimensional geometry techniques based on rigorous 4D geometry theory for 3D/4D mapping back and forth to amend the distortion. Our system can preserve not only focus region, but also context region and global shape. We demonstrate the effectiveness, robustness and efficacy of our framework with a variety of models ranging from tetrahedral meshes to volume datasets.     

Cloud‐based Real‐time Outsourcing Localization for a Ground Mobile Robot in Large‐scale Outdoor Environments

Cloud robotics is the application of cloud computing concepts to robotic systems. It utilizes modern cloud computing infrastructure to distribute computing resources and datasets. Cloud‐based real‐time outsourcing localization architecture is proposed in this paper to allow a ground mobile robot to identify its location relative to a road network map and reference images in the cloud. An update of the road network map is executed in the cloud, as is the extraction of the robot‐terrain inclination (RTI) model as well as reference image matching. A particle filter with a network‐delay‐compensation localization algorithm is executed on the mobile robot based on the local RTI model and the recognized location both of which are sent from the cloud. The proposed methods are tested in different challenging outdoor scenarios with a ground mobile robot equipped with minimal onboard hardware, where the longest trajectory was 13.1 km. Experimental results show that this method could be applicable to large‐scale outdoor environments for autonomous robots in real time.     

A New Sonar-based Landmark for Localization in Indoor Environments

This paper presents a new sonar based landmark to represent significant places in an environment for localization purposes. This landmark is based on extracting the contour free of obstacles around the robot from a local evidence grid. This contour is represented by its curvature, calculated by a noise-resistant function which adapts to the natural scale of the contour at each point. Then, curvature is reduced to a short feature vector by using Principal Component Analysis. The landmark calculation method has been successfully tested in a medium scale real environment using a Pioneer robot with Polaroid sonar sensors.     

Trends and Developments in Computational Geometry

This paper discusses some trends and achievements in computational geometry during the past five years, with emphasis on problems related to computer graphics. Furthermore, a direction of research in computational geometry is discussed that could help in bringing the fields of computational geometry and computer graphics closer together.     

Human Tracking System Integrating Sound and Face Localization Using an Expectation-Maximization Algorithm in Real Environments

We have developed a human tracking system for use by robots that integrate sound and face localization. Conventional systems usually require many microphones and/or prior information to localize several sound sources. Moreover, they are incapable of coping with various types of background noise. Our system, the cross-power spectrum phase analysis of sound signals obtained with only two microphones, is used to localize the sound source without having to use prior information such as impulse response data. An expectation-maximization (EM) algorithm is used to help the system cope with several moving sound sources. The problem of distinguishing whether sounds are coming from the front or back is also solved with only two microphones by rotating the robot's head. A developed method that uses facial skin colors classified by another EM algorithm enables the system to detect faces in various poses. It can compensate for the error in the sound localization for a speaker and also identify noise signals entering from undesired directions by detecting a human face. A developed probability-based method is used to integrate the auditory and visual information in order to produce a reliable tracking path in real-time. Experiments using a robot showed that our system can localize two sounds at the same time and track a communication partner while dealing with various types of background noise.     

A Stereovision Method for Obstacle Detection and Tracking in Non-Flat Urban Environments

Obstacle detection is an essential capability for the safe guidance of autonomous vehicles, especially in urban environments. This paper presents an efficient method to integrate spatial and temporal constraints for detecting and tracking obstacles in urban environments. In order to enhance the reliability of the obstacle detection task, we do not consider the urban roads as rigid planes, but as quasi-planes, whose normal vectors have orientation constraints. Under this flexible road model, we propose a fast, robust stereovision based obstacle detection method. A watershed transformation is employed for obstacle segmentation in dense traffic conditions, even with partial occlusions, in urban environments. Finally a UKF (Unscented Kalman filter) is applied to estimate the obstacles parameters under a nonlinear observation model. To avoid the difficulty of the computation in metric space, the whole detection process is performed in the disparity image. Various experimental results are presented, showing the advantages of this method.Qian Yu received the B.E. degree in Computer Science from Tsinghua University, Beijing, China, in 2001, and the Master degree in Computer Science also from Tsinghua University in 2004, working at the Artificial Intelligence Laboratory. From October 2002 to April 2003, he was a visiting student at the Institute of System and Robotics (ISR), University of Coimbra, Portugal. His current research interests are in computer vision and robotics.Helder Araujo is currently Associate Professor in the Department of Electrical and Computer Engineering, University of Coimbra, Portugal. He is co-founder of the Portuguese Institute for Systems and Robotics (ISR), where he is now a Researcher and Vice-Director of the Coimbra pole. His primary research interests are in computer vision and mobile robotics.Hong Wang received his Ph.D. degree from the Department of Computer Science and Technology, Tsinghua University in 1993. He is currently an associate professor at Department of Computer Science and Technology, Tsinghua University. He worked as a visiting researcher at the Department of Intelligent Assistant Driving, Daimler-Benz Research, Stuttgart, Germany, from August 1996 to August 1997. His main research interests include Artificial Intelligence, Mobile Robotics, Vision Navigation, Multi-sensor Data Fusion. He has published over 40 papers in international conference and journals. He is a member of Special Committee of Machine Perception and Virtual Reality of the Chinese Association of Artificial Intelligence and a member of Scientific Committee of the Olympiad in Informatics of the Chinese Computer Association. He has served as an Associated Director of the Central Laboratory of the State Key Laboratory of Intelligent Technology and Systems, Tsinghua University.     

一种基于多分辨率的图像跟踪算法

提出了一种基于多分辨率的图像处理方法，图像在不同分辨率的层上进行处理，从而获得较快的处理速度，同时保证了一定的鲁棒性。     

一种自动的唇部定位及唇轮廓提取、跟踪方法

实现一种结合CbCr颜色空间、Fisher变换及变形模板的自动唇部定位及唇轮廓提取、跟踪方法.首先在CbCr空间建立肤色模型进行人脸检测、定位,并由人脸几何特征进行唇部粗定位.然后结合唇色模型进行Fisher变换使肤、唇色差别明显化,提出根据亮度信息对变换结果预处理后用Otsu法进行图像分割,经唇色模型进一步验证后实现唇部精定位.再使用变形模板来进行嘴唇轮廓特征提取,为增强内轮廓定位的鲁棒性,本文提出对经亮度预处理和唇色模型验证得到的口腔区域边缘图进行曲线拟合来实现内轮廓定位.最后,将唇读图像序列中上一帧的唇部定位结果拓展后作为当前帧的预测区域再进行处理来实现唇动跟踪.     

计算网格环境下一个统一的资源映射策略

由于资源具有广域分布、异构、动态等特性,计算网格环境下资源的管理和调度是一个非常复杂且具有挑战性的问题.提出了计算网格环境下一组相互独立的计算任务(meta-task)的资源映射策略.该策略采用重复映射方法,以更好地适应网格计算环境下的动态性和自治性.算法考虑到任务的输入数据位置对映射效果的影响;通过定义效益函数,该策略在追求较小的任务完成时间的同时兼顾任务的服务质量(QoS)需求.模拟实验结果显示,该映射策略更符合计算网格的复杂环境,能够更好地满足不同用户的实际需要.     

Adaptive Robust Three‐dimensional Trajectory Tracking for Actively Articulated Tracked Vehicles*

A new approach is proposed for an adaptive robust three‐dimensional (3D) trajectory‐tracking controller design. The controller is modeled for actively articulated tracked vehicles (AATVs). These vehicles have active sub‐tracks, called flippers, linked to the ends of the main tracks, to extend the locomotion capabilities in hazardous environments, such as rescue scenarios. The proposed controller adapts the flippers configuration and simultaneously generates the track velocities, to allow the vehicle to autonomously follow a given feasible 3D path. The approach develops both a direct and differential kinematic model of the AATV for traversal task execution correlating the robot body motion to the flippers motion. The benefit of this approach is to allow the controller to flexibly manage all the degrees of freedom of the AATV as well as the steering. The differential kinematic model integrates a differential drive robot model, compensating the slippage between the vehicle tracks and the traversed terrain. The underlying feedback control law dynamically accounts for the kinematic singularities of the mechanical vehicle structure. The designed controller integrates a strategy selector too, which has the role of locally modifying the rail path of the flipper end points. This serves to reduce both the effort of the flipper servo motors and the traction force on the robot body, recognizing when the robot is moving on a horizontal plane surface. Several experiments have been performed, in both virtual and real scenarios, to validate the designed trajectory‐tracking controller, while the AATV negotiates rubble, stairs, and complex terrain surfaces. Results are compared with both the performance of an alternative control strategy and the ability of skilled human operators, manually controlling the actively articulated components of the robot.     

机动目标跟踪的滤波方法研究

目标跟踪应用中,一类常见的混合估计问题是:目标运动建模在直角坐标系下且是非线性的,同时量测数据由传感器直接获得.通常处理该问题的做法是使用推广卡尔曼滤波器,但效果欠佳.为此,通过将无迹变换(UT)和BLUE算法相结合,提出了一种新型的UT-BLUE滤波器.该滤波器首先利用无迹变换对直角坐标系中的目标状态及其协方差作出预测,然后在保持传感器坐标系(极坐标系)下所固有的量测误差的同时,直接对它们作出更新估计.通过仿真, 将UT-BLUE滤波方法和EKF滤波方法进行比较,表明了该滤波方法的有效性和优越性.     

A New Approach for Automatic Theorem Proving in Real Geometry

We present a new method for proving geometric theorems in the real plane or higher dimension. The method is derived from elimination set ideas for quantifier elimination in linear and quadratic formulas over the reals. In contrast to other approaches, our method can also prove theorems whose complex analogues fail. Moreover, the problem formulation may involve order inequalities. After specification of independent variables, nondegeneracy conditions are generated automatically. Moreover, when trying to prove conjectures that – apart from nondegeneracy conditions – do not hold in the claimed generality, missing premises are found automatically. We demonstrate the applicability of our method to nontrivial examples.