Robust Gyro‐free Attitude Estimation for a Small Fixed‐wing Unmanned Aerial Vehicle

This paper proposes a backup attitude estimation scheme for small fixed‐wing unmanned aerial vehicles (UAVs) in the event of gyroscopic failure. The attitude is propagated in terms of 3 degrees‐of‐freedom (DoF) aircraft dynamics. The errors in attitude propagation are updated using indirect attitude information obtained from accelerations as sensed by onboard accelerometers and a global positioning system (GPS) receiver. In the event of gyroscopic failure, large uncertainties are introduced into the attitude propagation model. Such uncertainties in states and parameters are modeled as norm‐bound uncertainties and a discrete‐time robust extended Kalman filter (REKF) is implemented to estimate the attitude of the UAV.     

Detection of aircraft below the horizon for vision‐based detect and avoid in unmanned aircraft systems

Vision‐based aircraft detection technology may provide a credible sensing option for automated detect and avoid in small‐to‐medium size fixed‐wing unmanned aircraft systems (UAS). Reliable vision‐based aircraft detection has previously been demonstrated in sky‐region sensing environments. This paper describes a novel vision‐based system for detecting aircraft below the horizon in the presence of ground clutter. We examine the performance of our system on a data set of 63 near collision encounters we collected between a camera‐equipped manned aircraft and a below‐horizon target. In these 63 encounters, our system successfully detects all aircraft, at an average detection range of 1890 m (with a standard error of 43 m and no false alarms in 1.1 h). Furthermore, our system does not require access to inertial sensor data (which significantly reduces system cost) and operates at over 12 frames per second.     

Automated Vision‐based Recovery of a Rotary Wing Unmanned Aerial Vehicle onto a Moving Platform

This paper describes an integrated control demonstration in which a rotary wing unmanned aerial vehicle (UAV) landed on a moving platform using vision‐based localization of the target vehicle. Three key areas are covered in this paper: the system components and integration, the control system structure and design, and finally flight test data including the first such successful landing. All the development and demonstration flights were carried out in the field, and key system challenges and lessons learned are highlighted throughout the paper. Results are given for landings carried out on both stationary and translating targets, in wind speeds of up to 15 knots. While the translating target was constrained to linear motion, there are several applications such as recovery to a land‐based convoy for which this capability would be applicable. The results show that the approach adopted for this demonstration could form part of a practical solution to the problem of automated UAV recovery in both civilian and military environments.     

管制空域内无人机与有人机侧向碰撞风险研究

随着无人机的逐渐发展和使用范围的扩大,空域将变得越来越拥挤。把无人机系统融入有人机管制空域将是未来解决空域拥挤问题的必要手段之一。基于无人机性能特征和机载设备设施要求,结合有人机和无人机的延迟推导出二者之间的管制防撞间隔,并根据等效安全水平原则求出二者之间可行的目标安全水平。在Reich模型的基础上进行改进,从而建立有人机与无人机的侧向碰撞风险模型,再通过仿真算例对不同管制空域内的有人机和无人机之间进行风险评估,计算碰撞风险并与目标安全水平进行比较。仿真结果表明在1000 m以上的管制空域采用所推导的防碰撞间隔满足ICAO规定的碰撞风险要求,所以该间隔可以作为无人机融入有人机管制空域时的侧向间隔使用。     

Characterization of Sky‐region Morphological‐temporal Airborne Collision Detection

Automated airborne collision‐detection systems are a key enabling technology for facilitating the integration of unmanned aerial vehicles (UAVs) into the national airspace. These safety‐critical systems must be sensitive enough to provide timely warnings of genuine airborne collision threats, but not so sensitive as to cause excessive false alarms. Hence, an accurate characterization of detection and false‐alarm sensitivity is essential for understanding performance tradeoffs, and system designers can exploit this characterization to help achieve a desired balance in system performance. In this paper, we experimentally evaluate a sky‐region, image‐based, aircraft collision‐detection system that is based on morphological and temporal processing techniques. (Note that the examined detection approaches are not suitable for the detection of potential collision threats against a ground clutter background.) A novel collection methodology for collecting realistic airborne collision‐course target footage in both head‐on and tail‐chase engagement geometries is described. Under (hazy) blue sky conditions, our proposed system achieved detection ranges greater than 1540 m in three flight test cases with no false‐alarm events in 14.14 h of nontarget data (under cloudy conditions, the system achieved detection ranges greater than 1170 m in four flight test cases with no false‐alarm events in 6.63 h of nontarget data). Importantly, this paper is the first documented presentation of detection range versus false‐alarm curves generated from airborne target and nontarget image data. © 2012 Wiley Periodicals, Inc.     

Vision‐based Localization and Robot‐centric Mapping in Riverine Environments

This paper presents a vision‐based localization and mapping algorithm developed for an unmanned aerial vehicle (UAV) that can operate in a riverine environment. Our algorithm estimates the three‐dimensional positions of point features along a river and the pose of the UAV. By detecting features surrounding a river and the corresponding reflections on the water's surface, we can exploit multiple‐view geometry to enhance the observability of the estimation system. We use a robot‐centric mapping framework to further improve the observability of the estimation system while reducing the computational burden. We analyze the performance of the proposed algorithm with numerical simulations and demonstrate its effectiveness through experiments with data from Crystal Lake Park in Urbana, Illinois. We also draw a comparison to existing approaches. Our experimental platform is equipped with a lightweight monocular camera, an inertial measurement unit, a magnetometer, an altimeter, and an onboard computer. To our knowledge, this is the first result that exploits the reflections of features in a riverine environment for localization and mapping.     

Multistage bayesian autonomy for high‐precision operation in a large field

This paper presents a generalized multistage bayesian framework to enable an autonomous robot to complete high‐precision operations on a static target in a large field. The proposed framework consists of two multistage approaches, capable of dealing with the complexity of high‐precision operation in a large field to detect and localize the target. In the multistage localization, locations of the robot and the target are estimated sequentially when the target is far away from the robot, whereas these locations are estimated simultaneously when the target is close. A level of confidence (LOC) for each detection criterion of a sensor and the associated probability of detection (POD) of the sensor are defined to make the target detectable with different LOCs at varying distances. Differential entropies of the robot and target are used as a precision metric for evaluating the performance of the proposed approach. The proposed multistage observation and localization approaches were applied to scenarios using an unmanned ground vehicle (UGV) and an unmanned aerial vehicle (UAV). Results with the UGV in simulated environments and then real environments show the effectiveness of the proposed approaches to real‐world problems. A successful demonstration using the UAV is also presented.     

State‐of‐the‐Art in Compressed GPU‐Based Direct Volume Rendering

Great advancements in commodity graphics hardware have favoured graphics processing unit (GPU)‐based volume rendering as the main adopted solution for interactive exploration of rectilinear scalar volumes on commodity platforms. Nevertheless, long data transfer times and GPU memory size limitations are often the main limiting factors, especially for massive, time‐varying or multi‐volume visualization, as well as for networked visualization on the emerging mobile devices. To address this issue, a variety of level‐of‐detail (LOD) data representations and compression techniques have been introduced. In order to improve capabilities and performance over the entire storage, distribution and rendering pipeline, the encoding/decoding process is typically highly asymmetric, and systems should ideally compress at data production time and decompress on demand at rendering time. Compression and LOD pre‐computation does not have to adhere to real‐time constraints and can be performed off‐line for high‐quality results. In contrast, adaptive real‐time rendering from compressed representations requires fast, transient and spatially independent decompression. In this report, we review the existing compressed GPU volume rendering approaches, covering sampling grid layouts, compact representation models, compression techniques, GPU rendering architectures and fast decoding techniques.     

Autonomous recovery of a fixed‐wing UAV using a net suspended by two multirotor UAVs

This article presents a novel recovery method for fixed‐wing unmanned aerial vehicles (UAVs), aimed at enabling operations from marine vessels. Instead of using the conventional method of using a fixed net on the ship deck, we propose to suspend a net under two cooperative multirotor UAVs. While keeping their relative formation, the multirotor UAVs are able to intercept the incoming fixed‐wing UAV along a virtual runway over the sea and transport it back to the ship. In addition to discussing the concept and design a control system, this paper also presents experimental validation of the proposed concept for a small‐scale UAV platform.     

Post‐disaster assessment with unmanned aerial vehicles: A survey on practical implementations and research approaches

In this paper, we provide a review of the principal aspects related to search & rescue (SAR) with unmanned aerial vehicles (UAVs), with particular interest in the phase of post‐disaster assessment (PDA). Some areas of interest related to this topic have been chosen for the analysis: the aerial platforms used in the field, multirobot software architectures, onboard sensors and simultaneous localization and mapping approaches, terrain coverage algorithms, autonomous navigation techniques, and human‐swarm interfaces. All these aspects have been analyzed with respect to the state‐of‐the‐art, and also in relation to the project PRISMA, which focuses on the development and deployment of robots and autonomous systems that can operate in emergency scenarios, with a specific reference to monitoring and real‐time intervention.     

Obstacle Avoidance for Unmanned Aerial Vehicles

This work is framed within the PITVANT project and aims to contribute to the development of obstacle avoidance techniques for unmanned aerial vehicles (UAVs). The paper describes the design, implementation and experimental evaluation of a potential field obstacle avoidance algorithm based on the fluid mechanics panel methods. Obstacles and the UAV goal position are modeled by harmonic functions thus avoiding the presence of local minima. Adaptations are made to apply the method to the automatic control of a fixed wing aircraft, relying only on a local map of the environment that is updated with information from sensors onboard the aircraft. Hardware-In-Loop simulations show the good performance of the proposed algorithm in the envisioned mission scenarios for the PITVANT vehicles.     

RECODE: an image‐based collision detection algorithm

Object interactions are ubiquitous in interactive computer graphics, 3D object motion simulations, virtual reality and robotics applications. Most collision detection algorithms are based on geometrical object‐space interference tests. Some algorithms have employed an image‐space approach to the collision detection problem. In this paper we demonstrate an image‐space collision detection process that allows substantial computational savings during the image‐space interference test. This approach makes efficient use of the graphics rendering hardware for real‐time complex object interactions. Copyright © 1999 John Wiley & Sons, Ltd.     

Cooperative digital magnetic‐elevation maps by small autonomous aerial robots

One of the steps to provide fundamental data for planning a mining effort is the magnetic surveying of a target area, which is typically carried out by conventional aircraft campaigns. However, besides the high cost, fixed‐wing aerial vehicles present shortcomings especially for drape flights on mountainous regions, where steep slopes are often present. Traditional human‐crewed flights have to perform tedious and dangerous trajectories, under strict velocity and attitude constraints. In this paper, we deal with the problem of accomplishing digital magnetic‐elevation maps using autonomous and cooperative aerial robots. The proposed approach for autonomous mapping utilizes a custom‐built fluxgate sensor and off the shelf cameras adapted for small airborne platforms. We also propose an innovative approach for generating a digital magnetic‐elevation model from the gathered data. Our method was evaluated and validated in field tests in an industrial scenario to detect scrap metals in ore piles. Results show that the proposed method could reliably detect magnetic anomalies while generating accurate three‐dimensional magnetic maps.     

GPS‐denied Indoor and Outdoor Monocular Vision Aided Navigation and Control of Unmanned Aircraft

GPS‐denied closed‐loop autonomous control of unstable Unmanned Aerial Vehicles (UAVs) such as rotorcraft using information from a monocular camera has been an open problem. Most proposed Vision aided Inertial Navigation Systems (V‐INSs) have been too computationally intensive or do not have sufficient integrity for closed‐loop flight. We provide an affirmative answer to the question of whether V‐INSs can be used to sustain prolonged real‐world GPS‐denied flight by presenting a V‐INS that is validated through autonomous flight‐tests over prolonged closed‐loop dynamic operation in both indoor and outdoor GPS‐denied environments with two rotorcraft unmanned aircraft systems (UASs). The architecture efficiently combines visual feature information from a monocular camera with measurements from inertial sensors. Inertial measurements are used to predict frame‐to‐frame transition of online selected feature locations, and the difference between predicted and observed feature locations is used to bind in real‐time the inertial measurement unit drift, estimate its bias, and account for initial misalignment errors. A novel algorithm to manage a library of features online is presented that can add or remove features based on a measure of relative confidence in each feature location. The resulting V‐INS is sufficiently efficient and reliable to enable real‐time implementation on resource‐constrained aerial vehicles. The presented algorithms are validated on multiple platforms in real‐world conditions: through a 16‐min flight test, including an autonomous landing, of a 66 kg rotorcraft UAV operating in an unconctrolled outdoor environment without using GPS and through a Micro‐UAV operating in a cluttered, unmapped, and gusty indoor environment. © 2013 Wiley Periodicals, Inc.     

Autonomous aerial cinematography in unstructured environments with learned artistic decision‐making

Aerial cinematography is revolutionizing industries that require live and dynamic camera viewpoints such as entertainment, sports, and security. However, safely piloting a drone while filming a moving target in the presence of obstacles is immensely taxing, often requiring multiple expert human operators. Hence, there is a demand for an autonomous cinematographer that can reason about both geometry and scene context in real‐time. Existing approaches do not address all aspects of this problem; they either require high‐precision motion‐capture systems or global positioning system tags to localize targets, rely on prior maps of the environment, plan for short time horizons, or only follow fixed artistic guidelines specified before the flight. In this study, we address the problem in its entirety and propose a complete system for real‐time aerial cinematography that for the first time combines: (a) vision‐based target estimation; (b) 3D signed‐distance mapping for occlusion estimation; (c) efficient trajectory optimization for long time‐horizon camera motion; and (d) learning‐based artistic shot selection. We extensively evaluate our system both in simulation and in field experiments by filming dynamic targets moving through unstructured environments. Our results indicate that our system can operate reliably in the real world without restrictive assumptions. We also provide in‐depth analysis and discussions for each module, with the hope that our design tradeoffs can generalize to other related applications. Videos of the complete system can be found at https://youtu.be/ookhHnqmlaU .     

Vision techniques for on‐board detection,following, and mapping of moving targets

This study presents computer vision modules of a multi‐unmanned aerial vehicle (UAV) system, which scored gold, silver, and bronze medals at the Mohamed Bin Zayed International Robotics Challenge 2017. This autonomous system, which was running completely on board and in real time, had to address two complex tasks in challenging outdoor conditions. In the first task, an autonomous UAV had to find, track, and land on a human‐driven car moving at 15 km/hr on a figure‐eight‐shaped track. During the second task, a group of three UAVs had to find small colored objects in a wide area, pick them up, and deliver them into a specified drop‐off zone. The computer vision modules presented here achieved computationally efficient detection, accurate localization, robust velocity estimation, and reliable future position prediction of both the colored objects and the car. These properties had to be achieved in adverse outdoor environments with changing light conditions. Lighting varied from intense direct sunlight with sharp shadows cast over the objects by the UAV itself, to reduced visibility caused by overcast to dust and sand in the air. The results presented in this paper demonstrate good performance of the modules both during testing, which took place in the harsh desert environment of the central area of United Arab Emirates, as well as during the contest, which took place at a racing complex in the urban, near‐sea location of Abu Dhabi. The stability and reliability of these modules contributed to the overall result of the contest, where our multi‐UAV system outperformed teams from world’s leading robotic laboratories in two challenging scenarios.     

Reducing the complexity of visual navigation: Optical track controller for long‐range unmanned aerial vehicles

Using vision for navigation of airborne systems provides an opportunity for motion relative to the ground to be controlled in the absence of other supporting sensors including global navigation satellite systems. Rather than relying on computationally intensive localization techniques such as online map construction, identification and tracking of landmarks, or otherwise producing an explicit quantitative estimate of position, we propose and have experimentally demonstrated a closed‐loop visual navigation reflex which we term the optical ground course controller. The behavior is applicable to fixed wing aircraft traversing long ranges, and reduces the online computation and sensors required compared to other visual methods. This method combines the kinematics of fixed wing aircraft flight, the direction of apparent motion of an image sequence, and a magnetic compass to create a bioinspired optomotor reflex similar to those observed in insects. This behavior accurately controls track in the inertial reference frame (path taken over the ground) with only limited dependence on altitude, speed, and wind. We show that the proposed behavior is naturally convergent and stable, and present experimental results from simulation and real‐world flight demonstrating that the method performs robustly, producing improvement over both magnetic‐referenced and visual odometry‐based navigation within the limits of the sensor.     

Decentralized nonlinear robust control of UAVs in close formation

This paper treats the design of a decentralized nonlinear robust control system for formation flying of multiple unmanned aerial vehicles (UAVs). In close formation, it is assumed that vortex of any UAV affects the motion of all the UAVs behind it. The forces produced by these vortices are complex functions of relative position co‐ordinates of the UAVs. In this paper, these forces are treated as unknown functions, which cannot be parameterized. Since the system is not invertible in the wind axes system, a simplified co‐ordinate system obtained from the wind axes system for which the velocity roll (bank angle) is zero, is considered for the design of the control system. A nonlinear robust control system for the separation trajectory control of the wing aircraft in the simplified wind coordinate system is derived. Uncertain functions and unmeasured variables are estimated using a high‐gain observer for the synthesis of the control system. Each wing UAV synthesizes its control law using its own state variables and the relative position of the preceding UAV with respect to the wing UAV. Thus the control system is decentralized since each UAV has to communicate (depending on sensors for position measurement) with at most one (preceding) UAV, and no data transmission from the remaining vehicles is required. Simulation results for two UAVs are presented which show precise separation trajectory control of each wing UAV in spite of the presence of unknown and unstructured vortex forces, while the lead aircraft maneuvers. Furthermore, these results confirm that when the wing aircraft is positioned properly in the vortex of the lead aircraft, it experiences reduction in its required flight power. Copyright © 2003 John Wiley & Sons, Ltd.     

Learning‐based Nonlinear Model Predictive Control to Improve Vision‐based Mobile Robot Path Tracking

This paper presents a Learning‐based Nonlinear Model Predictive Control (LB‐NMPC) algorithm to achieve high‐performance path tracking in challenging off‐road terrain through learning. The LB‐NMPC algorithm uses a simple a priori vehicle model and a learned disturbance model. Disturbances are modeled as a Gaussian process (GP) as a function of system state, input, and other relevant variables. The GP is updated based on experience collected during previous trials. Localization for the controller is provided by an onboard, vision‐based mapping and navigation system enabling operation in large‐scale, GPS‐denied environments. The paper presents experimental results including over 3 km of travel by three significantly different robot platforms with masses ranging from 50 to 600 kg and at speeds ranging from 0.35 to 1.2 m/s (associated video at http://tiny.cc/RoverLearnsDisturbances ). Planned speeds are generated by a novel experience‐based speed scheduler that balances overall travel time, path‐tracking errors, and localization reliability. The results show that the controller can start from a generic a priori vehicle model and subsequently learn to reduce vehicle‐ and trajectory‐specific path‐tracking errors based on experience.