Robotic technologies for solar‐powered UAVs: Fully autonomous updraft‐aware aerial sensing for multiday search‐and‐rescue missions

Large‐scale aerial sensing missions can greatly benefit from the perpetual endurance capability provided by high‐performance low‐altitude solar‐powered unmanned aerial vehicles (UAVs). However, today these UAVs suffer from small payload capacity, low energetic margins, and high operational complexity. To tackle these problems, this paper presents four individual technical contributions and integrates them into an existing solar‐powered UAV system: First, a lightweight and power‐efficient day/night‐capable sensing system is discussed. Second, means to optimize the UAV platform to the specific payload and to thereby achieve sufficient energetic margins for day/night flight with payload are presented. Third, existing autonomous launch and landing functionality is extended for solar‐powered UAVs. Fourth, as a main contribution an extended Kalman filter (EKF)‐based autonomous thermal updraft tracking framework is developed. Its novelty is that it allows the end‐to‐end integration of the thermal‐induced roll moment into the estimation process. It is assessed against unscented Kalman filter and particle filter methods in simulation and implemented on the aircraft's low‐power autopilot. The complete system is verified during a 26 h search‐and‐rescue aerial sensing mock‐up mission that represents the first‐ever fully autonomous perpetual endurance flight of a small solar‐powered UAV with a day/night‐capable sensing payload. It also represents the first time that solar‐electric propulsion and autonomous thermal updraft tracking are combined in flight. In contrast to previous work that has focused on the energetic feasibility of perpetual flight, the individual technical contributions of this paper are considered core functionality to guarantee ease‐of‐use, effectivity, and reliability in future multiday aerial sensing operations with small solar‐powered UAVs.     

High performance and safe flight of full‐scale helicopters from takeoff to landing with an ensemble of planners

Autonomous flight of unmanned full‐size rotor‐craft has the potential to enable many new applications. However, the dynamics of these aircraft, prevailing wind conditions, the need to operate over a variety of speeds and stringent safety requirements make it difficult to generate safe plans for these systems. Prior work has shown results for only parts of the problem. Here we present the first comprehensive approach to planning safe trajectories for autonomous helicopters from takeoff to landing. Our approach is based on two key insights. First, we compose an approximate solution by cascading various modules that can efficiently solve different relaxations of the planning problem. Our framework invokes a long‐term route optimizer, which feeds a receding‐horizon planner which in turn feeds a high‐fidelity safety executive. Secondly, to deal with the diverse planning scenarios that may arise, we hedge our bets with an ensemble of planners. We use a data‐driven approach that maps a planning context to a diverse list of planning algorithms that maximize the likelihood of success. Our approach was extensively evaluated in simulation and in real‐world flight tests on three different helicopter systems for duration of more than 109 autonomous hours and 590 pilot‐in‐the‐loop hours. We provide an in‐depth analysis and discuss the various tradeoffs of decoupling the problem, using approximations and leveraging statistical techniques. We summarize the insights with the hope that it generalizes to other platforms and applications.     

基于改进D*算法的无人机室内路径规划

针对多旋翼飞行器室内无GPS信号时的导航问题,本文采用二维码阵列构建室内定位系统,基于改进D*算法实现无人机室内路径规划,从而实现飞行器在室内的自主导航和避障。基于ArUco二维码设计了地面阵列为无人机提供了全局精确定位信息,使用改进D*算法保证了无人机在飞行过程中能自主进行路径规划和飞行。通过设计实验对改进D*算法进行了数值仿真验证,并在实际无人机的飞行中应用。实验结果证明:所提改进算法较传统D*算法能更好地保证无人机的飞行安全,同时基于二维码阵列的定位方式不但具有较高精度同时成本低易于实现。     

Design of small hand‐launched solar‐powered UAVs: From concept study to a multi‐day world endurance record flight

We present the development process behind AtlantikSolar, a small 6.9 kg hand‐launchable low‐altitude solar‐powered unmanned aerial vehicle (UAV) that recently completed an 81‐hour continuous flight and thereby established a new flight endurance world record for all aircraft below 50 kg mass. The goal of our work is to increase the usability of such solar‐powered robotic aircraft by maximizing their perpetual flight robustness to meteorological deteriorations such as clouds or winds. We present energetic system models and a design methodology, implement them in our publicly available conceptual design framework for perpetual flight‐capable solar‐powered UAVs, and finally apply the framework to the AtlantikSolar UAV. We present the detailed AtlantikSolar characteristics as a practical design example. Airframe, avionics, hardware, state estimation, and control method development for autonomous flight operations are described. Flight data are used to validate the conceptual design framework. Flight results from the continuous 81‐hour and 2,338 km covered ground distance flight show that AtlantikSolar achieves 39% minimum state‐of‐charge, 6.8 h excess time and 6.2 h charge margin. These performance metrics are a significant improvement over previous solar‐powered UAVs. A performance outlook shows that AtlantikSolar allows perpetual flight in a 6‐month window around June 21 at mid‐European latitudes, and that multi‐day flights with small optical‐ or infrared‐camera payloads are possible for the first time. The demonstrated performance represents the current state‐of‐the‐art in solar‐powered low‐altitude perpetual flight performance. We conclude with lessons learned from the three‐year AtlantikSolar UAV development process and with a sensitivity analysis that identifies the most promising technological areas for future solar‐powered UAV performance improvements.     

Dependable Low‐altitude Obstacle Avoidance for Robotic Helicopters Operating in Rural Areas

This paper presents a system enabling robotic helicopters to fly safely without user interaction at low altitude over unknown terrain with static obstacles. The system includes a novel reactive behavior‐based method that guides rotorcraft reliably to specified locations in sparsely occupied environments. System dependability is, among other things, achieved by utilizing proven system components in a component‐based design and incorporating safety margins and safety modes. Obstacle and terrain detection is based on a vertically mounted off‐the‐shelf two‐dimensional LIDAR system. We introduce two flight modes, pirouette descent and waggle cruise, which extend the field of view of the sensor by yawing the aircraft. The two flight modes ensure that all obstacles above a minimum size are detected in the direction of travel. The proposed system is designed for robotic helicopters with velocity and yaw control inputs and a navigation system that provides position, velocity, and attitude information. It is cost effective and can be easily implemented on a variety of helicopters of different sizes. We provide sufficient detail to facilitate the implementation on single‐rotor helicopters with a rotor diameter of approximately 1.8 m. The system was extensively flight‐tested in different real‐world scenarios in Queensland, Australia. The tests included flights beyond visual range without a backup pilot. Experimental results show that it is feasible to perform dependable autonomous flight using simple but effective methods. © 2013 Wiley Periodicals, Inc.     

Detection of aircraft below the horizon for vision‐based detect and avoid in unmanned aircraft systems

Vision‐based aircraft detection technology may provide a credible sensing option for automated detect and avoid in small‐to‐medium size fixed‐wing unmanned aircraft systems (UAS). Reliable vision‐based aircraft detection has previously been demonstrated in sky‐region sensing environments. This paper describes a novel vision‐based system for detecting aircraft below the horizon in the presence of ground clutter. We examine the performance of our system on a data set of 63 near collision encounters we collected between a camera‐equipped manned aircraft and a below‐horizon target. In these 63 encounters, our system successfully detects all aircraft, at an average detection range of 1890 m (with a standard error of 43 m and no false alarms in 1.1 h). Furthermore, our system does not require access to inertial sensor data (which significantly reduces system cost) and operates at over 12 frames per second.     

考虑信息成功传递概率的多无人机协同目标最优观测与跟踪

针对考虑通信因素的多无人机协同目标最优观测与跟踪问题, 引入费舍信息矩阵对无人机探测所获取的信息进行表征, 考虑无线通信链路特性并对无人机间信息成功传递概率进行建模. 以无人机群体所获取的关于目标的信息量为指标函数, 分别建立是否考虑通信因素情况下的多机协同目标最优观测及跟踪问题模型. 对两种情况下的多机协同目标观测与跟踪进行仿真比较, 仿真结果验证了所建模型的有效性, 并体现了通信因素的重要影响.

     

Airborne vision‐based collision‐detection system

Machine vision represents a particularly attractive solution for sensing and detecting potential collision‐course targets due to the relatively low cost, size, weight, and power requirements of vision sensors (as opposed to radar and Traffic Alert and Collision Avoidance System). This paper describes the development and evaluation of a real‐time, vision‐based collision‐detection system suitable for fixed‐wing aerial robotics. Using two fixed‐wing unmanned aerial vehicles (UAVs) to recreate various collision‐course scenarios, we were able to capture highly realistic vision (from an onboard camera perspective) of the moments leading up to a collision. This type of image data is extremely scarce and was invaluable in evaluating the detection performance of two candidate target detection approaches. Based on the collected data, our detection approaches were able to detect targets at distances ranging from 400 to about 900 m. These distances (with some assumptions about closing speeds and aircraft trajectories) translate to an advance warning of between 8 and 10 s ahead of impact, which approaches the 12.5‐s response time recommended for human pilots. We overcame the challenge of achieving real‐time computational speeds by exploiting the parallel processing architectures of graphics processing units (GPUs) found on commercial‐off‐the‐shelf graphics devices. Our chosen GPU device suitable for integration onto UAV platforms can be expected to handle real‐time processing of 1,024 × 768 pixel image frames at a rate of approximately 30 Hz. Flight trials using manned Cessna aircraft in which all processing is performed onboard will be conducted in the near future, followed by further experiments with fully autonomous UAV platforms. © 2010 Wiley Periodicals, Inc.     

Field observation of tornadic supercells by multiple autonomous fixed‐wing unmanned aircraft

This paper presents the results of the design and field deployment of multiple autonomous fixed‐wing unmanned aircraft into supercell thunderstorms. As part of a field campaign in Spring 2019, up to three fixed‐wing unmanned aircraft were deployed simultaneously into different regions of supercell thunderstorms, To learn more about the atmospheric conditions that lead to the formation of tornadoes. Successful field deployment is attributed to (a) a nomadic concept of operations that allows the unmanned aircraft system team and science team to work seamlessly together while satisfying all aviation regulations and (b) the ruggedized RAAVEN unmanned aircraft system with modular features that favor rapid, ease‐of‐use over the brute strength of previous designs. The concept of operations and the unmanned aircraft system are described along with results from a 4 day window where four storms were sampled: two of these storms were tornadic (formed tornadoes before, during, or after being sampled) and two were not. These results validate the feasibility of nomadic operation of multiple unmanned aircraft simultaneously in severe weather conditions. Further, the successful field deployments demonstrate the importance of the modular unmanned aircraft design.     

RANGE–Robust autonomous navigation in GPS‐denied environments

This paper addresses the problem of autonomous navigation of a micro air vehicle (MAV) in GPS‐denied environments. We present experimental validation and analysis for our system that enables a quadrotor helicopter, equipped with a laser range finder sensor, to autonomously explore and map unstructured and unknown environments. The key challenge for enabling GPS‐denied flight of a MAV is that the system must be able to estimate its position and velocity by sensing unknown environmental structure with sufficient accuracy and low enough latency to stably control the vehicle. Our solution overcomes this challenge in the face of MAV payload limitations imposed on sensing, computational, and communication resources. We first analyze the requirements to achieve fully autonomous quadrotor helicopter flight in GPS‐denied areas, highlighting the differences between ground and air robots that make it difficult to use algorithms developed for ground robots. We report on experiments that validate our solutions to key challenges, namely a multilevel sensing and control hierarchy that incorporates a high‐speed laser scan‐matching algorithm, data fusion filter, high‐level simultaneous localization and mapping, and a goal‐directed exploration module. These experiments illustrate the quadrotor helicopter's ability to accurately and autonomously navigate in a number of large‐scale unknown environments, both indoors and in the urban canyon. The system was further validated in the field by our winning entry in the 2009 International Aerial Robotics Competition, which required the quadrotor to autonomously enter a hazardous unknown environment through a window, explore the indoor structure without GPS, and search for a visual target. © 2011 Wiley Periodicals, Inc.     

GPS‐denied Indoor and Outdoor Monocular Vision Aided Navigation and Control of Unmanned Aircraft

GPS‐denied closed‐loop autonomous control of unstable Unmanned Aerial Vehicles (UAVs) such as rotorcraft using information from a monocular camera has been an open problem. Most proposed Vision aided Inertial Navigation Systems (V‐INSs) have been too computationally intensive or do not have sufficient integrity for closed‐loop flight. We provide an affirmative answer to the question of whether V‐INSs can be used to sustain prolonged real‐world GPS‐denied flight by presenting a V‐INS that is validated through autonomous flight‐tests over prolonged closed‐loop dynamic operation in both indoor and outdoor GPS‐denied environments with two rotorcraft unmanned aircraft systems (UASs). The architecture efficiently combines visual feature information from a monocular camera with measurements from inertial sensors. Inertial measurements are used to predict frame‐to‐frame transition of online selected feature locations, and the difference between predicted and observed feature locations is used to bind in real‐time the inertial measurement unit drift, estimate its bias, and account for initial misalignment errors. A novel algorithm to manage a library of features online is presented that can add or remove features based on a measure of relative confidence in each feature location. The resulting V‐INS is sufficiently efficient and reliable to enable real‐time implementation on resource‐constrained aerial vehicles. The presented algorithms are validated on multiple platforms in real‐world conditions: through a 16‐min flight test, including an autonomous landing, of a 66 kg rotorcraft UAV operating in an unconctrolled outdoor environment without using GPS and through a Micro‐UAV operating in a cluttered, unmapped, and gusty indoor environment. © 2013 Wiley Periodicals, Inc.     

基于GPS/北斗的超低空防撞警告系统设计与实现

随着我国民航业尤其是通用航空领域的飞速发展,以及低空开放政策的实施,飞机数量迅速增长,空域变得更加繁忙和拥挤,同时增加了飞机在超低空飞行危险接近的可能性。同时,随着超低空无人机数目的增加,给民航系统带来了一定的安全隐患。基于GPS/北斗的超低空飞机防撞警告系统在超低空范围内为飞行器提供其附近空域中的交通状况,通过4G移动通信实现对飞行数据信息的实时接收处理和对飞行器的实时监控,在当飞行器存在潜在的安全隐患时,作出相应等级的警告并提前向相应飞行器发出警告信息,帮助飞行器选择安全的航路,避免飞行器间的碰撞危险。     

Vision‐based navigation through urban canyons

We address the problem of navigating unmanned vehicles safely through urban canyons in two dimensions using only vision‐based techniques. Two commonly used vision‐based obstacle avoidance techniques (namely stereo vision and optic flow) are implemented on an aerial and a ground‐based robotic platform and evaluated for urban canyon navigation. Optic flow is evaluated for its ability to produce a centering response between obstacles, and stereo vision is evaluated for detecting obstacles to the front. We also evaluate a combination of these two techniques, which allows a vehicle to detect obstacles to the front while remaining centered between obstacles to the side. Through experiments on an unmanned ground vehicle and in simulation, this combination is shown to be beneficial for navigating urban canyons, including T‐junctions and 90‐deg bends. Experiments on a rotorcraft unmanned aerial vehicle, which was constrained to two‐dimensional flight, demonstrate that stereo vision allowed it to detect an obstacle to the front, and optic flow allowed it to turn away from obstacles to the side. We discuss the theory behind these techniques, our experience in implementing them on the robotic platforms, and their suitability to the urban canyon navigation problem. © 2009 Wiley Periodicals, Inc.     

Characterization of Sky‐region Morphological‐temporal Airborne Collision Detection

Automated airborne collision‐detection systems are a key enabling technology for facilitating the integration of unmanned aerial vehicles (UAVs) into the national airspace. These safety‐critical systems must be sensitive enough to provide timely warnings of genuine airborne collision threats, but not so sensitive as to cause excessive false alarms. Hence, an accurate characterization of detection and false‐alarm sensitivity is essential for understanding performance tradeoffs, and system designers can exploit this characterization to help achieve a desired balance in system performance. In this paper, we experimentally evaluate a sky‐region, image‐based, aircraft collision‐detection system that is based on morphological and temporal processing techniques. (Note that the examined detection approaches are not suitable for the detection of potential collision threats against a ground clutter background.) A novel collection methodology for collecting realistic airborne collision‐course target footage in both head‐on and tail‐chase engagement geometries is described. Under (hazy) blue sky conditions, our proposed system achieved detection ranges greater than 1540 m in three flight test cases with no false‐alarm events in 14.14 h of nontarget data (under cloudy conditions, the system achieved detection ranges greater than 1170 m in four flight test cases with no false‐alarm events in 6.63 h of nontarget data). Importantly, this paper is the first documented presentation of detection range versus false‐alarm curves generated from airborne target and nontarget image data. © 2012 Wiley Periodicals, Inc.     

Decomposition‐based mission planning for fixed‐wing UAVs surveying in wind

This paper presents a new method for planning fixed‐wing aerial survey paths that ensures efficient image coverage of a large complex agricultural field in the presence of wind. By decomposing any complex polygonal field into multiple convex polygons, the traditional back‐and‐forth boustrophedon paths can be used to ensure coverage of these decomposed regions. To decompose a complex field in an efficient and fast manner, a top‐down recursive greedy approach is used to traverse the search space to minimize the flight time of the survey. This optimization can be computed fast enough for use in the field. As wind can severely affect flight time, it is included in the flight time calculation in a systematic way using a verified cost function that offers greatly reduced survey times in the wind. Other improved cost functions have been developed to take into account real‐world problems, for example, No‐Fly Zones, in addition to flight time. A number of real surveys are performed to show the flight time in wind model is accurate, to make further comparisons to previous techniques and to show that the proposed method works in real‐world conditions providing total image coverage. A number of missions are generated and flown for real complex agricultural fields. In addition to this, the wind field around a survey area is measured from a multirotor carrying an ultrasonic wind speed sensor. This shows that the assumption of steady uniform wind holds true for the small areas and time scales of an unmanned aerial vehicle aerial survey.     

Improved 3D Interpolation-Based Path Planning for a Fixed-Wing Unmanned Aircraft

Path planning for unmanned aircraft has attracted a remarkable amount of interest from the research community. However, planning in large environments such as the civil airspace has not been addressed extensively. In this paper we apply a heuristic incremental interpolation-based search algorithm with efficient replanning capabilities to the path planning problem for a fixed-wing aircraft operating in a natural environment to plan and re-plan long flight paths. We modified the algorithm to account for the minimum turning radius and the limited flight path angles of a fixed-wing aircraft. Additionally, we present a method to consider a desired minimum cruising altitude and a post-processing algorithm to improve the path and remove unnecessary path points. These properties specific to aircraft operation could not be addressed with the original algorithm. Simulation results show that the planner produces intuitive, short paths and is capable of exploiting previous planning efforts, when unknown obstacles are encountered.     

Vision‐aided Guidance and Navigation for Close Formation Flight

Close formation flight can extend an unmanned aerial vehicle's (UAV) range and endurance by utilizing lift from a wingman's wake vortices and by autonomous midair refueling or recharging. The prohibitive challenge in each of these applications is the highly accurate and reliable relative positioning that is required to station‐keep in the wingman's wake and to dock, amid external disturbances. Global navigation satellite systems are well‐suited to reliable absolute positioning, but they fall short for accurate relative positioning. This work proposes a relative positioning solution for UAV rendezvous and close formation flight that has been verified in multiple flight tests. A nonlinear estimation framework uses precise air‐to‐air measurements to correct onboard sensor measurements and produce an accurate relative state estimate that is resilient to intermittent relative measurement outages and degrades gracefully during extended outages. A guidance strategy compensates for wingman turn dynamics, acts explicitly on the estimated relative state, and is applicable to both rendezvous and formation flight. Ground testing showed a relative position estimate accuracy that is 2% of the separation distance, with successful detection and correspondence at up to 36 m. Autonomous close formation flight tests verified the relative positioning solution over extended periods, as close as two wingspans, in winds that were 30%–40% of the cruise airspeed, and at altitudes as low as 15 m. Root‐mean‐square relative position errors were 1.2 m horizontally and 0.44 m vertically during flights at the closest separation.     

雷达监测协同飞行器列阵飞行控制系统设计

当前飞行器列阵飞行控制技术中单机间相对位置不易确定,造成列阵飞行状态监测和姿态感知较难,存在控制准确度低、可靠性差的问题。据此,设计一种基于改进降噪目标提取算法的地面站可见光监测控制系统。通过对列阵中各单机飞行状态的图像监测,确定位置与姿态,以及其在列阵中的位置偏差,给出实时动态调整指令;设计的改进型降噪目标提取算法,降低了复杂背景噪声对飞行器目标提取的影响,提高了监测准确性。该系统将飞行状态监测和姿态感知功能设置于地面站中,极大减少了对飞机的载荷依赖,实现对飞行器列阵飞行的智能管理和任务规划。经实验验证,该智能控制系统的偏差率维持在2%以下,结果表明本智能控制系统能够准确、可靠的控制50架飞行器列阵的静态和动态行动。     

Multiobjective optimization–based fault‐tolerant flight control system design

The loss of measurements used for controller scheduling or envelope protection in modern flight control systems due to sensor failures leads to a challenging fault‐tolerant control law design problem. In this article, an approach to design such a robust fault‐tolerant control system, including full envelope protections using multiobjective optimization techniques, is proposed. The generic controller design and controller verification problems are derived and solved using novel multiobjective hybrid genetic optimization algorithms. These algorithms combine the multiobjective genetic search strategy with local, single‐objective optimization to improve convergence speed. The proposed strategies are applied to the design of a fault‐tolerant flight control system for a modern civil aircraft. The results of an industrial controller verification and validation campaign using an industrial benchmark simulator are reported.     

Design and Implementation of a Flight Control System for an Unmanned Rotorcraft using RPT Control Approach

In this paper, we apply a so‐called robust and perfect tracking (RPT) control technique to the design and implementation of the flight control system of a miniature unmanned rotorcraft, named HeLion. To make the presented work self‐contained, we will first outline some background knowledge, including mainly the nonlinear flight dynamics model and the inner‐loop flight control system design. Next, the highlight of this paper, that is, the outer‐loop flight control system design procedure using RPT control technique, will be detailed. Generally speaking, RPT control technique aims to design a controller such that (i) the resulting closed‐loop system is asymptotically stable, and (ii) the controlled output almost perfectly tracks a given reference signal in the presence of any initial conditions and external disturbances. Since it makes use of all possible information including the system measurement output and the command reference signal together with all its derivatives (if available) for control, RPT control technique is particularly useful for the outer‐loop layer of an unmanned aircraft. Both simulation and flight‐test results will be presented and analyzed at the end of this paper, and the efficiency of the RPT control approach will be evaluated comprehensively.