Experiments in unmanned aerial vehicle/unmanned ground vehicle radiation search

This paper discusses the results of a field experiment conducted at Savannah River National Laboratory to test the performance of several algorithms for the localization of radioactive materials. In this multirobot system, both an unmanned aerial vehicle, a custom hexacopter, and an unmanned ground vehicle (UGV), the ClearPath Jackal, equipped with γ‐ray spectrometers, were used to collect data from two radioactive source configurations. Both the Fourier scattering transform and the Laplacian eigenmap algorithms for source detection were tested on the collected data sets. These algorithms transform raw spectral measurements into alternate spaces to allow clustering to detect trends within the data which indicate the presence of radioactive sources. This study also presents a point source model and accompanying information‐theoretic active exploration algorithm. Field testing validated the ability of this model to fuse aerial and ground collected radiation measurements, and the exploration algorithm’s ability to select informative actions to reduce model uncertainty, allowing the UGV to locate radioactive material online.     

Detection of bodies in maritime rescue operations using unmanned aerial vehicles with multispectral cameras

In this study, we use unmanned aerial vehicles equipped with multispectral cameras to search for bodies in maritime rescue operations. A series of flights were performed in open‐water scenarios in the northwest of Spain, using a certified aquatic rescue dummy in dangerous areas and real people when the weather conditions allowed it. The multispectral images were aligned and used to train a convolutional neural network for body detection. An exhaustive evaluation was performed to assess the best combination of spectral channels for this task. Three approaches based on a MobileNet topology were evaluated, using (a) the full image, (b) a sliding window, and (c) a precise localization method. The first method classifies an input image as containing a body or not, the second uses a sliding window to yield a class for each subimage, and the third uses transposed convolutions returning a binary output in which the body pixels are marked. In all cases, the MobileNet architecture was modified by adding custom layers and preprocessing the input to align the multispectral camera channels. Evaluation shows that the proposed methods yield reliable results, obtaining the best classification performance when combining green, red‐edge, and near‐infrared channels. We conclude that the precise localization approach is the most suitable method, obtaining a similar accuracy as the sliding window but achieving a spatial localization close to 1 m. The presented system is about to be implemented for real maritime rescue operations carried out by Babcock Mission Critical Services Spain.     

A mission management system for complex aerial logistics by multiple unmanned aerial vehicles in MBZIRC 2017

In this study, we present a system that manages multiple unmanned aerial vehicles (UAVs) for a search, pickup, and drop mission in the 2017 Mohamed Bin Zayed International Robotics Challenge (MBZIRC). Three UAVs picked up and dropped 23 circular and rectangular targets into a designated drop box. To control the operation of three UAVs flying over an arena of 90 × 60 m, we designed and integrated a set of technologies into our system: airspace allocation, communication framework among UAVs, anticollision based on geofencing, and a token‐based prioritization for coordination. The proposed UAV system uses a single GPS and its error of a few meters is solved by means of the following component technologies: (a) flight path generator based on one reference point, (b) vision‐based redefinition of a reference point for GPS correction, and (c) calibration of flight path to update the reference point. The pickup‐and‐drop mission is conducted via color‐ and shape‐based vision processing and a magnetic gripper to pickup and drop‐off the targets. Our proposed system is able to successfully manage three UAVs, recognize targets on the ground, and drop the targets into a drop box in the drop zone. Finally, we achieved fourth place among 18 teams in Challenge 3.     

Algorithms and experiments on routing of unmanned aerial vehicles with mobile recharging stations

We study the problem of planning a tour for an energy‐limited Unmanned Aerial Vehicle (UAV) to visit a set of sites in the least amount of time. We envision scenarios where the UAV can be recharged at a site or along an edge either by landing on stationary recharging stations or on Unmanned Ground Vehicles (UGVs) acting as mobile recharging stations. This leads to a new variant of the Traveling Salesperson Problem (TSP) with mobile recharging stations. We present an algorithm that finds not only the order in which to visit the sites but also when and where to land on the charging stations to recharge. Our algorithm plans tours for the UGVs as well as determines the best locations to place stationary charging stations. We study three variants for charging: Multiple stationary charging stations, single mobile charging station, and multiple mobile charging stations. As the problems we study are nondeterministic polynomial time (NP)‐Hard, we present a practical solution using Generalized TSP that finds the optimal solution that minimizes the total time, subject to the discretization of battery levels. If the UGVs are slower than the UAVs, then the algorithm also finds the minimum number of UGVs required to support the UAV mission such that the UAV is not required to wait for the UGV. Our simulation results show that the running time is acceptable for reasonably sized instances in practice. We evaluate the performance of our algorithm through simulations and proof‐of‐concept field experiments with a fully autonomous system of one UAV and UGV.     

Unsupervised learning‐based flexible framework for surveillance planning with aerial vehicles

The herein studied problem is motivated by practical needs of our participation in the Mohamed Bin Zayed International Robotics Challenge (MBZIRC) 2017 in which a team of unmanned aerial vehicles (UAVs) is requested to collect objects in the given area as quickly as possible and score according to the rewards associated with the objects. The mission time is limited, and the most time‐consuming operation is the collection of the objects themselves. Therefore, we address the problem to quickly identify the most valuable objects as surveillance planning with curvature‐constrained trajectories. The problem is formulated as a multivehicle variant of the Dubins traveling salesman problem with neighborhoods (DTSPN). Based on the evaluation of existing approaches to the DTSPN, we propose to use unsupervised learning to find satisfiable solutions with low computational requirements. Moreover, the flexibility of unsupervised learning allows considering trajectory parametrization that better fits the motion constraints of the utilized hexacopters that are not limited by the minimal turning radius as the Dubins vehicle. We propose to use Bézier curves to exploit the maximal vehicle velocity and acceleration limits. Besides, we further generalize the proposed approach to 3D surveillance planning. We report on evaluation results of the developed algorithms and experimental verification of the planned trajectories using the real UAVs utilized in our participation in MBZIRC 2017.     

Source term estimation of a hazardous airborne release using an unmanned aerial vehicle

Gaining information about an unknown gas source is a task of great importance with applications in several areas, including responding to gas leaks or suspicious smells, quantifying sources of emissions, or in an emergency response to an industrial accident or act of terrorism. In this paper, a method to estimate the source term of a gaseous release using measurements of concentration obtained from an unmanned aerial vehicle (UAV) is described. The source term parameters estimated include the three‐dimensional location of the release, its emission rate and other important variables needed to forecast the spread of the gas using an atmospheric transport and dispersion model. The parameters of the source are estimated by fusing concentration observations from a gas detector on‐board the aircraft, with meteorological data and an appropriate model of dispersion. Two models are compared in this paper, both derived from analytical solutions to the advection–diffusion equation. Bayes’ theorem, implemented using a sequential Monte Carlo algorithm, is used to estimate the source parameters to take into account the large uncertainties in the observations and formulated models. The system is verified with novel, outdoor, fully automated experiments, where observations from the UAV are used to estimate the parameters of a diffusive source. The estimation performance of the algorithm is assessed subject to various flight path configurations and wind speeds. Observations and lessons learned during these unique experiments are discussed, and areas for future research are identified.     

Team NimbRo at MBZIRC 2017: Fast landing on a moving target and treasure hunting with a team of micro aerial vehicles

The Mohamed Bin Zayed International Robotics Challenge (MBZIRC) 2017 has defined ambitious new benchmarks to advance the state‐of‐the‐art in autonomous operation of ground‐based and flying robots. This study covers our approaches to solve the two challenges that involved micro aerial vehicles (MAV). Challenge 1 required reliable target perception, fast trajectory planning, and stable control of an MAV to land on a moving vehicle. Challenge 3 demanded a team of MAVs to perform a search and transportation task, coined “Treasure Hunt,” which required mission planning and multirobot coordination as well as adaptive control to account for the additional object weight. We describe our base MAV setup and the challenge‐specific extensions, cover the camera‐based perception, explain control and trajectory‐planning in detail, and elaborate on mission planning and team coordination. We evaluated our systems in simulation as well as with real‐robot experiments during the competition in Abu Dhabi. With our system, we—as part of the larger team NimbRo—won the MBZIRC Grand Challenge and achieved a third place in both subchallenges involving flying robots.     

A neuro-evolutionary approach to control surface segmentation for micro aerial vehicles

This paper addresses control surface segmentation in micro aerial vehicles (MAVs) by leveraging neuro-evolutionary techniques that allow the control of a higher number of control surfaces. Applying classical control methods to MAVs is a difficult process due to the complexity of the control laws with fast and highly non-linear dynamics. These methods are mostly based on models that are difficult to obtain for dynamic and stochastic environments. Moreover, these problems are exacerbated when both the number of control surfaces increases and the model’s accuracy in determining the impact of each control surface decreases. Instead, we focus on neuro-evolutionary techniques that have been successfully applied in many domains with limited models and highly non-linear dynamics. Wind tunnel simulations with Athena Vortex Lattice show that MAV performances are improved in terms of both reduced deflection angles and reduced drag (up to 5%) over a simplified model in two sets of experiments with different objective functions. We also show robustness to actuator failure with desired roll moment values still attained with failed actuators in the system through the neuro-controller.     

Cooperative autonomous search,grasping, and delivering in a treasure hunt scenario by a team of unmanned aerial vehicles

This paper addresses the problem of autonomous cooperative localization, grasping and delivering of colored ferrous objects by a team of unmanned aerial vehicles (UAVs). In the proposed scenario, a team of UAVs is required to maximize the reward by collecting colored objects and delivering them to a predefined location. This task consists of several subtasks such as cooperative coverage path planning, object detection and state estimation, UAV self‐localization, precise motion control, trajectory tracking, aerial grasping and dropping, and decentralized team coordination. The failure recovery and synchronization job manager is used to integrate all the presented subtasks together and also to decrease the vulnerability to individual subtask failures in real‐world conditions. The whole system was developed for the Mohamed Bin Zayed International Robotics Challenge (MBZIRC) 2017, where it achieved the highest score and won Challenge No. 3—Treasure Hunt. This paper does not only contain results from the MBZIRC 2017 competition but it also evaluates the system performance in simulations and field tests that were conducted throughout the year‐long development and preparations for the competition.     

Low computational-complexity algorithms for vision-aided inertial navigation of micro aerial vehicles

This paper presents low computational-complexity methods for micro-aerial-vehicle localization in GPS-denied environments. All the presented algorithms rely only on the data provided by a single onboard camera and an Inertial Measurement Unit (IMU). This paper deals with outlier rejection and relative-pose estimation. Regarding outlier rejection, we describe two methods. The former only requires the observation of a single feature in the scene and the knowledge of the angular rates from an IMU, under the assumption that the local camera motion lies in a plane perpendicular to the gravity vector. The latter requires the observation of at least two features, but it relaxes the hypothesis on the vehicle motion, being therefore suitable to tackle the outlier detection problem in the case of a 6DoF motion. We show also that if the camera is rigidly attached to the vehicle, motion priors from the IMU can be exploited to discard wrong estimations in the framework of a 2-point-RANSAC-based approach. Thanks to their inherent efficiency, the proposed methods are very suitable for resource-constrained systems. Regarding the pose estimation problem, we introduce a simple algorithm that computes the vehicle pose from the observation of three point features in a single camera image, once that the roll and pitch angles are estimated from IMU measurements. The proposed algorithm is based on the minimization of a cost function. The proposed method is very simple in terms of computational cost and, therefore, very suitable for real-time implementation. All the proposed methods are evaluated on both synthetic and real data.     

Design,control, and visual navigation of the DelftaCopter VTOL tail‐sitter UAV

To participate in the Outback Medical Express UAV Challenge 2016, a vehicle was designed and tested that can autonomously hover precisely, takeoff and land vertically, fly fast forward efficiently, and use computer vision to locate a person and a suitable landing location. The vehicle is a novel hybrid tail‐sitter combining a delta‐shaped biplane fixed‐wing and a conventional helicopter rotor. The rotor and wing are mounted perpendicularly to each other,and the entire vehicle pitches down to transition from hover to fast forward flight where the rotor serves as propulsion. To deliver sufficient thrust in hover while still being efficient in fast forward flight, a custom rotor system was designed. The theoretical design was validated with energy measurements, wind tunnel tests, and application in real‐world missions. A rotor‐head and corresponding control algorithm were developed to allow transitioning flight with the nonconventional rotor dynamics that are caused by the fuselage rotor interaction. Dedicated electronics were designed that meet vehicle needs and comply with regulations to allow safe flight beyond visual line of sight. Vision‐based search and guidance algorithms running on a stereo‐vision fish‐eye camera were developed and tested to locate a person in cluttered terrain never seen before. Flight tests and a competition participation illustrate the applicability of the DelftaCopter concept.     

Into the dirt: Datasets of sewer networks with aerial and ground platforms

This paper presents an unprecedented set of data in a challenging underground environment: the visitable sewers of Barcelona. To the best of our knowledge, this is the first data set involving ground and aerial robots in such scenario: the sewer inspection autonomous robot (SIAR) ground robot and the autonomous robot for sewer inspection aerial platform. These platforms captured data from a great variety of sensors, including sequences of red green blue‐depth (RGB‐D) images with their onboard cameras. The set consists of 14 logs of experiments that were obtained in more than 10 different days and in four different locations. The complete length of the experiments in the data set exceeds 5 km. In addition, we provide the users with a partial ground‐truth and baselines of the localization of the platforms, which can be used for testing their localization and simultaneous localization and mapping (SLAM) algorithms. We also provide details on the setup and execution of each mission and a partial labeling of the elements found in the sewers. All the data were recorded by using the rosbag tool from robot operating system framework. Our goal is to make the data available to the scientific community as a benchmark to test localization, SLAM and classification algorithms in underground environments. The data set are available at https://robotics.upo.es/datasets/echord .     

An adaptive neural network approach to the tracking control of micro aerial vehicles in constrained space

This paper presents an adaptive neural network approach to the trajectory tracking control of micro aerial vehicles especially when they are flying in a limited indoor area. Differing from conventional controllers, the proposed controller employs the outer position loop to directly generate angular velocity commands in the presence of unknown aerodynamics and disturbances and then the fast inner loop to handle the angular rate control. Adaptive neural networks are deployed to estimate all the uncertain factors with the adaptation law derived from the Lyapunov function. To achieve a real-time performance, a norm estimation approach of ideal weights is designed to achieve a high bandwidth and lighten the burden of computation burden. Meanwhile, a barrier Lyapunov function is introduced to guarantee the constraint of vehicle positions as well as the validity of the neural network estimation. Simulations and practical flight tests are conducted to verify the feasibility and effectiveness of the proposed control strategy.     

Autonomous aerial robot using dual‐fisheye cameras

Safety is undoubtedly the most fundamental requirement for any aerial robotic application. It is essential to equip aerial robots with omnidirectional perception coverage to ensure safe navigation in complex environments. In this paper, we present a light‐weight and low‐cost omnidirectional perception system, which consists of two ultrawide field‐of‐view (FOV) fisheye cameras and a low‐cost inertial measurement unit (IMU). The goal of the system is to achieve spherical omnidirectional sensing coverage with the minimum sensor suite. The two fisheye cameras are mounted rigidly facing upward and downward directions and provide omnidirectional perception coverage: 360° FOV horizontally, 50° FOV vertically for stereo, and whole spherical for monocular. We present a novel optimization‐based dual‐fisheye visual‐inertial state estimator to provide highly accurate state‐estimation. Real‐time omnidirectional three‐dimensional (3D) mapping is combined with stereo‐based depth perception for the horizontal direction and monocular depth perception for upward and downward directions. The omnidirectional perception system is integrated with online trajectory planners to achieve closed‐loop, fully autonomous navigation. All computations are done onboard on a heterogeneous computing suite. Extensive experimental results are presented to validate individual modules as well as the overall system in both indoor and outdoor environments.     

Autonomous landing on a moving vehicle with an unmanned aerial vehicle

This paper addresses the perception, control, and trajectory planning for an aerial platform to identify and land on a moving car at 15 km/hr. The hexacopter unmanned aerial vehicle (UAV), equipped with onboard sensors and a computer, detects the car using a monocular camera and predicts the car future movement using a nonlinear motion model. While following the car, the UAV lands on its roof, and it attaches itself using magnetic legs. The proposed system is fully autonomous from takeoff to landing. Numerous field tests were conducted throughout the year‐long development and preparations for the Mohamed Bin Zayed International Robotics Challenge (MBZIRC) 2017 competition, for which the system was designed. We propose a novel control system in which a model predictive controller is used in real time to generate a reference trajectory for the UAV, which are then tracked by the nonlinear feedback controller. This combination allows to track predictions of the car motion with minimal position error. The evaluation presents three successful autonomous landings during the MBZIRC 2017, where our system achieved the fastest landing among all competing teams.     

无人机非确定性等价自适应容错飞行控制

针对存在执行器复合故障的固定翼无人机跟踪控制问题,本文提出一种基于非确定性等价原理的自适应容错飞行控制策略.该策略能够有效地估计无人机纵向动态中执行器的失效及漂移故障,保证故障发生后闭环系统的最优性能指标.在自适应容错飞行控制设计中,通过引入辅助系统并动态调节因子,构造非确定性等价原理中偏微分方程的近似解,以简化自适应律设计复杂度.此外,借助Lyapunov稳定性分析方法,证明了在所设计的自适应容错控制器作用下闭环系统的稳定性.最后,仿真验证表明所设计的控制方法能够保证故障无人机的闭环系统性能.     

Bioinspired design of a landing system with soft shock absorbers for autonomous aerial robots

One of the main challenges for autonomous aerial robots is to land safely on a target position on varied surface structures in real‐world applications. Most of current aerial robots (especially multirotors) use only rigid landing gears, which limit the adaptability to environments and can cause damage to the sensitive cameras and other electronics onboard. This paper presents a bioinpsired landing system for autonomous aerial robots, built on the inspire–abstract–implement design paradigm and an additive manufacturing process for soft thermoplastic materials. This novel landing system consists of 3D printable Sarrus shock absorbers and soft landing pads which are integrated with an one‐degree‐of‐freedom actuation mechanism. Both designs of the Sarrus shock absorber and the soft landing pad are analyzed via finite element analysis, and are characterized with dynamic mechanical measurements. The landing system with 3D printed soft components is characterized by completing landing tests on flat, convex, and concave steel structures and grassy field in a total of 60 times at different speeds between 1 and 2 m/s. The adaptability and shock absorption capacity of the proposed landing system is then evaluated and benchmarked against rigid legs. It reveals that the system is able to adapt to varied surface structures and reduce impact force by 540N at maximum. The bioinspired landing strategy presented in this paper opens a promising avenue in Aerial Biorobotics, where a cross‐disciplinary approach in vehicle control and navigation is combined with soft technologies, enabled with adaptive morphology.     

Cooperative digital magnetic‐elevation maps by small autonomous aerial robots

One of the steps to provide fundamental data for planning a mining effort is the magnetic surveying of a target area, which is typically carried out by conventional aircraft campaigns. However, besides the high cost, fixed‐wing aerial vehicles present shortcomings especially for drape flights on mountainous regions, where steep slopes are often present. Traditional human‐crewed flights have to perform tedious and dangerous trajectories, under strict velocity and attitude constraints. In this paper, we deal with the problem of accomplishing digital magnetic‐elevation maps using autonomous and cooperative aerial robots. The proposed approach for autonomous mapping utilizes a custom‐built fluxgate sensor and off the shelf cameras adapted for small airborne platforms. We also propose an innovative approach for generating a digital magnetic‐elevation model from the gathered data. Our method was evaluated and validated in field tests in an industrial scenario to detect scrap metals in ore piles. Results show that the proposed method could reliably detect magnetic anomalies while generating accurate three‐dimensional magnetic maps.     

无人飞行器的北斗卫星组合导航算法研究

北斗卫星导航系统是中国自行研制的全球卫星定位与通信系统,为促进北斗卫星系统在导航方面的运用,降低我国导航方面对GPS的依靠,从而进一步降低成本,提高无人飞行器的导航精度,本文对捷联式惯性导航系统（INS）和北斗定位导航系统的组合导航算法进行了研究,通过惯性导航系统的原理和导航解算的过程,选择惯导系统和北斗系统的速度、位置差值作为观测量,建立组合导航系统的状态方程和观测方程,利用无迹卡尔曼滤波得到惯导系统状态量和惯性敏感器的误差,对惯导系统进行误差补偿,从而实现无人飞行器的高精度导航控制.并使用Matlab进行仿真,得出高精度的模拟输出轨迹.     

Antenna array synthesis with clusters of unmanned aerial vehicles

In this paper, we investigate the use of formations of Unmanned Aerial Vehicles (UAVs) as phased antenna arrays. This will help to improve communications with clusters of small unmanned aerial vehicles which are currently constrained by on-board power limitations. The problem of maximizing the power output from the array in the direction of the receiver is posed as an optimization problem which happens to be non-convex; a relaxation of this problem is then solved as a computationally tractable (convex) Second-Order Conic Program (SOCP). The performance obtained by the simplified approach is then tested against rigorous numerical bounds obtained using Semidefinite Programming (SDP) duality theory; these bounds are of independent interest in antenna theory. In order to maintain the objective value close to the optimal when the vehicles deviate from their positions (due to wind gusts, for example), a simple linear control law is proposed. Simulation results are given to show the effectiveness of the proposed approaches.