Sampling‐based hierarchical motion planning for a reconfigurable wheel‐on‐leg planetary analogue exploration rover

Reconfigurable mobile planetary rovers are versatile platforms that may safely traverse cluttered environments by morphing their physical geometry. Planning paths for these adaptive robots is challenging due to their many degrees of freedom, and the need to consider potentially continuous platform reconfiguration along the length of the path. We propose a novel hierarchical structure for asymptotically optimal (AO) sampling‐based planners and specifically apply it to the state‐of‐the‐art Fast Marching Tree (FMT*) AO planner. Our algorithm assumes a decomposition of the full configuration space into multiple subspaces, and begins by rapidly finding a set of paths through one such subspace. This set of solutions is used to generate a biased sampling distribution, which is then explored to find a solution in the full configuration space. This technique provides a novel way to incorporate prior knowledge of subspaces to efficiently bias search within existing AO sampling‐based planners. Importantly, probabilistic completeness and asymptotic optimality are preserved. Experimental results in simulation are provided that benchmark the algorithm against state‐of‐the‐art sampling‐based planners without the hierarchical variation. Additional experimental results performed with a physical wheel‐on‐leg platform demonstrate application to planetary rover mobility and showcase how constraints such as actuator failures and sensor pointing may be easily incorporated into the planning problem. In minimizing an energy objective that combines an approximation of the mechanical work required for platform locomotion with that required for reconfiguration, the planner produces intuitive behaviors where the robot dynamically adjusts its footprint, varies its height, and clambers over obstacles using legged locomotion. These results illustrate the generality of the planner in exploiting the platform's mechanical ability to fluidly transition between various physical geometric configurations, and wheeled/legged locomotion modes, without the need for predefined configurations.     

Adaptive localization and mapping with application to planetary rovers

Future exploration rovers will be equipped with substantial onboard autonomy. SLAM is a fundamental part and has a close connection with robot perception, planning, and control. The community has made great progress in the past decade by enabling real‐world solutions and is addressing important challenges in high‐level scalability, resources awareness, and domain adaptation. A novel adaptive SLAM system is proposed to accomplish rover navigation and computational demands. It starts from a three‐dimensional odometry dead reckoning solution and builds up to a full graph optimization that takes into account rover traction performance. A complete kinematics of the rover locomotion system improves the wheel odometry solution. In addition, an odometry error model is inferred using Gaussian processes (GPs) to predict nonsystematic errors induced by poor traction of the rover with the terrain. The nonparametric GP regression serves to adapt the localization and mapping to the current navigation demands (domain adaptation). The method brings scalability and adaptiveness to modern SLAM. Therefore, an adaptive strategy develops to adjust the image frame rate (active perception) and to influence the optimization backend by including high informative keyframes in the graph (adaptive information gain). The work is experimentally verified on a representative planetary rover under a realistic field test scenario. The results show a modern SLAM systems that adapt to the predicted error. The system maintains accuracy with less number of nodes taking the most benefit of both wheel and visual methods in a consistent graph‐based smoothing approach.     

The ARTEMIS under‐ice AUV docking system

The ARTEMIS docking system demonstrates autonomous docking capability applicable to robotic exploration of sub‐ice oceans and sub‐glacial lakes on planetary bodies, as well as here on Earth. In these applications, melted or drilled vertical access shafts restrict vehicle geometry as well as the in‐water infrastructure that may be deployed. The ability of the vehicle to return reliably and precisely to the access point is critical for data return, battery charging, and/or vehicle recovery. This paper presents the mechanical, sensor, and software components that make up the ARTEMIS docking system, as well as results from field deployment of the system to McMurdo Sound, Antarctica in the austral spring of 2015. The mechanical design of the system allows the vehicle to approach the dock from any direction and to pitch up after docking for recovery through a vertical access shaft. It uses only a small volume of in‐water equipment and may be deployed through a narrow vertical access shaft. The software of the system reduces position estimation error with a hierarchical combination of dead reckoning, acoustic aiding, and machine vision. The system provides critical operational robustness, enabling the vehicle to return autonomously and precisely to the access shaft and latch to the dock with no operator input.     

High‐speed mobility on planetary surfaces: A technical review

Despite the success so far accomplished in the robotic exploration of the Moon and Mars, the constraints associated with newly proposed mission concepts manifest the need for a faster surface prospection. Increasing driving velocities is being considered as a potential solution to the requirements introduced by these missions. This review presents the benefits and foreseeable challenges of using faster locomotive solutions for space exploration. Information is provided regarding the set of missions that would benefit most from faster locomotive capabilities. Starting by understanding the theoretical framework governing the interaction of wheeled robots operating over loose, sandy terrains, we delve into the foundation of Bekker's classic terramechanic equations—the most frequently used method to predict mobile robots off‐road performance. We highlight its limitations and review the efforts that have been made to expand the range of application of these theories to dynamic wheel–soil interactions. We analyze the existing experimental evidence on the effects of increasing traveling velocities under earthbound, off‐road conditions. By paying special attention to previous experiences on the lunar surface, we outline the challenges that the combination of irregular terrains and a reduced‐gravity field may pose to a fast‐moving exploration rover. The principles, mathematical models, experimental evidence, and experiences presented in this review are meant to aid in the identification of poorly understood and insufficiently studied aspects regarding high‐speed extraterrestrial surface mobility.     

Self‐reliant rovers for increased mission productivity

Achieving consistently high levels of productivity has been a challenge for Mars surface missions. While the rovers have made major discoveries and dramatically increased our understanding of Mars, they require a great deal of interaction from the operations teams, and achieving mission objectives can take longer than anticipated when productivity is paced by the ground teams' ability to react. We have conducted a project to explore technologies and techniques for creating self‐reliant rovers (SRR): rovers that are able to maintain high levels of productivity with reduced reliance on ground interactions. This paper describes the design of SRR and a prototype implementation that we deployed on a research rover. We evaluated the system by conducting a simulated campaign in which members of the Mars Science Laboratory (Curiosity rover) science team used our rover to explore a geographical region. The evaluation demonstrated the system's ability to maintain high levels of productivity with limited communication with operators.     

Cataglyphis: An autonomous sample return rover

This paper presents the design of Cataglyphis, a research rover that won the NASA Sample Return Robot Centennial Challenge in 2015. During the challenge, Cataglyphis was the only robot that was able to autonomously find, retrieve, and return multiple types of samples in a large natural environment without using Earth‐specific sensors such as GPS and magnetic compasses. It navigates through a fusion of measurements collected from inertial sensors, wheel encoders, a nodding Lidar, a set of ranging radios, a camera on a panning platform, and a sun sensor. In addition to visual detection of a homing beacon, computer vision algorithms provide the sample detection, identification, and localization capabilities, with low false positive and false negative rates demonstrated during the competition. The mission planning and control software enables robot behaviors, determines sequences of actions, and helps the robot to recover from various failure conditions. A compliant, under‐actuated manipulator conforms to the natural terrain before picking up samples of various size, weight, and shape.     

Data‐driven mobility risk prediction for planetary rovers

Mobility assessment and prediction continues to be an important and active area of research for planetary rovers, with the need illustrated by multiple examples of high slip events experienced by rovers on Mars. Despite slip versus slope being one of the strongest and most broadly used relationships in mobility prediction, this relationship is nonetheless far from precisely predictable. Although the literature has made significant advances in the predictability of average mobility, the other key related aspect of the problem is the risk caused by edge cases. A key contribution of this study is a metric for explicitly assessing mobility risk based on data‐driven nonparametric slip versus slope relationships. The data‐driven approach is meant to address limitations of past model‐based approaches. The metric is informed by past work in terramechanics relating drawbar pull (i.e., net traction) to slip: High slip fraction (HSF), defined as the proportion of slip data points above 20%. Another contribution is a low complexity mobility prediction framework, the autonomous soil assessment system. Field tests demonstrate that, for sand and gravel, rover trafficability becomes nonlinear and highly variable above the 20% slip threshold. HSF is shown to be a useful metric for categorizing rover‐terrain interactions into low, medium, or high risk, correctly and consistently. Furthermore, the metric is shown to be useful for early detection of potentially hazardous changes in rover‐terrain conditions. The combination of HSF with an appropriately sized queue structure for modeling slip versus slope enables an appropriate balance between responsiveness and stability.     

Terrain‐adaptive wheel speed control on the Curiosity Mars rover: Algorithm and flight results

NASA's Mars Science Laboratory Curiosity rover landed in August 2012 and began experiencing higher rates of wheel damage beginning in October 2013. While the wheels were designed to accumulate considerable damage, the unexpected damage rate raised concerns regarding wheel lifetime. In response, the Jet Propulsion Laboratory developed and deployed mobility flight software on Curiosity that reduces the forces on the wheels. The new algorithm adapts each wheel's speed to fit the terrain topography in real time, by leveraging the rover's measured attitude rates and rocker/bogie suspension angles and rates. Together with a rigid‐body kinematics model, it estimates the real‐time wheel‐terrain contact angles and commands idealized, no‐slip wheel angular rates. In addition, free‐floating “wheelies” are detected and autonomously corrected. Ground test data indicate that the forces on the wheels are reduced by 19% for leading wheels and 11% for middle leading wheels. On the ground, the required data volume increased by up to 129%, and drive duration increased by up to 25%. In flight, data collected over 3.6 km and 149 drives confirmed a reduction in wheel current, correlated with wheel torque, of 18.7%. The new algorithm proved to use fewer resources in flight than ground estimates suggested, as only a 10% increase in drive duration and double the drive data volume were experienced. These data indicate the promise of the new algorithm to extend the life of the wheels for the Curiosity rover. This paper describes the algorithm, its ground testing campaign and associated challenges, and its validation, implementation, and performance in flight.     

Hierarchical fiducial marker design for pose estimation in large‐scale scenarios

In this paper, a hierarchical fiducial marker, called HArCo, is designed to guarantee a smooth pose estimation for large‐scale applications. HArCo markers have a visually identifiable structure on multiple scales, so they can be used for consistent pose estimation across a range of altitudes. Experimental results are presented to validate the performance of the proposed methodologies; oscillating platform landing experiments were conducted to show the ability of HArCo to be used in real landing tasks on the deck of a ship.     

ARDEA—An MAV with skills for future planetary missions

We introduce a prototype flying platform for planetary exploration: autonomous robot design for extraterrestrial applications (ARDEA). Communication with unmanned missions beyond Earth orbit suffers from time delay, thus a key criterion for robotic exploration is a robot's ability to perform tasks without human intervention. For autonomous operation, all computations should be done on‐board and Global Navigation Satellite System (GNSS) should not be relied on for navigation purposes. Given these objectives ARDEA is equipped with two pairs of wide‐angle stereo cameras and an inertial measurement unit (IMU) for robust visual‐inertial navigation and time‐efficient, omni‐directional 3D mapping. The four cameras cover a $240^{°}$ vertical field of view, enabling the system to operate in confined environments such as caves formed by lava tubes. The captured images are split into several pinhole cameras, which are used for simultaneously running visual odometries. The stereo output is used for simultaneous localization and mapping, 3D map generation and collision‐free motion planning. To operate the vehicle efficiently for a variety of missions, ARDEA's capabilities have been modularized into skills which can be assembled to fulfill a mission's objectives. These skills are defined generically so that they are independent of the robot configuration, making the approach suitable for different heterogeneous robotic teams. The diverse skill set also makes the micro aerial vehicle (MAV) useful for any task where autonomous exploration is needed. For example terrestrial search and rescue missions where visual navigation in GNSS‐denied indoor environments is crucial, such as partially collapsed man‐made structures like buildings or tunnels. We have demonstrated the robustness of our system in indoor and outdoor field tests.     

Introducing a globally consistent orbital‐based localization system

In spite of the good performance of space exploratory missions, open issues still await to be solved. In autonomous or composite semi‐autonomous exploration of planetary land surfaces, rover localization is such an issue. The rovers of these missions (e.g., the MER and MSL) navigate relatively to their landing spot, ignoring their exact position on the coordinate system defined for the celestial body they explore. However, future advanced missions, like the Mars Sample Return, will require the localization of rovers on a global frame rather than the arbitrarily defined landing frame. In this paper we attempt to retrieve the absolute rover's location by identifying matching Regions of Interest (ROIs) between orbital and land images. In particular, we propose a system comprising two parts, an offline and an onboard one, which functions as follows: in advance of the mission a Global ROI Network (GN) is built offline by investigating the satellite images near the predicted touchdown ellipse, while during the mission a Local ROI Network (LN) is constructed counting on the images acquired by the vision system of the rover along its traverse. The last procedure relies on the accurate VO‐based relative rover localization. The LN is then paired with the GN through a modified 2D DARCES algorithm. The system has been assessed on real data collected by the ESA at the Atacama desert. The results demonstrate the system's potential to perform absolute localization, on condition that the area includes discriminative ROIs. The main contribution of this work is the enablement of global localization performed on contemporary rovers without requiring any additional hardware, such as long range LIDARs.     

Underwater place recognition using forward‐looking sonar images: A topological approach

Autonomous underwater vehicles are a prominent tool for underwater exploration because they can access dangerous places avoiding the risks for the human beings. However, the autonomous navigation still a challenge due to the characteristics of the environment that decrease the performance of the sensor and the robot perception. In this context, this paper proposes a loop closure detector addressed to the simultaneous localization and mapping problem at semistructured environments using acoustic images acquired by forward‐looking sonars. The images are segmented by an adaptative approach based on the acoustic beams analysis. A pose‐invariant topological graph is build to represent the relationship between image features. The loop closure detection is achieved using a graph comparison. The approach is evaluated in a real environment at a marina. The results reveal all loop closures of the data set are detected with a high precision and present an invariant to image rotation.     

Long‐duration fully autonomous operation of rotorcraft unmanned aerial systems for remote‐sensing data acquisition

Recent applications of unmanned aerial systems (UAS) to precision agriculture have shown increased ease and efficiency in data collection at precise remote locations. However, further enhancement of the field requires operation over long periods of time, for example, days or weeks. This has so far been impractical due to the limited flight times of such platforms and the requirement of humans in the loop for operation. To overcome these limitations, we propose a fully autonomous rotorcraft UAS that is capable of performing repeated flights for long‐term observation missions without any human intervention. We address two key technologies that are critical for such a system: full platform autonomy to enable mission execution independently from human operators and the ability of vision‐based precision landing on a recharging station for automated energy replenishment. High‐level autonomous decision making is implemented as a hierarchy of master and slave state machines. Vision‐based precision landing is enabled by estimating the landing pad's pose using a bundle of AprilTag fiducials configured for detection from a wide range of altitudes. We provide an extensive evaluation of the landing pad pose estimation accuracy as a function of the bundle's geometry. The functionality of the complete system is demonstrated through two indoor experiments with duration of 11 and 10.6 hr, and one outdoor experiment with a duration of 4 hr. The UAS executed 16, 48, and 22 flights, respectively, during these experiments. In the outdoor experiment, the ratio between flying to collect data and charging was 1–10, which is similar to past work in this domain. All flights were fully autonomous with no human in the loop. To our best knowledge, this is the first research publication about the long‐term outdoor operation of a quadrotor system with no human interaction.     

Registration of three‐dimensional scanning LiDAR sensors: An evaluation of model‐based and model‐free methods

Registration, also know as extrinsic calibration, is the process of determining the position and orientation of a sensor relative to a known frame of reference. For ranging sensors such as light detection and ranging (LiDAR) used in field robotic applications, the quality of the registration determines the utility of the range measurements. This paper makes two contributions. The first is the introduction of a new method, termed maximum sum of evidence (MSoE) for registering three‐dimensional LiDAR sensors to moving platforms. This method is shown to produce more accurate registration solutions than two leading methods for these sensors, the adaptive structure registration filter (ASRF) and Rényi quadratic entropy (RQE). The second contribution of the paper is to study the accuracy of the MSoE registration against these two other approaches. One of these, like the MSoE, requires a truth model of the environment. The other, a model‐free method, seeks the registration that minimizes the RQE of a compound point cloud. The main finding of this investigation is that while the model‐based methods prove more accurate than the model‐free approach, the results of all three methods are fit for their intended field robotic applications. This leads us to conclude that registration based on RQE is preferable in many, if not all, field robotic applications for reasons of convenience, since a truth model of the environment is not required.     

Experimental analysis of low‐altitude terrain following for hover‐capable flight‐style autonomous underwater vehicles

Operating an autonomous underwater vehicle (AUV) in close proximity to terrain typically relies solely on the vehicle sensors for terrain detection, and challenges the manoeuvrability of energy efficient flight‐style AUVs. This paper gives new results on altitude tracking limits of such vehicles by using the fully understood environment of a lake to perform repeated experiments while varying the altitude demand, obstacle detection and actuator use of a hover‐capable flight‐style AUV. The results are analysed for mission success, vehicle risk and repeatability, demonstrating the terrain following capabilities of the overactuated AUV over a range of altitude tracking strategies and how these measures better inform vehicle operators. A major conclusion is that the effects of range limits, bias and false detections of the sensors used for altitude tracking must be fully accounted for to enable mission success. Furthermore it was found that switching between hover‐ and flight‐style actuations based on speed, whilst varying the operation speed, has advantages for performance improvement over combining hover‐ and flight‐style actuators at high speeds.     

Remote mobile manipulation with the centauro robot: Full‐body telepresence and autonomous operator assistance

Solving mobile manipulation tasks in inaccessible and dangerous environments is an important application of robots to support humans. Example domains are construction and maintenance of manned and unmanned stations on the moon and other planets. Suitable platforms require flexible and robust hardware, a locomotion approach that allows for navigating a wide variety of terrains, dexterous manipulation capabilities, and respective user interfaces. We present the CENTAURO system which has been designed for these requirements and consists of the Centauro robot and a set of advanced operator interfaces with complementary strength enabling the system to solve a wide range of realistic mobile manipulation tasks. The robot possesses a centaur‐like body plan and is driven by torque‐controlled compliant actuators. Four articulated legs ending in steerable wheels allow for omnidirectional driving as well as for making steps. An anthropomorphic upper body with two arms ending in five‐finger hands enables human‐like manipulation. The robot perceives its environment through a suite of multimodal sensors. The resulting platform complexity goes beyond the complexity of most known systems which puts the focus on a suitable operator interface. An operator can control the robot through a telepresence suit, which allows for flexibly solving a large variety of mobile manipulation tasks. Locomotion and manipulation functionalities on different levels of autonomy support the operation. The proposed user interfaces enable solving a wide variety of tasks without previous task‐specific training. The integrated system is evaluated in numerous teleoperated experiments that are described along with lessons learned.     

Vision techniques for on‐board detection,following, and mapping of moving targets

This study presents computer vision modules of a multi‐unmanned aerial vehicle (UAV) system, which scored gold, silver, and bronze medals at the Mohamed Bin Zayed International Robotics Challenge 2017. This autonomous system, which was running completely on board and in real time, had to address two complex tasks in challenging outdoor conditions. In the first task, an autonomous UAV had to find, track, and land on a human‐driven car moving at 15 km/hr on a figure‐eight‐shaped track. During the second task, a group of three UAVs had to find small colored objects in a wide area, pick them up, and deliver them into a specified drop‐off zone. The computer vision modules presented here achieved computationally efficient detection, accurate localization, robust velocity estimation, and reliable future position prediction of both the colored objects and the car. These properties had to be achieved in adverse outdoor environments with changing light conditions. Lighting varied from intense direct sunlight with sharp shadows cast over the objects by the UAV itself, to reduced visibility caused by overcast to dust and sand in the air. The results presented in this paper demonstrate good performance of the modules both during testing, which took place in the harsh desert environment of the central area of United Arab Emirates, as well as during the contest, which took place at a racing complex in the urban, near‐sea location of Abu Dhabi. The stability and reliability of these modules contributed to the overall result of the contest, where our multi‐UAV system outperformed teams from world’s leading robotic laboratories in two challenging scenarios.     

Perception‐aware autonomous mast motion planning for planetary exploration rovers

Highly accurate real‐time localization is of fundamental importance for the safety and efficiency of planetary rovers exploring the surface of Mars. Mars rover operations rely on vision‐based systems to avoid hazards as well as plan safe routes. However, vision‐based systems operate on the assumption that sufficient visual texture is visible in the scene. This poses a challenge for vision‐based navigation on Mars where regions lacking visual texture are prevalent. To overcome this, we make use of the ability of the rover to actively steer the visual sensor to improve fault tolerance and maximize the perception performance. This paper answers the question of where and when to look by presenting a method for predicting the sensor trajectory that maximizes the localization performance of the rover. This is accomplished by an online assessment of possible trajectories using synthetic, future camera views created from previous observations of the scene. The proposed trajectories are quantified and chosen based on the expected localization performance. In this study, we validate the proposed method in field experiments at the Jet Propulsion Laboratory (JPL) Mars Yard. Furthermore, multiple performance metrics are identified and evaluated for reducing the overall runtime of the algorithm. We show how actively steering the perception system increases the localization accuracy compared with traditional fixed‐sensor configurations.     

RTAB‐Map as an open‐source lidar and visual simultaneous localization and mapping library for large‐scale and long‐term online operation

Distributed as an open‐source library since 2013, real‐time appearance‐based mapping (RTAB‐Map) started as an appearance‐based loop closure detection approach with memory management to deal with large‐scale and long‐term online operation. It then grew to implement simultaneous localization and mapping (SLAM) on various robots and mobile platforms. As each application brings its own set of constraints on sensors, processing capabilities, and locomotion, it raises the question of which SLAM approach is the most appropriate to use in terms of cost, accuracy, computation power, and ease of integration. Since most of SLAM approaches are either visual‐ or lidar‐based, comparison is difficult. Therefore, we decided to extend RTAB‐Map to support both visual and lidar SLAM, providing in one package a tool allowing users to implement and compare a variety of 3D and 2D solutions for a wide range of applications with different robots and sensors. This paper presents this extended version of RTAB‐Map and its use in comparing, both quantitatively and qualitatively, a large selection of popular real‐world datasets (e.g., KITTI, EuRoC, TUM RGB‐D, MIT Stata Center on PR2 robot), outlining strengths, and limitations of visual and lidar SLAM configurations from a practical perspective for autonomous navigation applications.     

Kriging‐based robotic exploration for soil moisture mapping using a cosmic‐ray sensor

Soil moisture monitoring is a fundamental process to enhance agricultural outcomes and to protect the environment. The traditional methods for measuring moisture content in the soil are laborious and expensive, and therefore there is a growing interest in developing sensors and technologies which can reduce the effort and costs. In this work, we propose to use an autonomous mobile robot equipped with a state‐of‐the‐art noncontact soil moisture sensor building moisture maps on the fly and automatically selecting the most optimal sampling locations. We introduce an autonomous exploration strategy driven by the quality of the soil moisture model indicating areas of the field where the information is less precise. The sensor model follows the Poisson distribution and we demonstrate how to integrate such measurements into the kriging framework. We also investigate a range of different exploration strategies and assess their usefulness through a set of evaluation experiments based on real soil moisture data collected from two different fields. We demonstrate the benefits of using the adaptive measurement interval and adaptive sampling strategies for building better quality soil moisture models. The presented method is general and can be applied to other scenarios where the measured phenomena directly affect the acquisition time and need to be spatially mapped.