Modeling Regions of Interest on Orbital and Rover Imagery for Planetary Exploration Missions

Planetary rover exploration missions require accurate and computationally efficient robot localization in order to perform complex and cooperative tasks. The global localization on planetary environments can be competently addressed by incorporating orbital and ground rover imagery. An indicative approach could include (1) the extraction of regions of interest (ROIs) in orbital images, (2) the extraction of ROIs in rover images, (3) the ROI matching, and (4) the localization. In order to perform adequately in ROI matching, a model should be able to detect common ROIs. The work in hand tackles the problem of extracting such regions of interest that are observable on both orbital and rover images. The dedicated model that was designed and implemented contains a detection and a classification part. The detection of the ROIs is based on both their texture and their geometrical properties. Classification was performed on the result of the detection in order to annotate the ROIs and discard any outliers caused by false detection. The results prove that the model is able to detect commonly observable regions and, therefore, is considered to be an adequate preprocessing step in the context of a global rover localization system.     

Self‐reliant rovers for increased mission productivity

Achieving consistently high levels of productivity has been a challenge for Mars surface missions. While the rovers have made major discoveries and dramatically increased our understanding of Mars, they require a great deal of interaction from the operations teams, and achieving mission objectives can take longer than anticipated when productivity is paced by the ground teams' ability to react. We have conducted a project to explore technologies and techniques for creating self‐reliant rovers (SRR): rovers that are able to maintain high levels of productivity with reduced reliance on ground interactions. This paper describes the design of SRR and a prototype implementation that we deployed on a research rover. We evaluated the system by conducting a simulated campaign in which members of the Mars Science Laboratory (Curiosity rover) science team used our rover to explore a geographical region. The evaluation demonstrated the system's ability to maintain high levels of productivity with limited communication with operators.     

Improved Rover State Estimation in Challenging Terrain

Given ambitious mission objectives and long delay times between command-uplink/data-downlink sessions, increased autonomy is required for planetary rovers. Specifically, NASA's planned 2003 and 2005 Mars rover missions must incorporate increased autonomy if their desired mission goals are to be realized. Increased autonomy, including autonomous path planning and navigation to user designated goals, relies on good quality estimates of the rover's state, e.g., its position and orientation relative to some initial reference frame. The challenging terrain over which the rover will necessarily traverse tends to seriously degrade a dead-reckoned state estimate, given severe wheel slip and/or interaction with obstacles. In this paper, we present the implementation of a complete rover navigation system. First, the system is able to adaptively construct semi-sparse terrain maps based on the current ground texture and distances to possible nearby obstacles. Second, the rover is able to match successively constructed terrain maps to obtain a vision-based state estimate which can then be fused with wheel odometry to obtain a much improved state estimate. Finally the rover makes use of this state estimate to perform autonomous real-time path planning and navigation to user designated goals. Reactive obstacle avoidance is also implemented for roaming in an environment in the absence of a user designated goal. The system is demonstrated in soft soil and relatively dense rock fields, achieving state estimates that are significantly improved with respect to dead reckoning alone (e.g., 0.38 m mean absolute error vs. 1.34 m), and successfully navigating in multiple trials to user designated goals.     

Efficient autonomous navigation for planetary rovers with limited resources

Rovers operating on Mars require more and more autonomous features to fulfill their challenging mission requirements. However, the inherent constraints of space systems render the implementation of complex algorithms an expensive and difficult task. In this paper, we propose an architecture for autonomous navigation. Efficient implementations of autonomous features are built on top of the ExoMars path following navigation approach to enhance the safety and traversing capabilities of the rover. These features allow the rover to detect and avoid hazards and perform significantly longer traverses planned by operators on the ground. The efficient navigation approach has been implemented and tested during field test campaigns on a planetary analogue terrain. The experiments evaluated the proposed architecture by autonomously completing several traverses of variable lengths while avoiding hazards. The approach relies only on the optical Localization Cameras stereo bench, a sensor that is found in all current rovers, and potentially allows for computationally inexpensive long‐range autonomous navigation in terrains of medium difficulty.     

Adaptive localization and mapping with application to planetary rovers

Future exploration rovers will be equipped with substantial onboard autonomy. SLAM is a fundamental part and has a close connection with robot perception, planning, and control. The community has made great progress in the past decade by enabling real‐world solutions and is addressing important challenges in high‐level scalability, resources awareness, and domain adaptation. A novel adaptive SLAM system is proposed to accomplish rover navigation and computational demands. It starts from a three‐dimensional odometry dead reckoning solution and builds up to a full graph optimization that takes into account rover traction performance. A complete kinematics of the rover locomotion system improves the wheel odometry solution. In addition, an odometry error model is inferred using Gaussian processes (GPs) to predict nonsystematic errors induced by poor traction of the rover with the terrain. The nonparametric GP regression serves to adapt the localization and mapping to the current navigation demands (domain adaptation). The method brings scalability and adaptiveness to modern SLAM. Therefore, an adaptive strategy develops to adjust the image frame rate (active perception) and to influence the optimization backend by including high informative keyframes in the graph (adaptive information gain). The work is experimentally verified on a representative planetary rover under a realistic field test scenario. The results show a modern SLAM systems that adapt to the predicted error. The system maintains accuracy with less number of nodes taking the most benefit of both wheel and visual methods in a consistent graph‐based smoothing approach.     

Computer Vision on Mars

Increasing the level of spacecraft autonomy is essential for broadening the reach of solar system exploration. Computer vision has and will continue to play an important role in increasing autonomy of both spacecraft and Earth-based robotic vehicles. This article addresses progress on computer vision for planetary rovers and landers and has four main parts. First, we review major milestones in the development of computer vision for robotic vehicles over the last four decades. Since research on applications for Earth and space has often been closely intertwined, the review includes elements of both. Second, we summarize the design and performance of computer vision algorithms used on Mars in the NASA/JPL Mars Exploration Rover (MER) mission, which was a major step forward in the use of computer vision in space. These algorithms did stereo vision and visual odometry for rover navigation and feature tracking for horizontal velocity estimation for the landers. Third, we summarize ongoing research to improve vision systems for planetary rovers, which includes various aspects of noise reduction, FPGA implementation, and vision-based slip perception. Finally, we briefly survey other opportunities for computer vision to impact rovers, landers, and orbiters in future solar system exploration missions.     

Precise pose estimation of the NASA Mars 2020 Perseverance rover through a stereo-vision-based approach

Visual Odometry (VO) is a fundamental technique to enhance the navigation capabilities of planetary exploration rovers. By processing the images acquired during the motion, VO methods provide estimates of the relative position and attitude between navigation steps with the detection and tracking of two-dimensional (2D) image keypoints. This method allows one to mitigate trajectory inconsistencies associated with slippage conditions resulting from dead-reckoning techniques. We present here an independent analysis of the high-resolution stereo images of the NASA Mars 2020 Perseverance rover to retrieve its accurate localization on sols 65, 66, 72, and 120. The stereo pairs are processed by using a 3D-to-3D stereo-VO approach that is based on consolidated techniques and accounts for the main nonlinear optical effects characterizing real cameras. The algorithm is first validated through the analysis of rectified stereo images acquired by the NASA Mars Exploration Rover Opportunity, and then applied to the determination of Perseverance's path. The results suggest that our reconstructed path is consistent with the telemetered trajectory, which was directly retrieved onboard the rover's system. The estimated pose is in full agreement with the archived rover's position and attitude after short navigation steps. Significant differences (~10–30 cm) between our reconstructed and telemetered trajectories are observed when Perseverance traveled distances larger than 1 m between the acquisition of stereo pairs.     

The evolution of the curiosity rover sampling chain

The Mars Science Laboratory (MSL) Curiosity rover landed in Gale crater in August of 2012 on its mission to explore Mt. Sharp as the first planetary rover to collect and analyze rock and regolith samples. On this new mission, sampling operations were conceived to be executed serially and in situ, on a “sample chain” along which sample would be collected, then processed, then delivered to sample analysis instruments, analyzed there, and then discarded so the chain could be repeated. This paper describes the evolution of this relatively simple chain into a richer sampling network, responding to science and engineering desires that came into focus only as the mission matured, scientific discoveries were made, and anomalies were encountered. The rover flight and ground system architectures retained significant heritage from past missions, while extending capabilities in anticipation of the need for adaptation. As evolution occurred, the architecture permitted nimble extension of sampling behavior without time‐consuming flight software updates or significant impact to daily operations. This paper presents the major components of this architecture and discusses some of the results of successful adaptation across thousands of Sols of Mars operations.     

ARDEA—An MAV with skills for future planetary missions

We introduce a prototype flying platform for planetary exploration: autonomous robot design for extraterrestrial applications (ARDEA). Communication with unmanned missions beyond Earth orbit suffers from time delay, thus a key criterion for robotic exploration is a robot's ability to perform tasks without human intervention. For autonomous operation, all computations should be done on‐board and Global Navigation Satellite System (GNSS) should not be relied on for navigation purposes. Given these objectives ARDEA is equipped with two pairs of wide‐angle stereo cameras and an inertial measurement unit (IMU) for robust visual‐inertial navigation and time‐efficient, omni‐directional 3D mapping. The four cameras cover a $240^{°}$ vertical field of view, enabling the system to operate in confined environments such as caves formed by lava tubes. The captured images are split into several pinhole cameras, which are used for simultaneously running visual odometries. The stereo output is used for simultaneous localization and mapping, 3D map generation and collision‐free motion planning. To operate the vehicle efficiently for a variety of missions, ARDEA's capabilities have been modularized into skills which can be assembled to fulfill a mission's objectives. These skills are defined generically so that they are independent of the robot configuration, making the approach suitable for different heterogeneous robotic teams. The diverse skill set also makes the micro aerial vehicle (MAV) useful for any task where autonomous exploration is needed. For example terrestrial search and rescue missions where visual navigation in GNSS‐denied indoor environments is crucial, such as partially collapsed man‐made structures like buildings or tunnels. We have demonstrated the robustness of our system in indoor and outdoor field tests.     

Mars microrover navigation: Performance evaluation and enhancement

In 1996, NASA will launch the Mars Pathfinder spacecraft, which will carry an 11 kg rover to explore the immediate vicinity of the lander. To assess the capabilities of the rover, as well as to set priorities for future rover research, it is essential to evaluate the performance of its autonomous navigation system as a function of terrain characteristics. Unfortunately, very little of this kind of evaluation has been done, for either planetary rovers or terrestrial applications. To fill this gap, we have constructed a new microrover testbed consisting of the Rocky 3.2 vehicle and an indoor test arena with overhead cameras for automatic, real-time tracking of the true rover position and heading. We create Mars analog terrains in this arena by randomly distributing rocks according to an exponential model of Mars rock size frequency created from Viking lander imagery. To date, we have recorded detailed logs from over 85 navigation trials in this testbed. In this paper, we outline current plans for Mars exploration over the next decade, summarize the design of the lander and rover for the 1996 Pathfinder mission, and introduce a decomposition of rover navigation into four major functions: goal designation, rover localization, hazard detection, and path selection. We then describe the Pathfinder approach to each function, present results to date of evaluating the performance of each function, and outline our approach to enhancing performance for future missions. The results show key limitations in the quality of rover localization, the speed of hazard detection, and the ability of behavior control algorithms for path selection to negotiate the rock frequencies likely to be encountered on Mars. We believe that the facilities, methodologies, and to some extent the specific performance results presented here will provide valuable examples for efforts to evaluate robotic vehicle performance in other applications.     

Data‐driven mobility risk prediction for planetary rovers

Mobility assessment and prediction continues to be an important and active area of research for planetary rovers, with the need illustrated by multiple examples of high slip events experienced by rovers on Mars. Despite slip versus slope being one of the strongest and most broadly used relationships in mobility prediction, this relationship is nonetheless far from precisely predictable. Although the literature has made significant advances in the predictability of average mobility, the other key related aspect of the problem is the risk caused by edge cases. A key contribution of this study is a metric for explicitly assessing mobility risk based on data‐driven nonparametric slip versus slope relationships. The data‐driven approach is meant to address limitations of past model‐based approaches. The metric is informed by past work in terramechanics relating drawbar pull (i.e., net traction) to slip: High slip fraction (HSF), defined as the proportion of slip data points above 20%. Another contribution is a low complexity mobility prediction framework, the autonomous soil assessment system. Field tests demonstrate that, for sand and gravel, rover trafficability becomes nonlinear and highly variable above the 20% slip threshold. HSF is shown to be a useful metric for categorizing rover‐terrain interactions into low, medium, or high risk, correctly and consistently. Furthermore, the metric is shown to be useful for early detection of potentially hazardous changes in rover‐terrain conditions. The combination of HSF with an appropriately sized queue structure for modeling slip versus slope enables an appropriate balance between responsiveness and stability.     

月球巡视探测器定位技术研究

介绍了月球巡视探测器进行定位的目的及意义．通过对月球探测环境的综合分析，界定了月球环境下探测机器人进行导航定位的相应局限性．对当前国内国际可用的星际探测机器人的定位算法进行了研究，对其中的定位技术进行了归类总结，通过相关文献分析对其中关键算法的可用性、可靠性与自主性进行了评定．最后指出了目前月球巡视探测器定位算法研究中的难点及存在的问题．     

Perception‐aware autonomous mast motion planning for planetary exploration rovers

Highly accurate real‐time localization is of fundamental importance for the safety and efficiency of planetary rovers exploring the surface of Mars. Mars rover operations rely on vision‐based systems to avoid hazards as well as plan safe routes. However, vision‐based systems operate on the assumption that sufficient visual texture is visible in the scene. This poses a challenge for vision‐based navigation on Mars where regions lacking visual texture are prevalent. To overcome this, we make use of the ability of the rover to actively steer the visual sensor to improve fault tolerance and maximize the perception performance. This paper answers the question of where and when to look by presenting a method for predicting the sensor trajectory that maximizes the localization performance of the rover. This is accomplished by an online assessment of possible trajectories using synthetic, future camera views created from previous observations of the scene. The proposed trajectories are quantified and chosen based on the expected localization performance. In this study, we validate the proposed method in field experiments at the Jet Propulsion Laboratory (JPL) Mars Yard. Furthermore, multiple performance metrics are identified and evaluated for reducing the overall runtime of the algorithm. We show how actively steering the perception system increases the localization accuracy compared with traditional fixed‐sensor configurations.     

High‐accuracy absolute positioning for the stationary planetary rover by integrating the star sensor and inclinometer

In this paper, we introduce a novel method for the high‐accuracy absolute position determination for planetary rovers using the star sensor and inclinometer. We describe the star sensor and inclinometer model and the alignment method for the two sensors. We deduce the compensation algorithm for the atmosphere refraction correction error in detail and provide the rover's position solution, which effectively eliminates the tilt correction error. The experimental site and hardware configuration are introduced, and the experimental steps for the one‐time positioning are described. Three field tests on Earth indicate that the accuracy of the one‐time positioning is higher than 40 m (1σ) using 8 star images and relative inclinometer measurements. Multiple positionings in one night can improve the accuracy to approximately 15 m.     

Outdoor Visual Position Estimation for Planetary Rovers

This paper describes (1) a novel, effective algorithm for outdoor visual position estimation; (2) the implementation of this algorithm in the Viper system; and (3) the extensive tests that have demonstrated the superior accuracy and speed of the algorithm. The Viper system (Visual Position Estimator for Rovers) is geared towards robotic space missions, and the central purpose of the system is to increase the situational awareness of a rover operator by presenting accurate position estimates. The system has been extensively tested with terrestrial and lunar imagery, in terrains ranging from moderate—the rounded hills of Pittsburgh and the high deserts of Chile—to rugged—the dramatic relief of the Apollo 17 landing site—to extreme—the jagged peaks of the Rockies. Results have consistently demonstrated that the visual estimation algorithm estimates position with an accuracy and reliability that greatly surpass previous work.     

Cataglyphis: An autonomous sample return rover

This paper presents the design of Cataglyphis, a research rover that won the NASA Sample Return Robot Centennial Challenge in 2015. During the challenge, Cataglyphis was the only robot that was able to autonomously find, retrieve, and return multiple types of samples in a large natural environment without using Earth‐specific sensors such as GPS and magnetic compasses. It navigates through a fusion of measurements collected from inertial sensors, wheel encoders, a nodding Lidar, a set of ranging radios, a camera on a panning platform, and a sun sensor. In addition to visual detection of a homing beacon, computer vision algorithms provide the sample detection, identification, and localization capabilities, with low false positive and false negative rates demonstrated during the competition. The mission planning and control software enables robot behaviors, determines sequences of actions, and helps the robot to recover from various failure conditions. A compliant, under‐actuated manipulator conforms to the natural terrain before picking up samples of various size, weight, and shape.     

Long‐duration fully autonomous operation of rotorcraft unmanned aerial systems for remote‐sensing data acquisition

Recent applications of unmanned aerial systems (UAS) to precision agriculture have shown increased ease and efficiency in data collection at precise remote locations. However, further enhancement of the field requires operation over long periods of time, for example, days or weeks. This has so far been impractical due to the limited flight times of such platforms and the requirement of humans in the loop for operation. To overcome these limitations, we propose a fully autonomous rotorcraft UAS that is capable of performing repeated flights for long‐term observation missions without any human intervention. We address two key technologies that are critical for such a system: full platform autonomy to enable mission execution independently from human operators and the ability of vision‐based precision landing on a recharging station for automated energy replenishment. High‐level autonomous decision making is implemented as a hierarchy of master and slave state machines. Vision‐based precision landing is enabled by estimating the landing pad's pose using a bundle of AprilTag fiducials configured for detection from a wide range of altitudes. We provide an extensive evaluation of the landing pad pose estimation accuracy as a function of the bundle's geometry. The functionality of the complete system is demonstrated through two indoor experiments with duration of 11 and 10.6 hr, and one outdoor experiment with a duration of 4 hr. The UAS executed 16, 48, and 22 flights, respectively, during these experiments. In the outdoor experiment, the ratio between flying to collect data and charging was 1–10, which is similar to past work in this domain. All flights were fully autonomous with no human in the loop. To our best knowledge, this is the first research publication about the long‐term outdoor operation of a quadrotor system with no human interaction.     

Robots in Space: U.S. Missions and Technology Requirements into the Next Century

The Telerobotics Program of the National Aeronautics and Space Administration (NASA) Office of Space Science is developing innovative telerobotics technologies to enable or support a wide range of space missions over the next decade and beyond. These technologies fall into four core application areas: landers, surface vehicles (rovers), and aerovehicles for solar system exploration and science; rovers for commercially supported lunar activities; free-flying and platform-attached robots for in-orbit servicing and assembly; and robots supporting in-orbit biotechnology and microgravity experiments. Such advanced robots will enable missions to explore Mars, Venus, and Saturn's moon Titan, as well as probes to sample comets and asteroids. They may also play an important role in commercially funded exploration of large regions on Earth's Moon, as well as the eventual development of a human-supporting Lunar Outpost. In addition, in-orbit servicing of satellites and maintenance of large platforms like the International Space Station will require extensive robotics capabilities.     

Visual terrain mapping for Mars exploration

One goal for future Mars missions is for a rover to be able to navigate autonomously to science targets not visible to the rover, but seen in orbital or descent images. This can be accomplished if accurate maps of the terrain are available for the rover to use in planning and localization. We describe techniques to generate such terrain maps using images with a variety of resolutions and scales, including surface images from the lander and rover, descent images captured by the lander as it approaches the planetary surface, and orbital images from current and future Mars orbiters. At the highest resolution, we process surface images captured by rovers and landers using bundle adjustment. At the next lower resolution (and larger scale), we use wide-baseline stereo vision to map terrain distant from a rover with surface images. Mapping the lander descent images using a structure-from-motion algorithm generates data at a hierarchy of resolutions. These provide a link between the high-resolution surface images and the low-resolution orbital images. Orbital images are mapped using similar techniques, although with the added complication that the images may be captured with a variety of sensors. Robust multi-modal matching techniques are applied to these images. The terrain maps are combined using a system for unifying multi-resolution models and integrating three-dimensional terrains. The result is a multi-resolution map that can be used to generate fixed-resolution maps at any desired scale.     

Mars Pathfinder Microrover

When the Mars Pathfinder (MPF) spacecraft lands on Mars, the Microrover Flight Experiment (MFEX) will be deployed and perform its mission to conduct technology experiments verifying the engineering design, to deploy an alpha proton x-ray spectrometer (APXS) to measure elemental properties of rocks and soil, and to image the MPF lander. In accomplishing this mission the MFEX rover must determine a safe path to goal locations traversing over a poorly known Martian surface. The rover does this mission with a capable mobile platform executing on-board autonomous functions of navigation and hazard avoidance. In this paper we describe the rover, its operational environment and the implementation of the on-board autonomous functions.