Vision‐based terrain characterization and traversability assessment

This article presents novel techniques for real‐time terrain characterization and assessment of terrain traversability for a field mobile robot using a vision system and artificial neural networks. The key terrain traversability characteristics are identified as roughness, slope, discontinuity, and hardness. These characteristics are extracted from imagery data obtained from cameras mounted on the robot and are represented in a fuzzy logic framework using perceptual, linguistic fuzzy sets. The approach adopted is highly robust and tolerant to imprecision and uncertainty inherent in sensing and perception of natural environments. The four traversability characteristics are combined to form a single Fuzzy Traversability Index, which quantifies the ease‐of‐traversal of the terrain by the mobile robot. Experimental results are presented to demonstrate the capability of the proposed approach for classification of different terrain segments based on their traversability. © 2001 John Wiley & Sons, Inc.     

星球车松软星壤的轮腿式探测系统:Ⅱ-感知车轮

为增强星球车松软星壤的穿越通过能力,提出了轮壤交互接触信息的感知车轮设计。该车轮是一种星球车的前置轮腿式探测系统(WOLS)的关键部分,可实现动态轮壤交互的关键量测量(轮壤作用力/力矩、轮壤接触角和车轮沉陷量)。研究分析了轮壤力学的关键测量参量及其分组,完成了感知车轮的硬件设计和集成,提出了待测参数的在线测量模型和方法,通过标定校准和实车测试验证了该感知车轮的性能。     

New Traversability Indices and Traversability Grid for Integrated Sensor/Map‐Based Navigation

This paper presents new measures of terrain traversability at short range and long range of a mobile robot; namely, local and global traversability indices. The sensor‐based local traversability index is related by a set of linguistic rules to large obstacles and surface softness within a short range of the robot measured by on‐board sensors. The map‐based global traversability index is obtained from the terrain topographic map, and is based on major surface features such as hills and lakes within a long range of the robot. These traversability indices complement the mid‐range sensor‐based regional traversability index introduced earlier. Each traversability index is represented by four fuzzy sets with the linguistic labels {POOR, LOW, MODERATE, HIGH}, corresponding to surfaces that are unsafe, moderately‐unsafe, moderately‐safe, and safe for traversal, respectively. The global terrain analysis also leads to the new concepts of traversability map and traversability grid for representation of terrain quality based on the global map information. The traversability indices are used in two sensor‐based traverse‐local and traverse‐regional behaviors and one map‐based traverse‐global behavior. These behaviors are integrated with a map‐based seek‐goal behavior to ensure that the mobile robot reaches its goal safely while avoiding both sensed and mapped terrain hazards. This provides a unified system in which the two independent sources of terrain quality information, i.e., prior maps and on‐board sensors, are integrated together for reactive robot navigation. The paper is concluded by a graphical simulation study. © 2003 Wiley Periodicals, Inc.     

Rule–based reasoning and neural network perception for safe off–road robot mobility

Operational safety and health monitoring are critical matters for autonomous field mobile robots such as planetary rovers operating on challenging terrain. This paper describes relevant rover safety and health issues and presents an approach to maintaining vehicle safety in a mobility and navigation context. The proposed rover safety module is composed of two distinct components: safe attitude (pitch and roll) management and safe traction management. Fuzzy logic approaches to reasoning about safe attitude and traction management are presented, wherein inertial sensing of safety status and vision–based neural network perception of terrain quality are used to infer safe speeds of traversal. Results of initial field tests and laboratory experiments are also described. The approach provides an intrinsic safety cognizance and a capacity for reactive mitigation of robot mobility and navigation risks.     

Terrain‐adaptive wheel speed control on the Curiosity Mars rover: Algorithm and flight results

NASA's Mars Science Laboratory Curiosity rover landed in August 2012 and began experiencing higher rates of wheel damage beginning in October 2013. While the wheels were designed to accumulate considerable damage, the unexpected damage rate raised concerns regarding wheel lifetime. In response, the Jet Propulsion Laboratory developed and deployed mobility flight software on Curiosity that reduces the forces on the wheels. The new algorithm adapts each wheel's speed to fit the terrain topography in real time, by leveraging the rover's measured attitude rates and rocker/bogie suspension angles and rates. Together with a rigid‐body kinematics model, it estimates the real‐time wheel‐terrain contact angles and commands idealized, no‐slip wheel angular rates. In addition, free‐floating “wheelies” are detected and autonomously corrected. Ground test data indicate that the forces on the wheels are reduced by 19% for leading wheels and 11% for middle leading wheels. On the ground, the required data volume increased by up to 129%, and drive duration increased by up to 25%. In flight, data collected over 3.6 km and 149 drives confirmed a reduction in wheel current, correlated with wheel torque, of 18.7%. The new algorithm proved to use fewer resources in flight than ground estimates suggested, as only a 10% increase in drive duration and double the drive data volume were experienced. These data indicate the promise of the new algorithm to extend the life of the wheels for the Curiosity rover. This paper describes the algorithm, its ground testing campaign and associated challenges, and its validation, implementation, and performance in flight.     

Precise pose estimation of the NASA Mars 2020 Perseverance rover through a stereo-vision-based approach

Visual Odometry (VO) is a fundamental technique to enhance the navigation capabilities of planetary exploration rovers. By processing the images acquired during the motion, VO methods provide estimates of the relative position and attitude between navigation steps with the detection and tracking of two-dimensional (2D) image keypoints. This method allows one to mitigate trajectory inconsistencies associated with slippage conditions resulting from dead-reckoning techniques. We present here an independent analysis of the high-resolution stereo images of the NASA Mars 2020 Perseverance rover to retrieve its accurate localization on sols 65, 66, 72, and 120. The stereo pairs are processed by using a 3D-to-3D stereo-VO approach that is based on consolidated techniques and accounts for the main nonlinear optical effects characterizing real cameras. The algorithm is first validated through the analysis of rectified stereo images acquired by the NASA Mars Exploration Rover Opportunity, and then applied to the determination of Perseverance's path. The results suggest that our reconstructed path is consistent with the telemetered trajectory, which was directly retrieved onboard the rover's system. The estimated pose is in full agreement with the archived rover's position and attitude after short navigation steps. Significant differences (~10–30 cm) between our reconstructed and telemetered trajectories are observed when Perseverance traveled distances larger than 1 m between the acquisition of stereo pairs.     

Fast approximate clearance evaluation for rovers with articulated suspension systems

We present a light‐weight body‐terrain clearance evaluation algorithm for the automated path planning of NASA's Mars 2020 rover. Extraterrestrial path planning is challenging due to the combination of terrain roughness and severe limitation in computational resources. Path planning on cluttered and/or uneven terrains requires repeated safety checks on all the candidate paths at a small interval. Predicting the future rover state requires simulating the vehicle settling on the terrain, which involves an inverse‐kinematics problem with iterative nonlinear optimization under geometric constraints. However, such expensive computation is intractable for slow spacecraft computers, such as RAD750, which is used by the Curiosity Mars rover and upcoming Mars 2020 rover. We propose the approximate clearance evaluation (ACE) algorithm, which obtains conservative bounds on vehicle clearance, attitude, and suspension angles without iterative computation. It obtains those bounds by estimating the lowest and highest heights that each wheel may reach given the underlying terrain, and calculating the worst‐case vehicle configuration associated with those extreme wheel heights. The bounds are guaranteed to be conservative, hence ensuring vehicle safety during autonomous navigation. ACE is planned to be used as part of the new onboard path planner of the Mars 2020 rover. This paper describes the algorithm in detail and validates our claim of conservatism and fast computation through experiments.     

Planetary Rover Developments Supporting Mars Exploration, Sample Return and Future Human-Robotic Colonization

We overview our recent research on planetary mobility. Products of this effort include the Field Integrated Design & Operations rover (FIDO), Sample Return Rover (SRR), reconfigurable rover units that function as an All Terrain Explorer (ATE), and a multi-Robot Work Crew of closely cooperating rovers (RWC). FIDO rover is an advanced technology prototype; its design and field testing support NASA's development of long range, in situ Mars surface science missions. Complementing this, SRR implements autonomous visual recognition, navigation, rendezvous, and manipulation functions enabling small object pick-up, handling, and precision terminal docking to a Mars ascent vehicle for future Mars Sample Return. ATE implements on-board reconfiguration of rover geometry and control for adaptive response to adverse and changing terrain, e.g., traversal of steep, sandy slopes. RWC implements coordinated control of two rovers under closed loop kinematics and force constraints, e.g., transport of large payloads, as would occur in robotic colonies at future Mars outposts. RWC is based in a new extensible architecture for decentralized control of, and collective state estimation by multiple heterogeneous robotic platforms—CAMPOUT; we overview the key architectural features. We have conducted experiments with all these new rover system concepts over variable natural terrain. For each of the above developments, we summarize our approach, some of our key experimental results to date, and our future directions of planned development.     

Study on climbing strategy and analysis of Mars rover

To ensure the safety and efficiency of Zhurong Mars rover when climbing a slope on Mars, the forces of the rover under four climbing methods, which are normal climbing, Z-type climbing, diagonal climbing, and bionic wriggle climbing, are analyzed. Each method corresponds to different maximum climbing slopes. The experiments are carried out with a backup rover on dense and soft terrains to determine the range of climbing slope for different climbing methods. According to the slope, peak current, cost of transport, and state of terrain, the climbing strategy is given. For dense and soft terrains, the soil cohesive is 0.99 and 1.4 kN/mⁿ⁺¹ and soil friction modules are 1528 and 700 kN/mⁿ⁺², respectively. Specifically, normal climbing is recommended for low-range slopes, while Z-type or diagonal climbing are suggested for medium-range slopes, and bionic wriggle climbing is found to be optimal for high-range slopes. To ensure the safety of the Zhurong Mars rover, it fails climbing if the critical values are exceeded. These results provide valuable insights for human operators when planning the rover's slope-climbing actions on Mars.     

Adaptive localization and mapping with application to planetary rovers

Future exploration rovers will be equipped with substantial onboard autonomy. SLAM is a fundamental part and has a close connection with robot perception, planning, and control. The community has made great progress in the past decade by enabling real‐world solutions and is addressing important challenges in high‐level scalability, resources awareness, and domain adaptation. A novel adaptive SLAM system is proposed to accomplish rover navigation and computational demands. It starts from a three‐dimensional odometry dead reckoning solution and builds up to a full graph optimization that takes into account rover traction performance. A complete kinematics of the rover locomotion system improves the wheel odometry solution. In addition, an odometry error model is inferred using Gaussian processes (GPs) to predict nonsystematic errors induced by poor traction of the rover with the terrain. The nonparametric GP regression serves to adapt the localization and mapping to the current navigation demands (domain adaptation). The method brings scalability and adaptiveness to modern SLAM. Therefore, an adaptive strategy develops to adjust the image frame rate (active perception) and to influence the optimization backend by including high informative keyframes in the graph (adaptive information gain). The work is experimentally verified on a representative planetary rover under a realistic field test scenario. The results show a modern SLAM systems that adapt to the predicted error. The system maintains accuracy with less number of nodes taking the most benefit of both wheel and visual methods in a consistent graph‐based smoothing approach.     

Autonomous science during large‐scale robotic survey

Today's planetary exploration robots rarely travel beyond the yesterday imagery. However, advances in autonomous mobility will soon permit single‐command site surveys of multiple kilometers. Here scientists cannot see the terrain in advance, and explorer robots must navigate and collect data autonomously. Onboard science data understanding can improve these surveys with image analysis, pattern recognition, learned classification, and information‐theoretic planning. We report on field experiments near Amboy Crater, California, that demonstrate fundamental capabilities for autonomous surficial mapping of geologic phenomena with a visible near‐infrared spectrometer. We develop an approach to “science on the fly'' that adapts the robot's exploration using collected instrument data. We demonstrate feature detection and visual servoing to acquire spectra from dozens of targets without human intervention. The rover interprets spectra onboard, learning spatial models of science phenomena that guide it toward informative areas. It discovers spatial structure (correlations between neighboring regions) and cross‐sensor structure (correlations between different scales). The rover uses surface observations to reinterpret satellite imagery and improve exploration efficiency. © 2011 Wiley Periodicals, Inc.     

Fuzzy-based speed estimation for navigation of unmanned robots

This study presents a framework for assessing the navigation speed of unmanned robots by combining information extracted from the 3D world model of natural terrain with regional traversability based on the fuzzy technique. The proposed method divides the world model into several patches, extracts the slope and roughness of each terrain patch along four heading directions, and then uses them to evaluate the level of difficulty associated with the traversal. The slope is estimated through curved surface fitting, and roughness is obtained using fractal-based analysis together with another two RMS metrics. As navigation systems can cope with the imprecision and uncertainty of input data, we modify the Seraji’s fuzzy-based measure to assess the traversability and navigation speed of each patch for path planning. The proposed method is tested on both fractal and real terrain to verify its effectiveness.     

The (in)accuracy of novice rover operators' perception of obstacle height from monoscopic images

Researchers have previously described a mobile robot, or rover, operator's difficulty in accurately perceiving the rover's tilt and roll, which can lead to rollover accidents. Safe mobile robot navigation and effective mission planning also require an operator to accurately interpret and understand the geometry and scale of features in the rover's environment. This work presents an experiment that measures an observer's ability to estimate height of distant (5-15 m) obstacles given an accurate local model (e.g., within 0-5 m of the rover), a panoramic image, and a physical mock-up of the local terrain. The experimental conditions were intended to represent a best-case scenario for a stopped rover equipped with short base-line stereoscopic cameras. The participants' task was to extrapolate the well-modeled local geometry to monoscopic images of the more distant terrain. The experiment compared two estimation techniques. With the first technique, each observer physically indicated his or her direct estimates of the obstacle distance and height. With the second estimation technique, which we call horizon analysis, the observer indicated the position of the top and bottom of each rock on an image and the height was calculated by measuring the visual angle between the theoretical horizon and the points indicated by the observer. The direct estimation technique overestimated the height of the rocks by an average of 190%; the horizon analysis technique overestimated by 80%. The results suggest that even when provided with a rich set of supplementary and context information, rover operators have significant difficulty in vertically perceiving the scale of distant terrain. The results also suggest that horizon analysis is a more accurate method for determining the height of distant rover navigation obstacles, when the local terrain is nearly level.     

月球车仿真系统中若干视景真实感实现技术

虚拟现实技术的出现为月球漫游车的设计、优化等提供了新的有效手段.为在虚拟现实环境下开发月球车仿真系统,视景真实感实现技术在其中具有霞要地位.研究了真实感地形生成技术、仿真过程中的粒子系统特效技术以及车轮沉陷和车轮轨迹生成技术.基于分形技术和月面特征地形生月面高程数据;利用Muhigen Creator构建和渲染了月面几何模型;基于OpenGL Performer开发了仿真程序并应用到一种月球车运动仿真系统中.系统可在微机和SGI图形工作站运行,仿真结果显示具有良好的真实感.     

Slippage and immobilization detection for planetary exploration rovers via machine learning and proprioceptive sensing

This paper presents a new methodology where machine learning is used for detecting various levels of slip in the context of planetary exploration robotic missions. This methodology aims at employing proprioceptive rover sensor signals. Consequently, no operational complexity is added to the rover's commanding and it is independent of lighting conditions. Two supervised learning methods (Support Vector Machines and Artificial Neural Networks) are compared to two unsupervised learning approaches (K‐means and Self‐Organizing Maps (SOM)). Physical experiments using a single‐wheel testbed equipped with an MSL spare wheel and a real planetary exploration rover validate the implemented methodology. Performance is evaluated in terms of well‐known metrics both considering single data points and subsets of consecutive data points (moving median filter). Computation time and storage requirements are also examined. One of the SOM‐based algorithms, semantic SOM method, demonstrates a proper balance between the benefits of supervised learning algorithms (high success rate, >96%) and the advantages of unsupervised learning methods (low storage requirements, 5 kb, and no need of manually‐labeled training data). This paper also addresses the most convenient placement of IMU sensors on the rover chassis such that slippage detection is maximized.     

Range‐dependent Terrain Mapping and Multipath Planning using Cylindrical Coordinates for a Planetary Exploration Rover

This paper presents terrain mapping and path‐planning techniques that are key issues for autonomous mobility of a planetary exploration rover. In this work, a LIDAR (light detection and ranging) sensor is used to obtain geometric information on the terrain. A point cloud of the terrain feature provided from the LIDAR sensor is usually converted to a digital elevation map. A sector‐shaped reference grid for the conversion process is proposed in this paper, resulting in an elevation map with cylindrical coordinates termed as C²DEM. This conversion approach achieves a range‐dependent resolution for the terrain mapping: a detailed terrain representation near the rover and a sparse representation far from the rover. The path planning utilizes a cost function composed of terrain inclination, terrain roughness, and path length indices, each of which is subject to a weighting factor. The multipath planning developed in this paper first explores possible sets of weighting factors and generates multiple candidate paths. The most feasible path is then determined by a comparative evaluation between the candidate paths. Field experiments with a rover prototype at a Lunar/Martian analog site were performed to confirm the feasibility of the proposed techniques, including the range‐dependent terrain mapping with C²DEM and the multipath‐planning method.     

Introducing a globally consistent orbital‐based localization system

In spite of the good performance of space exploratory missions, open issues still await to be solved. In autonomous or composite semi‐autonomous exploration of planetary land surfaces, rover localization is such an issue. The rovers of these missions (e.g., the MER and MSL) navigate relatively to their landing spot, ignoring their exact position on the coordinate system defined for the celestial body they explore. However, future advanced missions, like the Mars Sample Return, will require the localization of rovers on a global frame rather than the arbitrarily defined landing frame. In this paper we attempt to retrieve the absolute rover's location by identifying matching Regions of Interest (ROIs) between orbital and land images. In particular, we propose a system comprising two parts, an offline and an onboard one, which functions as follows: in advance of the mission a Global ROI Network (GN) is built offline by investigating the satellite images near the predicted touchdown ellipse, while during the mission a Local ROI Network (LN) is constructed counting on the images acquired by the vision system of the rover along its traverse. The last procedure relies on the accurate VO‐based relative rover localization. The LN is then paired with the GN through a modified 2D DARCES algorithm. The system has been assessed on real data collected by the ESA at the Atacama desert. The results demonstrate the system's potential to perform absolute localization, on condition that the area includes discriminative ROIs. The main contribution of this work is the enablement of global localization performed on contemporary rovers without requiring any additional hardware, such as long range LIDARs.     

Normal Distributions Transform Traversability Maps: LIDAR‐Only Approach for Traversability Mapping in Outdoor Environments

Safe and reliable autonomous navigation in unstructured environments remains a challenge for field robots. In particular, operating on vegetated terrain is problematic, because simple purely geometric traversability analysis methods typically classify dense foliage as nontraversable. As traversing through vegetated terrain is often possible and even preferable in some cases (e.g., to avoid executing longer paths), more complex multimodal traversability analysis methods are necessary. In this article, we propose a three‐dimensional (3D) traversability mapping algorithm for outdoor environments, able to classify sparsely vegetated areas as traversable, without compromising accuracy on other terrain types. The proposed normal distributions transform traversability mapping (NDT‐TM) representation exploits 3D LIDAR sensor data to incrementally expand normal distributions transform occupancy (NDT‐OM) maps. In addition to geometrical information, we propose to augment the NDT‐OM representation with statistical data of the permeability and reflectivity of each cell. Using these additional features, we train a support‐vector machine classifier to discriminate between traversable and nondrivable areas of the NDT‐TM maps. We evaluate classifier performance on a set of challenging outdoor environments and note improvements over previous purely geometrical traversability analysis approaches.     

SmartNav: A rule‐free fuzzy approach to rover navigation

This paper introduces the basic concepts of the SmartNav rule‐free fuzzy approach to safe rover navigation through hazardous natural terrain. This novel “rule‐free” approach reduces the complexity in rover navigation where many alternative paths must be evaluated and compared. The SmartNav rover navigation architecture integrates goal attainment with both local and regional hazard avoidance. Goal and safety preference factors differentiate between preferred and unpreferred terrain sectors. The goal‐preference factor is used to make sector evaluation based on the sector orientation relative to the designated goal position. The safety‐preference factors are used to make sector evaluations on the basis of the sector local and regional hazards. These sector evaluations are blended to find the effective preference factor for each sector. The preference factors of all sectors are then compared to choose the heading command for the rover. The rover speed command is also computed based on the goal distance and safety‐preference factor of the chosen sector. The above navigation steps are continuously repeated throughout the rover motion. Numerical examples are presented to illustrate the basic concepts introduced in this paper, and tests on a commercial rover are planned. © 2005 Wiley Periodicals, Inc.