A Bayesian exploration-exploitation approach for optimal online sensing and planning with a visually guided mobile robot

We address the problem of online path planning for optimal sensing with a mobile robot. The objective of the robot is to learn the most about its pose and the environment given time constraints. We use a POMDP with a utility function that depends on the belief state to model the finite horizon planning problem. We replan as the robot progresses throughout the environment. The POMDP is high-dimensional, continuous, non-differentiable, nonlinear, non-Gaussian and must be solved in real-time. Most existing techniques for stochastic planning and reinforcement learning are therefore inapplicable. To solve this extremely complex problem, we propose a Bayesian optimization method that dynamically trades off exploration (minimizing uncertainty in unknown parts of the policy space) and exploitation (capitalizing on the current best solution). We demonstrate our approach with a visually-guide mobile robot. The solution proposed here is also applicable to other closely-related domains, including active vision, sequential experimental design, dynamic sensing and calibration with mobile sensors.     

Real-time hierarchical POMDPs for autonomous robot navigation

This paper proposes a new hierarchical formulation of POMDPs for autonomous robot navigation that can be solved in real-time, and is memory efficient. It will be referred to in this paper as the Robot Navigation–Hierarchical POMDP (RN-HPOMDP). The RN-HPOMDP is utilized as a unified framework for autonomous robot navigation in dynamic environments. As such, it is used for localization, planning and local obstacle avoidance. Hence, the RN-HPOMDP decides at each time step the actions the robot should execute, without the intervention of any other external module for obstacle avoidance or localization. Our approach employs state space and action space hierarchy, and can effectively model large environments at a fine resolution. Finally, the notion of the reference POMDP is introduced. The latter holds all the information regarding motion and sensor uncertainty, which makes the proposed hierarchical structure memory efficient and enables fast learning. The RN-HPOMDP has been experimentally validated in real dynamic environments.     

一种动态不确定环境中机器人路径规划方法

针对动态不确定环境下的机器人路径规划问题,将部分可观察马尔可夫决策过程（POMDP）与人工势场法（APF）的优点相结合,提出一种新的机器人路径规划方法。该方法充分考虑了实际环境中信息的部分可观测性,并且利用APF无需大量计算的优点指导POMDP算法的奖赏值设定,以提高POMDP算法的决策效率。仿真实验表明,所提出的算法拥有较高的搜索效率,能够快速地到达目标点。     

Autonomous robotic exploration using a utility function based on Rényi’s general theory of entropy

In this paper we present a novel information-theoretic utility function for selecting actions in a robot-based autonomous exploration task. The robot’s goal in an autonomous exploration task is to create a complete, high-quality map of an unknown environment as quickly as possible. This implicitly requires the robot to maintain an accurate estimate of its pose as it explores both unknown and previously observed terrain in order to correctly incorporate new information into the map. Our utility function simultaneously considers uncertainty in both the robot pose and the map in a novel way and is computed as the difference between the Shannon and the Rényi entropy of the current distribution over maps. Rényi’s entropy is a family of functions parameterized by a scalar, with Shannon’s entropy being the limit as this scalar approaches unity. We link the value of this scalar parameter to the predicted future uncertainty in the robot’s pose after taking an exploratory action. This effectively decreases the expected information gain of the action, with higher uncertainty in the robot’s pose leading to a smaller expected information gain. Our objective function allows the robot to automatically trade off between exploration and exploitation in a way that does not require manually tuning parameter values, a significant advantage over many competing methods that only use Shannon’s definition of entropy. We use simulated experiments to compare the performance of our proposed utility function to these state-of-the-art utility functions. We show that robots that use our proposed utility function generate maps with less uncertainty and fewer visible artifacts and that the robots have less uncertainty in their pose during exploration. Finally, we demonstrate that a real-world robot using our proposed utility function is able to successfully create a high-quality map of an indoor office environment.     

智能空间下基于分层任务网络的服务机器人任务规划

针对在复杂、动态的家庭环境下,如何让机器人获取足够多的环境信息并根据环境信息进行自主的任务规划,提出了智能空间技术支持下基于分层任务网络的服务机器人任务规划方案.利用智能空间技术为机器人提供充足的环境上下文信息,用基于分层任务网络设计的JSHOP2规划器执行机器人任务规划.为了提高机器人任务规划的自主性和智能性,在规划领域文件中加入不同的模板信息,使机器人具有根据环境的不同自动对任务进行调整的能力.仿真实验结果表明利用该方法能够有效地提高机器人任务规划的性能.     

Parametric POMDPs for planning in continuous state spaces

This work addresses the problem of decision-making under uncertainty for robot navigation. Since robot navigation is most naturally represented in a continuous domain, the problem is cast as a continuous-state POMDP. Probability distributions over state space, or beliefs, are represented in parametric form using low-dimensional vectors of sufficient statistics. The belief space, over which the value function must be estimated, has dimensionality equal to the number of sufficient statistics. Compared to methods based on discretising the state space, this work trades the loss of the belief space’s convexity for a reduction in its dimensionality and an efficient closed-form solution for belief updates. Fitted value iteration is used to solve the POMDP. The approach is empirically compared to a discrete POMDP solution method on a simulated continuous navigation problem. We show that, for a suitable environment and parametric form, the proposed method is capable of scaling to large state-spaces.     

Incorporating decision-theoretic planning in a robot architecture

The goal of robotics research is to design a robot to fulfill a variety of tasks in the real world. Inherent in the real world is a high degree of uncertainty about the robot’s behavior and about the world. We introduce a robot task architecture, DTRC, that generates plans with actions that incorporate costs and uncertain effects, and states that yield rewards.The use of a decision-theoretic planner in a robot task architecture is demonstrated on the mobile robot domain of miniature golf. The miniature golf domain shows the application of decision-theoretic planning in an inherently uncertain domain, and demonstrates that by using decision-theoretic planning as the reasoning method in a robot task architecture, accommodation for uncertain information plays a direct role in the reasoning process.     

Planning for robotic exploration based on forward simulation

We address the problem of controlling a mobile robot to explore a partially known environment. The robot’s objective is the maximization of the amount of information collected about the environment. We formulate the problem as a partially observable Markov decision process (POMDP) with an information-theoretic objective function, and solve it applying forward simulation algorithms with an open-loop approximation. We present a new sample-based approximation for mutual information useful in mobile robotics. The approximation can be seamlessly integrated with forward simulation planning algorithms. We investigate the usefulness of POMDP based planning for exploration, and to alleviate some of its weaknesses propose a combination with frontier based exploration. Experimental results in simulated and real environments show that, depending on the environment, applying POMDP based planning for exploration can improve performance over frontier exploration.     

Exploiting submodular value functions for scaling up active perception

In active perception tasks, an agent aims to select sensory actions that reduce its uncertainty about one or more hidden variables. For example, a mobile robot takes sensory actions to efficiently navigate in a new environment. While partially observable Markov decision processes (POMDPs) provide a natural model for such problems, reward functions that directly penalize uncertainty in the agent’s belief can remove the piecewise-linear and convex (PWLC) property of the value function required by most POMDP planners. Furthermore, as the number of sensors available to the agent grows, the computational cost of POMDP planning grows exponentially with it, making POMDP planning infeasible with traditional methods. In this article, we address a twofold challenge of modeling and planning for active perception tasks. We analyze \(\rho \)POMDP and POMDP-IR, two frameworks for modeling active perception tasks, that restore the PWLC property of the value function. We show the mathematical equivalence of these two frameworks by showing that given a \(\rho \)POMDP along with a policy, they can be reduced to a POMDP-IR and an equivalent policy (and vice-versa). We prove that the value function for the given \(\rho \)POMDP (and the given policy) and the reduced POMDP-IR (and the reduced policy) is the same. To efficiently plan for active perception tasks, we identify and exploit the independence properties of POMDP-IR to reduce the computational cost of solving POMDP-IR (and \(\rho \)POMDP). We propose greedy point-based value iteration (PBVI), a new POMDP planning method that uses greedy maximization to greatly improve scalability in the action space of an active perception POMDP. Furthermore, we show that, under certain conditions, including submodularity, the value function computed using greedy PBVI is guaranteed to have bounded error with respect to the optimal value function. We establish the conditions under which the value function of an active perception POMDP is guaranteed to be submodular. Finally, we present a detailed empirical analysis on a dataset collected from a multi-camera tracking system employed in a shopping mall. Our method achieves similar performance to existing methods but at a fraction of the computational cost leading to better scalability for solving active perception tasks.     

Integrated planning and execution control for an autonomous mobile robot

We present a distributed hierarchical planning and execution monitoring system and its implementation on an actual mobile robot. The planning system is a distributed hierarchical domain independent system called FPS for Flexible Planning System. It is a rule based plan generation system with planning specific and domain specific rules. A planning solution to the ‘Boxes and Wedge’ Problem is presented.The Robot Control System (RCS) operates and monitors the robot in the real world. In order to allow real-time responses to asynchronous events (both internal and external), RCS consists of a rule-based decision kernel and a distributed set of sensor/effector monitors. RCS contains an execution model and may authorize local corrective actions, e.g., unexpected obstacle avoidance during execution of a trajectory. RCS also generates status and failure reports through which the PMs inform the different decision subsystems as to the robot's state and current capacities. The failure reports help the RCS and planners in correcting/replanning a plan that has aborted. An illustrative example of system behaviour is to be presented.     

Planning in partially-observable switching-mode continuous domains

Continuous-state POMDPs provide a natural representation for a variety of tasks, including many in robotics. However, most existing parametric continuous-state POMDP approaches are limited by their reliance on a single linear model to represent the world dynamics. We introduce a new switching-state dynamics model that can represent multi-modal state-dependent dynamics. We present the Switching Mode POMDP (SM-POMDP) planning algorithm for solving continuous-state POMDPs using this dynamics model. We also consider several procedures to approximate the value function as a mixture of a bounded number of Gaussians. Unlike the majority of prior work on approximate continuous-state POMDP planners, we provide a formal analysis of our SM-POMDP algorithm, providing bounds, where possible, on the quality of the resulting solution. We also analyze the computational complexity of SM-POMDP. Empirical results on an unmanned aerial vehicle collisions avoidance simulation, and a robot navigation simulation where the robot has faulty actuators, demonstrate the benefit of SM-POMDP over a prior parametric approach.     

基于模糊控制器的机器人路径规划研究

针对复杂环境下移动机器人路径规划实际问题,提出了一种基于行为的移动机器人控制体系结构,设计了一种基于模糊控制器的移动机器人实时路径规划算法,为移动机器人在未知环境中的导航提出了一种新的思路.仿真结果表明,移动机器人能够克服环境中的不确定性,可靠地完成复杂任务,该算法有计算量小,效率高,鲁棒性好等优点.     

Opportunistic sampling-based active visual SLAM for underwater inspection

This paper reports on an active SLAM framework for performing large-scale inspections with an underwater robot. We propose a path planning algorithm integrated with visual SLAM that plans loop-closure paths in order to decrease navigation uncertainty. While loop-closing revisit actions bound the robot’s uncertainty, they also lead to redundant area coverage and increased path length. Our proposed opportunistic framework leverages sampling-based techniques and information filtering to plan revisit paths that are coverage efficient. We employ Gaussian process regression for modeling the prediction of camera registrations and use a two-step optimization procedure for selecting revisit actions. We show that the proposed method offers many benefits over existing solutions and good performance for bounding navigation uncertainty in long-term autonomous operations with hybrid simulation experiments and real-world field trials performed by an underwater inspection robot.     

一种基于MAP估计的移动机器人视觉自定位方法

提出一种能够工作在三维路标环境中的视觉自定位系统. 机器人通过 MAP 估计器融合里程计和单向摄象机的图像信息递归估计其自身位姿状态. 本文构建了传感器的非线性模型并且在系统运行中嵌入和跟踪机器人运动和视觉信息的不确定性. 本文从概率几何观点阐述传感信息不确定性, 用 unscented 变换传播经过非线性变换的相关系统信息. 考虑到处理能力, 机器人在地图元素的投影特征附近提取相应图像特征并通过统计距离描述数据关联程度. 本文的一系列系统性实验证明了该系统的稳定性和精确性.     

Efficient vision-based navigation

In this article, we present a novel approach to learning efficient navigation policies for mobile robots that use visual features for localization. As fast movements of a mobile robot typically introduce inherent motion blur in the acquired images, the uncertainty of the robot about its pose increases in such situations. As a result, it cannot be ensured anymore that a navigation task can be executed efficiently since the robot’s pose estimate might not correspond to its true location. We present a reinforcement learning approach to determine a navigation policy to reach the destination reliably and, at the same time, as fast as possible. Using our technique, the robot learns to trade off velocity against localization accuracy and implicitly takes the impact of motion blur on observations into account. We furthermore developed a method to compress the learned policy via a clustering approach. In this way, the size of the policy representation is significantly reduced, which is especially desirable in the context of memory-constrained systems. Extensive simulated and real-world experiments carried out with two different robots demonstrate that our learned policy significantly outperforms policies using a constant velocity and more advanced heuristics. We furthermore show that the policy is generally applicable to different indoor and outdoor scenarios with varying landmark densities as well as to navigation tasks of different complexity.     

Human–robot communication for collaborative decision making — A probabilistic approach

Humans and robots need to exchange information if the objective is to achieve a task collaboratively. Two questions are considered in this paper: what and when to communicate. To answer these questions, we developed a human–robot communication framework which makes use of common probabilistic robotics representations. The data stored in the representation determines what to communicate, and probabilistic inference mechanisms determine when to communicate. One application domain of the framework is collaborative human–robot decision making: robots use decision theory to select actions based on perceptual information gathered from their sensors and human operators. In this paper, operators are regarded as remotely located, valuable information sources which need to be managed carefully. Robots decide when to query operators using Value-Of-Information theory, i.e. humans are only queried if the expected benefit of their observation exceeds the cost of obtaining it. This can be seen as a mechanism for adjustable autonomy whereby adjustments are triggered at run-time based on the uncertainty in the robots’ beliefs related to their task. This semi-autonomous system is demonstrated using a navigation task and evaluated by a user study. Participants navigated a robot in simulation using the proposed system and via classical teleoperation. Results show that our system has a number of advantages over teleoperation with respect to performance, operator workload, usability, and the users’ perception of the robot. We also show that despite these advantages, teleoperation may still be a preferable driving mode depending on the mission priorities.     

A Navigation System for Assistant Robots Using Visually Augmented POMDPs

Assistant robots have received special attention from the research community in the last years. One of the main applications of these robots is to perform care tasks in indoor environments such as houses, nursing homes or hospitals, and therefore they need to be able to navigate robustly for long periods of time. This paper focuses on the navigation system of SIRA, a robotic assistant for elderly and/or blind people based on a Partially Observable Markov Decision Process (POMDP) to global localize the robot and to direct its goal-oriented actions. The main novel feature of our approach is that it combines sonar and visual information in a natural way to produce state transitions and observations in the framework of Markov Decision Processes. Besides this multisensorial fusion, a two-level layered planning architecture that combines several planning objectives (such as guiding to a goal room and reducing locational uncertainty) improves the robustness of the navigation system, as its shown in our experiments with SIRA navigating corridors.     

Global motion planning under uncertain motion, sensing, and environment map

Uncertainty in motion planning is often caused by three main sources: motion error, sensing error, and imperfect environment map. Despite the significant effect of all three sources of uncertainty to motion planning problems, most planners take into account only one or at most two of them. We propose a new motion planner, called Guided Cluster Sampling (GCS), that takes into account all three sources of uncertainty for robots with active sensing capabilities. GCS uses the Partially Observable Markov Decision Process (POMDP) framework and the point-based POMDP approach. Although point-based POMDPs have shown impressive progress over the past few years, it performs poorly when the environment map is imperfect. This poor performance is due to the extremely high dimensional state space, which translates to the extremely large belief space?B. We alleviate this problem by constructing a more suitable sampling distribution based on the observations that when the robot has active sensing capability, B can be partitioned into a collection of much smaller sub-spaces, and an optimal policy can often be generated by sufficient sampling of a small subset of the collection. Utilizing these observations, GCS samples B in two-stages, a subspace is sampled from the collection and then a belief is sampled from the subspace. It uses information from the set of sampled sub-spaces and sampled beliefs to guide subsequent sampling. Simulation results on marine robotics scenarios suggest that GCS can generate reasonable policies for motion planning problems with uncertain motion, sensing, and environment map, that are unsolvable by the best point-based POMDPs today. Furthermore, GCS handles POMDPs with continuous state, action, and observation spaces. We show that for a class of POMDPs that often occur in robot motion planning, given enough time, GCS converges to the optimal policy. To the best of our knowledge, this is the first convergence result for point-based POMDPs with continuous action space.