Cooperative multi-robot belief space planning for autonomous navigation in unknown environments

We investigate the problem of cooperative multi-robot planning in unknown environments, which is important in numerous applications in robotics. The research community has been actively developing belief space planning approaches that account for the different sources of uncertainty within planning, recently also considering uncertainty in the environment observed by planning time. We further advance the state of the art by reasoning about future observations of environments that are unknown at planning time. The key idea is to incorporate within the belief indirect multi-robot constraints that correspond to these future observations. Such a formulation facilitates a framework for active collaborative state estimation while operating in unknown environments. In particular, it can be used to identify best robot actions or trajectories among given candidates generated by existing motion planning approaches, or to refine nominal trajectories into locally optimal paths using direct trajectory optimization techniques. We demonstrate our approach in a multi-robot autonomous navigation scenario and consider its applicability for autonomous navigation in unknown obstacle-free and obstacle-populated environments. Results indicate that modeling future multi-robot interaction within the belief allows to determine robot actions (paths) that yield significantly improved estimation accuracy.     

Global motion planning under uncertain motion, sensing, and environment map

Uncertainty in motion planning is often caused by three main sources: motion error, sensing error, and imperfect environment map. Despite the significant effect of all three sources of uncertainty to motion planning problems, most planners take into account only one or at most two of them. We propose a new motion planner, called Guided Cluster Sampling (GCS), that takes into account all three sources of uncertainty for robots with active sensing capabilities. GCS uses the Partially Observable Markov Decision Process (POMDP) framework and the point-based POMDP approach. Although point-based POMDPs have shown impressive progress over the past few years, it performs poorly when the environment map is imperfect. This poor performance is due to the extremely high dimensional state space, which translates to the extremely large belief space?B. We alleviate this problem by constructing a more suitable sampling distribution based on the observations that when the robot has active sensing capability, B can be partitioned into a collection of much smaller sub-spaces, and an optimal policy can often be generated by sufficient sampling of a small subset of the collection. Utilizing these observations, GCS samples B in two-stages, a subspace is sampled from the collection and then a belief is sampled from the subspace. It uses information from the set of sampled sub-spaces and sampled beliefs to guide subsequent sampling. Simulation results on marine robotics scenarios suggest that GCS can generate reasonable policies for motion planning problems with uncertain motion, sensing, and environment map, that are unsolvable by the best point-based POMDPs today. Furthermore, GCS handles POMDPs with continuous state, action, and observation spaces. We show that for a class of POMDPs that often occur in robot motion planning, given enough time, GCS converges to the optimal policy. To the best of our knowledge, this is the first convergence result for point-based POMDPs with continuous action space.     

A decomposition approach for commodity pickup and delivery with time-windows under uncertainty

We consider a special class of large-scale, network-based, resource allocation problems under uncertainty, namely that of multi-commodity flows with time-windows under uncertainty. In this class, we focus on problems involving commodity pickup and delivery with time-windows. Our work examines methods of proactive planning, that is, robust plan generation to protect against future uncertainty. By a priori modeling uncertainties in data corresponding to service times, resource availability, supplies and demands, we generate solutions that are more robust operationally, that is, more likely to be executed or easier to repair when disrupted. We propose a novel modeling and solution framework involving a decomposition scheme that separates problems into a routing master problem and Scheduling Sub-Problems; and iterates to find the optimal solution. Uncertainty is captured in part by the master problem and in part by the Scheduling Sub-Problem. We present proof-of-concept for our approach using real data involving routing and scheduling for a large shipment carrier’s ground network, and demonstrate the improved robustness of solutions from our approach.     

Creating automated plans for Semantic Web applications through planning as model checking

The uncertainties of planning engendered by nondeterminism and partial observability have led to a melding of model checking and artificial intelligence. The result is planning as model checking. Because planning as model checking tests sets of states and sets of transitions at once, rather than single states, the method remains robust and viable in domains of large state spaces and varying levels of uncertainty.We develop a test bench for Semantic Web agents and use model-based planning to derive strong plans, strong cyclic plans, and weak plans. Our results suggest potential robustness and efficacy in devising plans for agent actions in the Semantic Web environment.     

Learned Stochastic Mobility Prediction for Planning with Control Uncertainty on Unstructured Terrain

Motion planning for planetary rovers must consider control uncertainty in order to maintain the safety of the platform during navigation. Modeling such control uncertainty is difficult due to the complex interaction between the platform and its environment. In this paper, we propose a motion‐planning approach whereby the outcome of control actions is learned from experience and represented statistically using a Gaussian process regression model. This mobility prediction model is trained using sample executions of motion primitives on representative terrain, and it predicts the future outcome of control actions on similar terrain. Using Gaussian process regression allows us to exploit its inherent measure of prediction uncertainty in planning. We integrate mobility prediction into a Markov decision process framework and use dynamic programming to construct a control policy for navigation to a goal region in a terrain map built using an onboard depth sensor. We consider both rigid terrain, consisting of uneven ground, small rocks, and nontraversable rocks, and also deformable terrain. We introduce two methods for training the mobility prediction model from either proprioceptive or exteroceptive observations, and we report results from nearly 300 experimental trials using a planetary rover platform in a Mars‐analogue environment. Our results validate the approach and demonstrate the value of planning under uncertainty for safe and reliable navigation.     

Exploiting submodular value functions for scaling up active perception

In active perception tasks, an agent aims to select sensory actions that reduce its uncertainty about one or more hidden variables. For example, a mobile robot takes sensory actions to efficiently navigate in a new environment. While partially observable Markov decision processes (POMDPs) provide a natural model for such problems, reward functions that directly penalize uncertainty in the agent’s belief can remove the piecewise-linear and convex (PWLC) property of the value function required by most POMDP planners. Furthermore, as the number of sensors available to the agent grows, the computational cost of POMDP planning grows exponentially with it, making POMDP planning infeasible with traditional methods. In this article, we address a twofold challenge of modeling and planning for active perception tasks. We analyze \(\rho \)POMDP and POMDP-IR, two frameworks for modeling active perception tasks, that restore the PWLC property of the value function. We show the mathematical equivalence of these two frameworks by showing that given a \(\rho \)POMDP along with a policy, they can be reduced to a POMDP-IR and an equivalent policy (and vice-versa). We prove that the value function for the given \(\rho \)POMDP (and the given policy) and the reduced POMDP-IR (and the reduced policy) is the same. To efficiently plan for active perception tasks, we identify and exploit the independence properties of POMDP-IR to reduce the computational cost of solving POMDP-IR (and \(\rho \)POMDP). We propose greedy point-based value iteration (PBVI), a new POMDP planning method that uses greedy maximization to greatly improve scalability in the action space of an active perception POMDP. Furthermore, we show that, under certain conditions, including submodularity, the value function computed using greedy PBVI is guaranteed to have bounded error with respect to the optimal value function. We establish the conditions under which the value function of an active perception POMDP is guaranteed to be submodular. Finally, we present a detailed empirical analysis on a dataset collected from a multi-camera tracking system employed in a shopping mall. Our method achieves similar performance to existing methods but at a fraction of the computational cost leading to better scalability for solving active perception tasks.     

Linear logic as a tool for planning under temporal uncertainty

The typical AI problem is that of making a plan of the actions to be performed by a controller so that it could get into a set of final situations, if it started with a certain initial situation.The plans, and related winning strategies, happen to be finite in the case of a finite number of states and a finite number of instant actions.The situation becomes much more complex when we deal with planning under temporal uncertainty caused by actions with delayed effects.Here we introduce a tree-based formalism to express plans, or winning strategies, in finite state systems in which actions may have quantitatively delayed effects. Since the delays are non-deterministic and continuous, we need an infinite branching to display all possible delays. Nevertheless, under reasonable assumptions, we show that infinite winning strategies which may arise in this context can be captured by finite plans.The above planning problem is specified in logical terms within a Horn fragment of affine logic. Among other things, the advantage of linear logic approach is that we can easily capture ‘preemptive/anticipative’ plans (in which a new action β may be taken at some moment within the running time of an action α being carried out, in order to be prepared before completion of action α).In this paper we propose a comprehensive and adequate logical model of strong planning under temporal uncertainty which addresses infinity concerns. In particular, we establish a direct correspondence between linear logic proofs and plans, or winning strategies, for the actions with quantitative delayed effects.     

Existential assertions and quantum levels on the tree of the situation calculus

In a seminal paper, Reiter introduced a variant of the situation calculus along with a set of its properties. To the best of our knowledge, one of these properties has remained unproved and ignored despite its relevance to the planning problem and the expressivity of the theories of actions. We state this property in a more general form and provide its proof. Intuitively, whenever a theory of actions entails that there exists a situation satisfying a first order formula (e.g., a goal), at least one such situation must be found within a predetermined distance from the initial situation. This distance is finite and the same in all the models of the theory, since it depends only on the theory and the formula at hand.     

Active object tracking using context estimation: handling occlusions and detecting missing targets

When performing visual servoing or object tracking tasks, active sensor planning is essential to keep targets in sight or to relocate them when missing. In particular, when dealing with a known target missing from the sensor’s field of view, we propose using prior knowledge related to contextual information to estimate its possible location. To this end, this study proposes a Dynamic Bayesian Network that uses contextual information to effectively search for targets. Monte Carlo particle filtering is employed to approximate the posterior probability of the target’s state, from which uncertainty is defined. We define the robot’s utility function via information theoretic formalism as seeking the optimal action which reduces uncertainty of a task, prompting robot agents to investigate the location where the target most likely might exist. Using a context state model, we design the agent’s high-level decision framework using a Partially-Observable Markov Decision Process. Based on the estimated belief state of the context via sequential observations, the robot’s navigation actions are determined to conduct exploratory and detection tasks. By using this multi-modal context model, our agent can effectively handle basic dynamic events, such as obstruction of targets or their absence from the field of view. We implement and demonstrate these capabilities on a mobile robot in real-time.

     

Verifying plans for multiple agents

Abstract

Research in distributed artificial intelligence planning has historically focused on two distinct classes of problems. One paradigm has been that of 'planning for multiple agents', which considers issues inherent in centrally directed multi-agent execution. The second paradigm has been 'distributed planning', where multiple agents more autonomously participate in coordinating and deciding upon their own actions. The work described in this paper is in the first category, planning for multiple agents. Taking the STRIPS representation of actions, and directed acrylic graphs (DAGs) as plan representations particularly well suited to parallel execution, it formally analyses the following question: how can a DAG plan be verified (i.e. how can we be sure such a plan will be correct, given our uncertainty about exactly when unconstrained parallel actions will be performed)? A method is presented for verifying the correctness of plans for multiple agents, represented as DAGs. The technique allows for the efficient analysis of a plan, despite its many potential execution histories.     

A model of reaction for planning in dynamic environments

One shortcoming with most AI planning systems has been an inability to deal with execution-time discrepancies between actual and expected situations. Often, these exception situations jeopardize the immediate integrity and safety of the planning agent or its surroundings, with the only recourse being more time-consuming plan generation. In order to avoid such situations, potential exceptions must be predicted during plan execution. Since many application domains (particularly for autonomous systems) are inherently dynamic — in the sense that information is at best incomplete, perhaps erroneous, and changes over time independent of a planning agent's actions — managing action in the world becomes a difficult problem. Action and events in dynamic worlds must be monitored in order to coordinate an agent's actions with its surroundings. This allows the agent to predict and plan for potential future exception situations while acting in the present.This paper introduces an approach to autonomous reaction in dynamic environments. We have avoided the traditional distinction between generating and then executing plans through the use of a dynamic reaction system, which handles potential exception situations gracefully as it carries out assigned tasks. The reaction system manages constraints imposed by ongoing activity in the world, as well as those derived from long-term planning, to control observable behaviour. This approach provides the necessary stimulus/response behaviour required in dynamic situations, while using goal-directed constraints as heuristics for improved reactions.We present an overview of the salient features of dynamic worlds and their impact on traditional planning, introduce our model of dynamic reactivity, describe an implementation of the model and its performance in a dynamic simulation environment, and present an architecture incorporating long-term planning with short-term reactance suitable for autonomous systems applications.     

Real-time hierarchical POMDPs for autonomous robot navigation

This paper proposes a new hierarchical formulation of POMDPs for autonomous robot navigation that can be solved in real-time, and is memory efficient. It will be referred to in this paper as the Robot Navigation–Hierarchical POMDP (RN-HPOMDP). The RN-HPOMDP is utilized as a unified framework for autonomous robot navigation in dynamic environments. As such, it is used for localization, planning and local obstacle avoidance. Hence, the RN-HPOMDP decides at each time step the actions the robot should execute, without the intervention of any other external module for obstacle avoidance or localization. Our approach employs state space and action space hierarchy, and can effectively model large environments at a fine resolution. Finally, the notion of the reference POMDP is introduced. The latter holds all the information regarding motion and sensor uncertainty, which makes the proposed hierarchical structure memory efficient and enables fast learning. The RN-HPOMDP has been experimentally validated in real dynamic environments.     

INTEGRATING PLANNING,EXECUTION, AND LEARNING TO IMPROVE PLAN EXECUTION

Algorithms for planning under uncertainty require accurate action models that explicitly capture the uncertainty of the environment. Unfortunately, obtaining these models is usually complex. In environments with uncertainty, actions may produce countless outcomes and hence, specifying them and their probability is a hard task. As a consequence, when implementing agents with planning capabilities, practitioners frequently opt for architectures that interleave classical planning and execution monitoring following a replanning when failure paradigm. Though this approach is more practical, it may produce fragile plans that need continuous replanning episodes or even worse, that result in execution dead‐ends. In this paper, we propose a new architecture to relieve these shortcomings. The architecture is based on the integration of a relational learning component and the traditional planning and execution monitoring components. The new component allows the architecture to learn probabilistic rules of the success of actions from the execution of plans and to automatically upgrade the planning model with these rules. The upgraded models can be used by any classical planner that handles metric functions or, alternatively, by any probabilistic planner. This architecture proposal is designed to integrate off‐the‐shelf interchangeable planning and learning components so it can profit from the last advances in both fields without modifying the architecture.     

Generating effective tests for concurrent programs via AI automated planning techniques

Testing concurrent programs is a challenging problem due to interleaving explosion: even for a fixed set of inputs, there is a huge number of concurrent runs that need to be tested to account for scheduler behavior. Testing all possible schedules is not practical. Consequently, most effective testing algorithms only test a select subset of runs. For example, limiting testing to runs that contain data races or atomicity violations has been shown to capture a large proportion of concurrency bugs. In this paper we present a general approach to concurrent program testing that is based on techniques from artificial intelligence (AI) automated planning. We propose a framework for predicting concurrent program runs that violate a collection of generic correctness specifications for concurrent programs, namely runs that contain data races, atomicity violations, or null-pointer dereferences. Our prediction is based on observing an arbitrary run of the program, and using information collected from this run to model the behavior of the program, and to predict new runs that contain bugs with one of the above noted violation patterns. We characterize the problem of predicting such new runs as an AI sequential planning problem with the temporally extended goal of achieving a particular violation pattern. In contrast to many state-of-the-art approaches, in our approach feasibility of the predicted runs is guaranteed and, therefore, all generated runs are fully usable for testing. Moreover, our planning-based approach has the merit that it can easily accommodate a variety of violation patterns which serve as the selection criteria for guiding search in the state space of concurrent runs. This is achieved by simply modifying the planning goal. We have implemented our approach using state-of-the-art AI planning techniques and tested it within the Penelope concurrent program testing framework [35]. Nevertheless, the approach is general and is amenable to a variety of program testing frameworks. Our experiments with a benchmark suite showed that our approach is very fast and highly effective, finding all known bugs.     

Opportunistic sampling-based active visual SLAM for underwater inspection

This paper reports on an active SLAM framework for performing large-scale inspections with an underwater robot. We propose a path planning algorithm integrated with visual SLAM that plans loop-closure paths in order to decrease navigation uncertainty. While loop-closing revisit actions bound the robot’s uncertainty, they also lead to redundant area coverage and increased path length. Our proposed opportunistic framework leverages sampling-based techniques and information filtering to plan revisit paths that are coverage efficient. We employ Gaussian process regression for modeling the prediction of camera registrations and use a two-step optimization procedure for selecting revisit actions. We show that the proposed method offers many benefits over existing solutions and good performance for bounding navigation uncertainty in long-term autonomous operations with hybrid simulation experiments and real-world field trials performed by an underwater inspection robot.     

Illustrative uncertainty visualization of DTI fiber pathways

Diffusion Tensor Imaging (DTI) and fiber tracking provide unique insight into the 3D structure of fibrous tissues in the brain. However, the output of fiber tracking contains a significant amount of uncertainty accumulated in the various steps of the processing pipeline. Existing DTI visualization methods do not present these uncertainties to the end-user. This creates a false impression of precision and accuracy that can have serious consequences in applications that rely heavily on risk assessment and decision-making, such as neurosurgery. On the other hand, adding uncertainty to an already complex visualization can easily lead to information overload and visual clutter. In this work, we propose Illustrative Confidence Intervals to reduce the complexity of the visualization and present only those aspects of uncertainty that are of interest to the user. We look specifically at the uncertainty in fiber shape due to noise and modeling errors. To demonstrate the flexibility of our framework, we compute this uncertainty in two different ways, based on (1) fiber distance and (2) the probability of a fiber connection between two brain regions. We provide the user with interactive tools to define multiple confidence intervals, specify visual styles and explore the uncertainty with a Focus+Context approach. Finally, we have conducted a user evaluation with three neurosurgeons to evaluate the added value of our visualization.     

Collaborative Pushing and Grasping of Tightly Stacked Objects via Deep Reinforcement Learning

Directly grasping the tightly stacked objects may cause collisions and result in failures,degenerating the functionality of robotic arms.Inspired by the observation that first pushing objects to a state of mutual separation and then grasping them individually can effectively increase the success rate,we devise a novel deep Q-learning framework to achieve collaborative pushing and grasping.Specifically,an efficient non-maximum suppression policy(PolicyNMS)is proposed to dynamically evaluate pushing and grasping actions by enforcing a suppression constraint on unreasonable actions.Moreover,a novel data-driven pushing reward network called PR-Net is designed to effectively assess the degree of separation or aggregation between objects.To benchmark the proposed method,we establish a dataset containing common household items dataset(CHID)in both simulation and real scenarios.Although trained using simulation data only,experiment results validate that our method generalizes well to real scenarios and achieves a 97%grasp success rate at a fast speed for object separation in the real-world environment.     

On-line economic optimization of energy systems using weather forecast information

We establish an on-line optimization framework to exploit weather forecast information in the operation of energy systems. We argue that anticipating the weather conditions can lead to more proactive and cost-effective operations. The framework is based on the solution of a stochastic dynamic real-time optimization (D-RTO) problem incorporating forecasts generated from a state-of-the-art weather prediction model. The necessary uncertainty information is extracted from the weather model using an ensemble approach. The accuracy of the forecast trends and uncertainty bounds are validated using real meteorological data. We present a numerical simulation study in a building system to demonstrate the developments.