Sequential Monte Carlo in reachability heuristics for probabilistic planning

Some of the current best conformant probabilistic planners focus on finding a fixed length plan with maximal probability. While these approaches can find optimal solutions, they often do not scale for large problems or plan lengths. As has been shown in classical planning, heuristic search outperforms bounded length search (especially when an appropriate plan length is not given a priori). The problem with applying heuristic search in probabilistic planning is that effective heuristics are as yet lacking.In this work, we apply heuristic search to conformant probabilistic planning by adapting planning graph heuristics developed for non-deterministic planning. We evaluate a straight-forward application of these planning graph techniques, which amounts to exactly computing a distribution over many relaxed planning graphs (one planning graph for each joint outcome of uncertain actions at each time step). Computing this distribution is costly, so we apply Sequential Monte Carlo (SMC) to approximate it. One important issue that we explore in this work is how to automatically determine the number of samples required for effective heuristic computation. We empirically demonstrate on several domains how our efficient, but sometimes suboptimal, approach enables our planner to solve much larger problems than an existing optimal bounded length probabilistic planner and still find reasonable quality solutions.     

Conformant planning via heuristic forward search: A new approach

Conformant planning is the task of generating plans given uncertainty about the initial state and action effects, and without any sensing capabilities during plan execution. The plan should be successful regardless of which particular initial world we start from. It is well known that conformant planning can be transformed into a search problem in belief space, the space whose elements are sets of possible worlds. We introduce a new representation of that search space, replacing the need to store sets of possible worlds with a need to reason about the effects of action sequences. The reasoning is done by implication tests on propositional formulas in conjunctive normal form (CNF) that capture the action sequence semantics. Based on this approach, we extend the classical heuristic forward-search planning system FF to the conformant setting. The key to this extension is an appropriate extension of the relaxation that underlies FF's heuristic function, and of FF's machinery for solving relaxed planning problems: the extended machinery includes a stronger form of the CNF implication tests that we use to reason about the effects of action sequences. Our experimental evaluation shows the resulting planning system to be superior to the state-of-the-art conformant planners MBP, KACMBP, and GPT in a variety of benchmark domains.     

基于模态逻辑D公理系统的Conformant规划方法

2006年,conformant规划问题成为国际规划竞赛不确定性问题域中的标准测试问题,得到研究人员的广泛关注.目前,conformant规划系统都是将其看成信念状态空间上的启发式搜索问题予以求解.通过分析conformant规划问题的语法和语义,提出新的基于模态逻辑的规划框架.将其转换为模态逻辑D公理系统的一系列定理证明问题.提出2种基于模态逻辑的编码方式.构造相应的公理与推理规则形成模态公式集,保证对于D系统的定理证明过程等同于原问题的规划过程.并通过问题实例验证该方法的有效性.继基于SAT、CSP、线性规划、模型检测等求解技术的规划方法后,该规划框架是基于转换的规划方法的一种新的尝试.     

Parallel planning via the distribution of operators

This paper describes Operator Distribution Method for parallel Planning (ODMP), a parallelization method for efficient heuristic planning. The method innovates in that it parallelizes the application of the available operators to the current state and the evaluation of the successor states using the heuristic function. In order to achieve better load balancing and a lift in the scalability of the algorithm, the operator set is initially enlarged, by grounding the first argument of each operator. Additional load balancing is achieved through the reordering of the operator set, based on the expected amount of imposed work. ODMP is effective for heuristic planners, but it can be applied to planners that embody other search strategies as well. It has been applied to GRT, a domain-independent heuristic planner, and CL, a heuristic planner for simple logistics problems, and has been thoroughly tested on a set of logistics problems adopted from the AIPS-98 planning competition, giving quite promising results.     

一致性规划研究

针对一致性规划的高度求解复杂度,分析主流一致性规划器的求解策略,给出影响一致性规划器性能的主要因素：启发信息的有效性,信念状态表示方法的紧凑性和最终问题求解机制的效率。分析信念状态的表示方法和相应的求解机制,并比较不同表示方法在不同条件下的优劣。讨论一致性规划的未来研究方向和发展趋势。     

A case-based approach to heuristic planning

Most of the great success of heuristic search as an approach to AI Planning is due to the right design of domain-independent heuristics. Although many heuristic planners perform reasonably well, the computational cost of computing the heuristic function in every search node is very high, causing the planner to scale poorly when increasing the size of the planning tasks. For tackling this problem, planners can incorporate additional domain-dependent heuristics in order to improve their performance. Learning-based planners try to automatically acquire these domain-dependent heuristics using previous solved problems. In this work, we present a case-based reasoning approach that learns abstracted state transitions that serve as domain control knowledge for improving the planning process. The recommendations from the retrieved cases are used as guidance for pruning or ordering nodes in different heuristic search algorithms applied to planning tasks. We show that the CBR guidance is appropriate for a considerable number of planning benchmarks.     

Landmark-based heuristic online contingent planning

In contingent planning problems, agents have partial information about their state and use sensing actions to learn the value of some variables. When sensing and actuation are separated, plans for such problems can often be viewed as a tree of sensing actions, separated by conformant plans consisting of non-sensing actions that enable the execution of the next sensing action. We propose a heuristic, online method for contingent planning which focuses on identifying the next useful sensing action. We select the next sensing action based on a landmark heuristic, adapted from classical planning. We discuss landmarks for plan trees, providing several alternative definitions and discussing their merits. The key part of our planner is the novel landmarks-based heuristic, together with a projection method that uses classical planning to solve the intermediate conformant planning problems. The resulting heuristic contingent planner solves many more problems than state-of-the-art, translation-based online contingent planners, and in most cases, much faster, up to 3 times faster on simple problems, and 200 times faster on non-simple domains.     

基于因果图启发式的并行概率规划求解

并行概率规划(PPP)是近年来智能规划领域中的研究热点。在该类问题中,动作具有并发性和不确定性,非常贴近现实问题。然而,现有的两种针对PPP的主要求解方法都有明显的缺点。一种基于模拟抽样,以规划器PROST为代表,但求解速度慢;另一种基于迭代深化,以规划器Glutton为代表,但求解质量差。因此,我们尝试使用高效的启发式搜索方法来求解这类问题。目前,因果图启发式(CGH)是启发式规划方法中的佼佼者。考虑到PPP问题采用RDDL语言来描述,其中的条件概率函数(CPF)非常适合用于构建因果图(CG),所以我们引入因果图来对基于RDDL描述的PPP问题进行启发式求解。本文的主要启发式算法称为CGH_RDDL,整体求解方法是使用rddlsim模拟状态演化以及用CGH_RDDL引导搜索。具体做法是：先从领域描述构建出因果图及领域转换图(DTG);然后根据CG和DTG,计算单个状态变量任意一对取值间的转换代价;接着在rddlsim的模拟演化过程中,由CGH_RDDL推送具有最佳估值的后继状态,其中状态的启发值定义为状态轨迹的转换代价和立即回报值的总和;最后累加在限定轮数内rddlsim状态演化的回报值,即为最终的求解质量。在PPP基准领域上的实验结果表明,在不允许手工干预和参数调整的前提下,本文方法的求解效果要好于PROST和Glutton。更进一步地,与其它的基本启发式相比,CGH_RDDL的求解质量高于随机搜索,求解速度快于爬山法。这表明在经典规划领域中高效的启发式搜索策略可扩展去求解这一类非经典规划问题。而非经典规划问题更具有现实意义和应用前景,更值得探讨先进的规划方法去求解它们。     

Applications of parallel processing technologies in heuristic search planning: methodologies and experiments

The goal of this paper is to investigate the application of parallel programming techniques to boost the performance of heuristic search‐based planning systems in various aspects. It shows that an appropriate parallelization of a sequential planning system often brings gain in performance and/or scalability. We start by describing general schemes for parallelizing the construction of a plan. We then discuss the applications of these techniques to two domain‐independent heuristic search‐based planners—a competitive conformant planner (CP A) and a state‐of‐the‐art classical planner (FF). We present experimental results—on both shared memory and distributed memory platforms—which show that the performance improvements and scalability are obtained in both cases. Finally, we discuss the issues that should be taken into consideration when designing a parallel planning system and relate our work to the existing literature. Copyright © 2009 John Wiley & Sons, Ltd.     

HYBRID ACE: COMBINING SEARCH DIRECTIONS FOR HEURISTIC PLANNING

One of the most promising trends in Domain-Independent AI Planning, nowadays, is state-space heuristic planning. The planners of this category construct general but efficient heuristic functions, which are used as a guide to traverse the state space either in a forward or in a backward direction. Although specific problems may favor one or the other direction, there is no clear evidence why any of them should be generally preferred. This paper presents Hybrid-AcE, a domain-independent planning system that combines search in both directions utilizing a complex criterion that monitors the progress of the search, to switch between them. Hybrid AcE embodies two powerful domain-independent heuristic functions extending one of the AcE planning systems. Moreover, the system is equipped with a fact-ordering technique and two methods for problem simplification that limit the search space and guide the algorithm to the most promising states. The bi-directional system has been tested on a variety of problems adopted from the AIPS planning competitions with quite promising results.     

An approach to efficient planning with numerical fluents and multi-criteria plan quality

Dealing with numerical information is practically important in many real-world planning domains where the executability of an action can depend on certain numerical conditions, and the action effects can consume or renew some critical continuous resources, which in pddl can be represented by numerical fluents. When a planning problem involves numerical fluents, the quality of the solutions can be expressed by an objective function that can take different plan quality criteria into account.We propose an incremental approach to automated planning with numerical fluents and multi-criteria objective functions for pddl numerical planning problems. The techniques in this paper significantly extend the framework of planning with action graphs and local search implemented in the lpg planner. We define the numerical action graph (NA-graph) representation for numerical plans and we propose some new local search techniques using this representation, including a heuristic search neighborhood for NA-graphs, a heuristic evaluation function based on relaxed numerical plans, and an incremental method for plan quality optimization based on particular search restarts. Moreover, we analyze our approach through an extensive experimental study aimed at evaluating the importance of some specific techniques for the performance of the approach, and at analyzing its effectiveness in terms of fast computation of a valid plan and quality of the best plan that can be generated within a given CPU-time limit. Overall, the results show that our planner performs quite well compared to other state-of-the-art planners handling numerical fluents.     

一种优化的一致性规划状态变量选择算法

研究了一致性规划任务信念状态空间的表示方法。针对一致性有限域表示(CPT-FDR)算法在任务生成阶段选择状态变量的不足,提出了一种基于初始状态中文字相容互斥的状态变量选择算法——MECV算法。CPT-FDR未考虑初始信念状态中文字的互斥性,产生冗余的编码信息,降低了编码的效率。MECV算法利用有用正负文字构造新的未覆盖事实集,提取初始信念状态中处于不同世界状态的文字组成互斥组,再编码状态变量。实验结果表明该算法能有效地压缩信念状态空间。     

FMAP: Distributed cooperative multi-agent planning

This paper proposes FMAP (Forward Multi-Agent Planning), a fully-distributed multi-agent planning method that integrates planning and coordination. Although FMAP is specifically aimed at solving problems that require cooperation among agents, the flexibility of the domain-independent planning model allows FMAP to tackle multi-agent planning tasks of any type. In FMAP, agents jointly explore the plan space by building up refinement plans through a complete and flexible forward-chaining partial-order planner. The search is guided by h _{D T G} , a novel heuristic function that is based on the concepts of Domain Transition Graph and frontier state and is optimized to evaluate plans in distributed environments. Agents in FMAP apply an advanced privacy model that allows them to adequately keep private information while communicating only the data of the refinement plans that is relevant to each of the participating agents. Experimental results show that FMAP is a general-purpose approach that efficiently solves tightly-coupled domains that have specialized agents and cooperative goals as well as loosely-coupled problems. Specifically, the empirical evaluation shows that FMAP outperforms current MAP systems at solving complex planning tasks that are adapted from the International Planning Competition benchmarks.     

Causal graphs and structurally restricted planning

The causal graph is a directed graph that describes the variable dependencies present in a planning instance. A number of papers have studied the causal graph in both practical and theoretical settings. In this work, we systematically study the complexity of planning restricted by the causal graph. In particular, any set of causal graphs gives rise to a subcase of the planning problem. We give a complete classification theorem on causal graphs, showing that a set of graphs is either polynomial-time tractable, or is not polynomial-time tractable unless an established complexity-theoretic assumption fails; our theorem describes which graph sets correspond to each of the two cases. We also give a classification theorem for the case of reversible planning, and discuss the general direction of structurally restricted planning.     

Experimental evaluation of pheromone models in ACOPlan

In this paper the system ACOPlan for planning with non uniform action cost is introduced and analyzed. ACOPlan is a planner based on the ant colony optimization framework, in which a colony of planning ants searches for near optimal solution plans with respect to an overall plan cost metric. This approach is motivated by the strong similarity between the process used by artificial ants to build solutions and the methods used by state?Cbased planners to search solution plans. Planning ants perform a stochastic and heuristic based search by interacting through a pheromone model. The proposed heuristic and pheromone models are presented and compared through systematic experiments on benchmark planning domains. Experiments are also provided to compare the quality of ACOPlan solution plans with respect to state of the art satisficing planners. The analysis of the results confirm the good performance of the Action?CAction pheromone model and points out the promising performance of the novel Fuzzy?CLevel?CAction pheromone model. The analysis also suggests general principles for designing performant pheromone models for planning and further extensions of ACOPlan to other optimization models.     

Hybrid ant colony algorithms for path planning in sparse graphs

The general problem of path planning can be modeled as a traveling salesman problem which assumes that a graph is fully connected. Such a scenario of full connectivity is however not always realistic. One such motivating example for us is the application of path planning for unmanned reconnaissance aerial vehicles (URAVs). URAVs are widely deployed for photography or imagery gathering missions of sites of interest. These sites can be targets in a combat zone to be investigated or sites inaccessible by ground transportation, such as those hit by forest fires, earthquake or other forms of natural disasters. The navigation environment is one where the overall configuration of the problem is a sparse graph. Unlike graphs that are fully connected, sparse graphs are not always Hamiltonian. In this paper, we describe hybrid ant colony algorithms (HACAs) proposed for path planning in sparse graphs since existing ant colony solvers designed for solving TSP do not apply to the present context directly. HACAs represent ant inspired algorithms incorporated with a local search procedure and some heuristic techniques for uncovering feasible route(s) or path(s) in a sparse graph within tractable time. Empirical results conducted on a set of generated sparse graphs demonstrate the excellent convergence property and robustness of HACAs in uncovering low risk and Hamiltonian visitation paths. Further, the obtained results also indicate that HACAs converge to secondary closed paths in situations where a Hamiltonian cycle does not exist theoretically or is not attainable within the bounded computational time window.     

Sampling‐based hierarchical motion planning for a reconfigurable wheel‐on‐leg planetary analogue exploration rover

Reconfigurable mobile planetary rovers are versatile platforms that may safely traverse cluttered environments by morphing their physical geometry. Planning paths for these adaptive robots is challenging due to their many degrees of freedom, and the need to consider potentially continuous platform reconfiguration along the length of the path. We propose a novel hierarchical structure for asymptotically optimal (AO) sampling‐based planners and specifically apply it to the state‐of‐the‐art Fast Marching Tree (FMT*) AO planner. Our algorithm assumes a decomposition of the full configuration space into multiple subspaces, and begins by rapidly finding a set of paths through one such subspace. This set of solutions is used to generate a biased sampling distribution, which is then explored to find a solution in the full configuration space. This technique provides a novel way to incorporate prior knowledge of subspaces to efficiently bias search within existing AO sampling‐based planners. Importantly, probabilistic completeness and asymptotic optimality are preserved. Experimental results in simulation are provided that benchmark the algorithm against state‐of‐the‐art sampling‐based planners without the hierarchical variation. Additional experimental results performed with a physical wheel‐on‐leg platform demonstrate application to planetary rover mobility and showcase how constraints such as actuator failures and sensor pointing may be easily incorporated into the planning problem. In minimizing an energy objective that combines an approximation of the mechanical work required for platform locomotion with that required for reconfiguration, the planner produces intuitive behaviors where the robot dynamically adjusts its footprint, varies its height, and clambers over obstacles using legged locomotion. These results illustrate the generality of the planner in exploiting the platform's mechanical ability to fluidly transition between various physical geometric configurations, and wheeled/legged locomotion modes, without the need for predefined configurations.     

Planning as Propositional CSP: From Walksat to Local Search Techniques for Action Graphs

Graphplan-style of planning can be formulated as an incremental propositional CSP where the (boolean) variables correspond to operator instantiations (actions) that are or are not scheduled at certain time steps. In this paper we present a framework for solving this class of propositional CSPs using local search in planning graphs. The search space consists of particular subgraphs of a planning graph corresponding to (complete) variable assignments, and representing partial plans. The operators for moving from one search state to the next one are graph modifications corresponding to revisions of the current variable assignment (partial plan), or to an extension of the represented CSP.Our techniques are implemented in a planner called LPG using various types of heuristics based on a parametrized objective function, where the parameters weight different constraint violations, and are dynamically evaluated using Lagrange multipliers. LPG's basic heuristic was inspired by Walksat, which in Kautz and Selman's Blackbox can be used to solve the SAT-encoding of a planning graph. An advantage of LPG is that its heuristics exploit the structure of the planning graph, while Blackbox relies on general heuristics for SAT-problems, and requires the translation of the planning graph into propositional clauses. Another major difference is that LPG can handle action execution costs to produce good quality plans. This is achieved by an anytime process minimizing an objective function based on the number of constraint violations in a plan and on its overall cost. Experimental results illustrate the efficiency of our approach, showing, in particular, that LPG is significantly faster than Blackbox and other planners based on planning graphs.     

STABULUS: A technique for finding stable sets in large graphs with tabu search

Numerical experiments with tabu search have been carried out for constructing independent sets in large graphs. We present some variations on the independent set problem and discuss the results obtained by the tabu search technique. As for graph coloring, this method seems to be a very efficient heuristic procedure.