A cooperative pursuit-evasion game in wireless sensor and actor networks

This paper studies the problem of the pursuit-evasion game under the wireless sensor and actor networks (WSANs). In order to plan paths for pursuers to capture an evader in the pursuit-evasion game, a novel multi-step cooperative strategy is presented. Under this strategy, the pursuit-evasion game is studied in two stages. In the first stage we assume that the evader is always static in the workplace, and in the second stage the evader will move once it senses the existence of pursuers. A Daisy-Chain Formation algorithm and a sliding mode-based method are presented to control the pursuit. Based on Lyapunov stability theory, the proposed algorithm is proved to be convergent. At last, simulation results are provided to demonstrate the effectiveness of the proposed method.     

Equilibrium Strategy of the Pursuit-Evasion Game in Three-Dimensional Space

The pursuit-evasion game models the strategic interaction among players, attracting attention in many realistic scenarios, such as missile guidance, unmanned aerial vehicles, and target defense. Existing studies mainly concentrate on the cooperative pursuit of multiple players in two-dimensional pursuit-evasion games. However, these approaches can hardly be applied to practical situations where players usually move in three-dimensional space with a three-degree-of-freedom control. In this paper, we make the first attempt to investigate the equilibrium strategy of the realistic pursuit-evasion game, in which the pursuer follows a three-degree-of-freedom control, and the evader moves freely. First, we describe the pursuer’s three-degree-of-freedom control and the evader’s relative coordinate. We then rigorously derive the equilibrium strategy by solving the retrogressive path equation according to the Hamilton-Jacobi-Bellman-Isaacs (HJBI) method, which divides the pursuit-evasion process into the navigation and acceleration phases. Besides, we analyze the maximum allowable speed for the pursuer to capture the evader successfully and provide the strategy with which the evader can escape when the pursuer’s speed exceeds the threshold. We further conduct comparison tests with various unilateral deviations to verify that the proposed strategy forms a Nash equilibrium.

     

On Discrete-Time Pursuit-Evasion Games With Sensing Limitations

In this paper, we address discrete-time pursuit-evasion games in the plane where every player has identical sensing and motion ranges restricted to closed disks of given sensing and stepping radii. A single evader is initially located inside a bounded subset of the environment and does not move until detected. We propose a sweep-pursuit-capture pursuer strategy to capture the evader and apply it to two variants of the game. The first involves a single pursuer and an evader in a bounded convex environment, and the second involves multiple pursuers and an evader in a boundaryless environment. In the first game, we give a sufficient condition on the ratio of sensing to stepping radius of the players that guarantees capture. In the second, we determine the minimum probability of capture, which is a function of a novel pursuer formation and independent of the initial evader location. The sweep and pursuit phases reduce both games to previously studied problems with unlimited range sensing, and capture is achieved using available strategies. We obtain novel upper bounds on the capture time and present simulation studies that address the performance of the strategies under sensing errors, different ratios of sensing to stepping radius, greater evader speed, and a different number of pursuers.      

Randomized pursuit-evasion in a polygonal environment

This paper contains two main results. First, we revisit the well-known visibility-based pursuit-evasion problem, and show that in contrast to deterministic strategies, a single pursuer can locate an unpredictable evader in any simply connected polygonal environment, using a randomized strategy. The evader can be arbitrarily faster than the pursuer, and it may know the position of the pursuer at all times, but it does not have prior knowledge of the random decisions made by the pursuer. Second, using the randomized algorithm, together with the solution to a problem called the "lion and man problem" as subroutines, we present a strategy for two pursuers (one of which is at least as fast as the evader) to quickly capture an evader in a simply connected polygonal environment. We show how this strategy can be extended to obtain a strategy for a polygonal room with a door, two pursuers who have only line-of-sight communication, and a single pursuer (at the expense of increased capture time).     

A cooperative Homicidal Chauffeur game

We address a pursuit-evasion problem involving an unbounded planar environment, a single evader and multiple pursuers moving along curves of bounded curvature. The problem amounts to a multi-agent version of the classic Homicidal Chauffeur problem; we identify parameter ranges in which a single pursuer is not sufficient to guarantee evader capture. We propose a novel multi-phase cooperative strategy in which the pursuers move in specific formations and confine the evader to a bounded region. The proposed strategy is inspired by the hunting and foraging behaviors of various fish species. We characterize the required number of pursuers for which our strategy is guaranteed to lead to confinement.     

Evaluation of the effectiveness of open-loop evasion strategies in a pursuit-evasion problem

A stochastic pursuit-evasion optimal control problem in the (X, z)-plane is considered. Owing to thrust, drag and gravitational forces, both players have variable speeds. The pursuer applies a feedback pursuit strategy whereas the evader applies an open-loop evasion strategy. By reducing the state-space of the encounter, a method is proposed for evaluating the effectiveness of open-loop evasion strategies.     

多追捕者-单-逃跑者追逃问题实现成功捕获的约束条件

针对包含有n个追捕者及1个逃跑者的2维平面多机器人追逃问题,对实现成功捕获的约束条件进行了研究.经过理论分析得出:在机器人拥有全局视野的情况下,即使单一逃跑者性能优于每个追捕者,只要满足追捕者与逃跑者的速率比大于sin(π/n),逃跑机器人落在追捕机器人所构成的凸多边形内部且逃跑者和追捕者构成的相邻追-逃阿波罗尼奥斯圆满足两两相交(相切)这2个约束条件,则追捕者通过选择合适的追捕策略就一定可以实现成功抓捕.此外,还给出了在此约束条件下的追捕者和逃跑者的追逃策略.多组仿真实验同样证明了本文提出的约束条件是正确的.     

Scalable and practical pursuit-evasion with networked robots

In this paper, we consider the design and implementation of practical pursuit-evasion games with networked robots, where a communication network provides sensing-at-a-distance as well as a communication backbone that enables tighter coordination between pursuers. We first develop, using the theory of zero-sum games, an algorithm that computes the minimal completion time strategy for pursuit-evasion when pursuers and evaders have same speed, and when all players make optimal decisions based on complete knowledge. Then, we extend this algorithm to when evader are significantly faster than pursuers. Unfortunately, these algorithms do not scale beyond a small number of robots. To overcome this problem, we design and implement a partition algorithm where pursuers capture evaders by decomposing the game into multiple multi-pursuer single-evader games. We show that the partition algorithm terminates, has bounded capture time, is robust, and is scalable in the number of robots. We then describe the design of a real-world mobile robot-based pursuit evasion game. We validate our algorithms by experiments in a moderate-scale testbed in a challenging office environment. Overall, our work illustrates an innovative interplay between robotics and communication.     

Evasion strategies of a three-player lifeline game

This study examines a multi-player pursuit-evasion game, more specifically, a three-player lifeline game in a planar environment, where a single evader is tasked with reaching a lifeline prior to capture. A decomposition method based on an explicit policy is proposed to address the game qualitatively from two main aspects: (1) the evader’s position distribution to guarantee winning the game (i.e., the escape zone), which is based on the premise of knowing the pursuers’ positions initially, and (2) evasion strategies in the escape zone. First, this study decomposes the three-player lifeline game into two two-player sub-games and obtains an analytic expression of the escape zone by constructing a barrier, which is an integration of the solutions of two sub-games. This study then explicitly partitions the escape zone into several regions and derives an evasion strategy for each region. In particular, this study provides a resultant force method for the evader to balance the active goal of reaching the lifeline and the passive goal of avoiding capture. Finally, some examples from a lifeline game involving more than one pursuer are used to verify the effectiveness and scalability of the evasion strategies.     

Game and Information Theory Analysis of Electronic Countermeasures in Pursuit-Evasion Games

Two-player pursuit-evasion games in the literature typically either assume both players have perfect knowledge of the opponent's positions or use primitive sensing models. This unrealistically skews the problem in favor of the pursuer who needs only maintain a faster velocity at all turning radii. In real life, an evader usually escapes when the pursuer no longer knows the evader's position. In our previous work, we modeled pursuit evasion without perfect information as a two-player bimatrix game by using a realistic sensor model and information theory to compute game-theoretic payoff matrices. That game has a saddle point when the evader uses strategies that exploit sensor limitations, whereas the pursuer relies on strategies that ignore the sensing limitations. In this paper, we consider, for the first time, the effect of many types of electronic countermeasures (ECM) on pursuit-evasion games. The evader's decision to initiate its ECM is modeled as a function of the distance between the players. Simulations show how to find optimal strategies for ECM use when initial conditions are known. We also discuss the effectiveness of different ECM technologies in pursuit-evasion games.      

A simple group pursuit problem with equal opportunities and the presence of evader’s defenders

This paper derives necessary and sufficient conditions of simultaneous multiple capture in a simple group pursuit problem with equal opportunities and the presence of evader’s defenders.     

A Decentralized Fuzzy Learning Algorithm for Pursuit-Evasion Differential Games with Superior Evaders

In this paper, we consider multi-pursuer single-superior-evader pursuit-evasion differential games where the evader has a speed that is similar to or higher than the speed of each pursuer. A new fuzzy reinforcement learning algorithm is proposed in this work. The proposed algorithm uses the well-known Apollonius circle mechanism to define the capture region of the learning pursuer based on its location and the location of the superior evader. The proposed algorithm uses the Apollonius circle with a developed formation control approach in the tuning mechanism of the fuzzy logic controller (FLC) of the learning pursuer so that one or some of the learning pursuers can capture the superior evader. The formation control mechanism used by the proposed algorithm guarantees that the pursuers are distributed around the superior evader in order to avoid collision between pursuers. The formation control mechanism used by the proposed algorithm also makes the Apollonius circles of each two adjacent pursuers intersect or be at least tangent to each other so that the capture of the superior evader can occur. The proposed algorithm is a decentralized algorithm as no communication among the pursuers is required. The only information the proposed algorithm requires is the position and the speed of the superior evader. The proposed algorithm is used to learn different multi-pursuer single-superior-evader pursuit-evasion differential games. The simulation results show the effectiveness of the proposed algorithm.     

A discrete dynamic game modelling anti-missile defense scenarios

A discretized game model of an anti-missile defense scenario is presented. Such an engagement is a pursuit of a blind evader, committed to satisfy a terminal constraint. It can be formulated as an incomplete information zero-sum pursuit-evasion game with state constraint. Such a game becomes non trivial if the evader has higher maneuverability than the pursuer. In this case the game admits a saddle-point solution in mixed strategies. Two simple examples illustrate the major elements of the solution.     

复杂三维多面体环境中空地协作追逃问题

结合无人机(UAV)的空中移动和无人车(UGV)的地面移动特点,本文提出了一种UAV/UGV空地协作系统,并且针对其在复杂地形中的追逃问题,提出了一种复杂三维多面体环境中UAV/UGV空地协作追逃策略.首先介绍了UAV/UGV空地协作系统的结构与协作追逃问题描述.接着将边界值问题(BVP)改进并离散化作为博弈走法生成器...     

Evolving team behaviors with specialization

This article evaluates Collective Neuro-Evolution (CONE), a cooperative co-evolutionary method for solving collective behavior tasks and increasing task performance via facilitating behavioral specialization in agent teams. Specialization is used as a problem solving mechanism, and its emergence is guided and regulated by CONE. CONE is comparatively evaluated with related methods in a simulated evolutionary robotics pursuit-evasion task. This task required multiple pursuer robots to cooperatively capture evader robots. Results indicate that CONE is appropriate for evolving specialized behaviors. The interaction of specialized behaviors produces behavioral heterogeneity in teams and collective prey capture behaviors that yield significantly higher performances compared to related methods.     

基于终端诱导强化学习的航天器轨道追逃博弈

针对脉冲推力航天器轨道追逃博弈问题, 提出一种基于强化学习的决策方法, 实现追踪星在指定时刻抵近至逃逸星的特定区域, 其中两星都具备自主博弈能力. 首先, 充分考虑追踪星和逃逸星的燃料约束、推力约束、决策周期约束、运动范围约束等实际约束条件, 建立锥形安全接近区及追逃博弈过程的数学模型; 其次, 为了提升航天器面对不确定博弈对抗场景的自主决策能力, 以近端策略优化 (Proximal policy optimization, PPO) 算法框架为基础, 采用左右互搏的方式同时训练追踪星和逃逸星, 交替提升两星的决策能力; 在此基础上, 为了在指定时刻完成追逃任务, 提出一种终端诱导的奖励函数设计方法, 基于CW (Clohessy Wiltshire)方程预测两星在终端时刻的相对误差, 并将该预测误差引入奖励函数中, 有效引导追踪星在指定时刻进入逃逸星的安全接近区. 与现有基于当前误差设计奖励函数的方法相比, 所提方法能够有效提高追击成功率. 最后, 通过与其他学习方法仿真对比, 验证提出的训练方法和奖励函数设计方法的有效性和优越性.     

The pursuit-evasion problem under integral-geometric constraints on pursuer controls

We study the pursuit-evasion problem for a model game with simple motions when pursuer controls are subject to both integral and geometric constraints while evader controls are only subject to geometric ones. Depending on initial conditions of the players and parametric values participating in control constraints, we prove the theorem of alternative. To solve the pursuit problem, we propose a parallel pursuit strategy (Π-strategy) that ensures optimal convergence of the players and study its structure depending on the parameters. To solve the evasion problem, we find lower bounds on the convergence that also depend on given parameters. This work develops and extends the works of Isaacs, Petrosyan, Pshenichnyi and other researchers, including the author.     

On a class of pursuit-evasion problems

This paper shows that a pursuit-evasion problem can be made amenable to solution with nonlinear programming algorithms by operating the pursuer and evader systems in a "discrete" mode of control and by choosing the cost function judiciously.     

Optimal Control over the Approach of Conflicting Moving Objects under Uncertainty

A solution of a pursuit-evasion problem is obtained for two controlled moving points for the case where the coordinates of the evader are measured with bounded errors and the controls chosen by the evader are assumed to be known up to their membership in a given compact set. Moreover, the conditions of a guaranteed solution of the pursuit-evasion problem are obtained for the worst values of measurement errors and controls of the evader.     

基于博弈论及Q学习的多Agent协作追捕算法

多Agent协作追捕问题是多Agent协调与协作研究中的一个典型问题。针对具有学习能力的单逃跑者追捕问题，提出了一种基于博弈论及Q学习的多Agent协作追捕算法。首先,建立协作追捕团队，并构建协作追捕的博弈模型；其次,通过对逃跑者策略选择的学习，建立逃跑者有限的Step-T累积奖赏的运动轨迹，并把运动轨迹调整到追捕者的策略集中；最后,求解协作追捕博弈得到Nash均衡解，每个Agent执行均衡策略完成追捕任务。同时,针对在求解中可能存在多个均衡解的问题，加入了虚拟行动行为选择算法来选择最优的均衡策略。C#仿真实验表明，所提算法能够有效地解决障碍环境中单个具有学习能力的逃跑者的追捕问题，实验数据对比分析表明该算法在同等条件下的追捕效率要优于纯博弈或纯学习的追捕算法。