Connections between stochastic control and dynamic games

We consider duality relations between risk-sensitive stochastic control problems and dynamic games. They are derived from two basic duality results, the first involving free energy and relative entropy and resulting from a Legendre-type transformation, the second involving power functions. Our approach allows us to treat, in essentially the same way, continuous- and discrete-time problems, with complete and partial state observation, and leads to a very natural formal justification of the structure of the cost functional of the dual. It also allows us to obtain the solution of a stochastic game problem by solving a risk-sensitive control problem.     

General finite-dimensional risk-sensitive problems and small noiselimits

For a risk-sensitive, partially observed stochastic control problem, the modified Zakai equation includes an extra term related to the exponential running cost. The finite-dimensional solutions of this modified Zakai equation are obtained. These are analogs of the Kalman and Benes filters. The small noise limits of the finite-dimensional risk-sensitive problems are then obtained. These lead to differential games with deterministic disturbances     

Multiple-objective risk-sensitive control and its small noise limit

This paper is concerned with a (minimizing) multiple-objective risk-sensitive control problem. Asymptotic analysis leads to the introduction of a new class of two-player, zero-sum, deterministic differential games. The distinguishing feature of this class of games is that the cost functional is multiple-objective in nature, being composed of the risk-neutral integral costs associated with the original risk-sensitive problem. More precisely, the opposing player in such a game seeks to maximize the most ‘vulnerable’ member of a given set of cost functionals while the original controller seeks to minimize the worst ‘damage’ that the opponent can do over this set. It is then shown that the problem of finding an efficient risk-sensitive controller is equivalent, asymptotically, to solving this differential game. Surprisingly, this differential game is proved to be independent of the weights on the different objectives in the original multiple-objective risk-sensitive problem. As a by-product, our results generalize the existing results for the single-objective risk-sensitive control problem to a substantially larger class of nonlinear systems, including those with control-dependent diffusion terms.     

Abstracting Nash equilibria of supermodular games

Supermodular games are a well known class of noncooperative games which find significant applications in a variety of models, especially in operations research and economic applications. Supermodular games always have Nash equilibria which are characterized as fixed points of multivalued functions on complete lattices. Abstract interpretation is here applied to set up an approximation framework for Nash equilibria of supermodular games. This is achieved by extending the theory of abstract interpretation in order to cope with approximations of multivalued functions and by providing some methods for abstracting supermodular games, thus obtaining approximate Nash equilibria which are shown to be correct within the abstract interpretation framework.     

Discovering theorems in game theory: Two-person games with unique pure Nash equilibrium payoffs

In this paper we provide a logical framework for two-person finite games in strategic form, and use it to design a computer program for discovering some classes of games that have unique pure Nash equilibrium payoffs. The classes of games that we consider are those that can be expressed by a conjunction of two binary clauses, and our program re-discovered Kats and Thisse?s class of weakly unilaterally competitive two-person games, and came up with several other classes of games that have unique pure Nash equilibrium payoffs. It also came up with new classes of strict games that have unique pure Nash equilibria, where a game is strict if for both player different profiles have different payoffs.     

Equilibria,fixed points,and complexity classes

Many models from a variety of areas involve the computation of an equilibrium or fixed point of some kind. Examples include Nash equilibria in games; market equilibria; computing optimal strategies and the values of competitive games (stochastic and other games); stable configurations of neural networks; analysing basic stochastic models for evolution like branching processes and for language like stochastic context-free grammars; and models that incorporate the basic primitives of probability and recursion like recursive Markov chains. It is not known whether these problems can be solved in polynomial time. There are certain common computational principles underlying different types of equilibria, which are captured by the complexity classes PLS, PPAD, and FIXP. Representative complete problems for these classes are, respectively, pure Nash equilibria in games where they are guaranteed to exist, (mixed) Nash equilibria in two-player normal form games, and (mixed) Nash equilibria in normal form games with three (or more) players. This paper reviews the underlying computational principles and the corresponding classes.     

Equilibrium Trajectories in Dynamical Bimatrix Games with Average Integral Payoff Functionals

This paper considers models of evolutionary non-zero-sum games on the infinite time interval. Methods of differential game theory are used for the analysis of game interactions between two groups of participants. We assume that participants in these groups are controlled by signals for the behavior change. The payoffs of coalitions are defined as average integral functionals on the infinite horizon. We pose the design problem of a dynamical Nash equilibrium for the evolutionary game under consideration. The ideas and approaches of non-zero-sum differential games are employed for the determination of the Nash equilibrium solutions. The results derived in this paper involve the dynamic constructions and methods of evolutionary games. Much attention is focused on the formation of the dynamical Nash equilibrium with players strategies that maximize the corresponding payoff functions and have the guaranteed properties according to the minimax approach. An application of the minimax approach for constructing optimal control strategies generates dynamical Nash equilibrium trajectories yielding better results in comparison to static solutions and evolutionary models with the replicator dynamics. Finally, we make a comparison of the dynamical Nash equilibrium trajectories for evolutionary games with the average integral payoff functionals and the trajectories for evolutionary games with the global terminal payoff functionals on the infinite horizon.     

On the Complexity of Pareto-Optimal Nash and Strong Equilibria

We consider the computational complexity of coalitional solution concepts in scenarios related to load balancing such as anonymous and congestion games. In congestion games, Pareto-optimal Nash and strong equilibria, which are resilient to coalitional deviations, have recently been shown to yield significantly smaller inefficiency. Unfortunately, we show that several problems regarding existence, recognition, and computation of these concepts are hard, even in seemingly special classes of games. In anonymous games with constant number of strategies, we can efficiently recognize a state as Pareto-optimal Nash or strong equilibrium, but deciding existence for a game remains hard. In the case of player-specific singleton congestion games, we show that recognition and computation of both concepts can be done efficiently. In addition, in these games there are always short sequences of coalitional improvement moves to Pareto-optimal Nash and strong equilibria that can be computed efficiently.     

Analytical solution for a class of linear quadratic open-loop Nash game with multiple players

In this paper, the Nash equilibria for differential games with multiple players is studied. A method for solving the Riccati-type matrix differential equations for open-loop Nash strategy in linear quadratic game with multiple players is presented and analytical solution is given for a type of differential games in which the system matrix can be diagonalizable. As the special cases, the Nash equilibria for some type of differential games with particular structure is studied also, and some results in previous literatures are extended. Finally, a numerical example is given to illustrate the effectiveness of the solution procedure.     

Analytical solution for a class of linear quadratic open-loop Nash game with multiple players

In this paper, the Nash equilibria for differential games with multiple players is studied. A method for solving the Riccati-type matrix differential equations for open-loop Nash strategy in linear quadratic game with multiple players is presented and analytical solution is given for a type of differential games in which the system matrix can be diagonalizable. As the special cases, the Nash equilibria for some type of differential games with particular structure is studied also, and some results in previous literatures are extended. Finally, a numerical example is given to illustrate the effectiveness of the solution procedure.     

The Quantum Ultimatum Game

A quantum version of the ultimatum game is studied. Both a restricted version with classical moves and the unitary version are considered. With entangled initial states, Nash equilibria in quantum games are in general different from those of classical games. Quantum versions might therefore be useful as a framework for modeling deviations from classical Nash equilibrium in experimental games.PACS:02.50.Le; 03.67.-a     

一种面向多Agent交互的博弈Nash均衡求解方法

现有的图型博弈Nash均衡求解方法基本是在离散化剖面空间中搜索求解，最终只能得到近似Nash均衡。针对现有求解方法存在的不足，把求解图型博弈的Nash均衡看作是连续策略空间中的函数优化问题，定义Agents在策略剖面中的效用偏离度之和为优化目标，其最优解就是博弈的Nash均衡。本文基于对实例的分析指出目标函数下降梯度的计算可归结为一组线性规划，进而提出一种求解图型博弈Nash均衡的新型梯度下降算法。算法分析及实验研究表明，对于多Agent交互模型中的相关问题，本文提出的方法可求解任意图结构图型博弈Nash均衡，对于大规模图型博弈也有较好的求解精度和求解效率。     

The role of information state and adjoint in relating nonlinearoutput feedback risk-sensitive control and dynamic games

This paper employs logarithmic transformations to establish relations between continuous-time nonlinear partially observable risk-sensitive control problems and analogous output feedback dynamic games. The first logarithmic transformation is introduced to relate the stochastic information state to a deterministic information state. The second logarithmic transformation is applied to the risk-sensitive cost function using the Laplace-Varadhan lemma. In the small noise limit, this cost function is shown to be logarithmically equivalent to the cost function of an analogous dynamic game     

On the Structure of Weakly Acyclic Games

The class of weakly acyclic games, which includes potential games and dominance-solvable games, captures many practical application domains. In a weakly acyclic game, from any starting state, there is a sequence of better-response moves that leads to a pure Nash equilibrium; informally, these are games in which natural distributed dynamics, such as better-response dynamics, cannot enter inescapable oscillations. We establish a novel link between such games and the existence of pure Nash equilibria in subgames. Specifically, we show that the existence of a unique pure Nash equilibrium in every subgame implies the weak acyclicity of a game. In contrast, the possible existence of multiple pure Nash equilibria in every subgame is insufficient for weak acyclicity in general; here, we also systematically identify the special cases (in terms of the number of players and strategies) for which this is sufficient to guarantee weak acyclicity.     

Multi-agent discrete-time graphical games and reinforcement learning solutions

This paper introduces a new class of multi-agent discrete-time dynamic games, known in the literature as dynamic graphical games. For that reason a local performance index is defined for each agent that depends only on the local information available to each agent. Nash equilibrium policies and best-response policies are given in terms of the solutions to the discrete-time coupled Hamilton–Jacobi equations. Since in these games the interactions between the agents are prescribed by a communication graph structure we have to introduce a new notion of Nash equilibrium. It is proved that this notion holds if all agents are in Nash equilibrium and the graph is strongly connected. A novel reinforcement learning value iteration algorithm is given to solve the dynamic graphical games in an online manner along with its proof of convergence. The policies of the agents form a Nash equilibrium when all the agents in the neighborhood update their policies, and a best response outcome when the agents in the neighborhood are kept constant. The paper brings together discrete Hamiltonian mechanics, distributed multi-agent control, optimal control theory, and game theory to formulate and solve these multi-agent dynamic graphical games. A simulation example shows the effectiveness of the proposed approach in a leader-synchronization case along with optimality guarantees.     

Finite Horizon Minimax Optimal Control of Stochastic Partially Observed Time Varying Uncertain Systems

We consider a linear-quadratic problem of minimax optimal control for stochastic uncertain control systems with output measurement. The uncertainty in the system satisfies a stochastic integral quadratic constraint. To convert the constrained optimization problem into an unconstrained one, a special S-procedure is applied. The resulting unconstrained game-type optimization problem is then converted into a risk-sensitive stochastic control problem with an exponential-of-integral cost functional. This is achieved via a certain duality relation between stochastic dynamic games and risk-sensitive stochastic control. The solution of the risk-sensitive stochastic control problem in terms of a pair of differential matrix Riccati equations is then used to establish a minimax optimal control law for the original uncertain system with uncertainty subject to the stochastic integral quadratic constraint. Date received: May 13, 1997. Date revised: March 18, 1998.     

基于事件驱动控制的混杂动态博弈系统的纳什均衡分析

本文研究了基于事件驱动控制的混杂动态博弈系统的纳什均衡分析问题. 首先, 分析了事件驱动机制对混 杂动态博弈过程的影响, 进而, 在进行状态空间描述的基础上, 给出了混杂动态博弈的纳什均衡的定义, 并建立了对 应博弈系统的策略型模型. 其次, 结合Lanchester方程, 分别讨论了两类混杂动态博弈系统的均衡问题, 包括事件驱 动策略设计和固定的情况, 获得了均衡解存在的必要条件. 最后, 通过数值模拟进行了应用分析, 验证了所取得结果 的合理性和科学性, 并总结了混杂动态博弈研究的未来工作.     

异构无线网络用户网络关联优化:一种基于群体博弈的方法

针对异构无线网络(Heterogeneous Wireless Networks,HWNs)负载平衡问题,提出了一种基于群体博弈的用户网络关联方案.首先将HWNs系统用户网络关联问题抽象成一个群体博弈模型,根据用户在网络中得到的收益函数,证明该群体博弈满足势博弈的条件.利用复制动态作为演化动态工具,证明演化的结果最终会收敛到纳什均衡,这个特性确保了每个用户关联到一个效用最优的网络.然后证明纳什均衡点能最大化整个HWNs系统的吞吐量,保证了纳什均衡的有效性.最后,基于复制动态原理提出了用户网络关联算法.仿真实验模拟了用户网络选择过程,得到了均衡点,验证了理论分析的结果.     

On Dynamics and Nash Equilibriums of Networked Games

Networked noncooperative games are investigated, where each player (or agent) plays with all other players in its neighborhood. Assume the evolution is based on the fact that each player uses its neighbors' current information to decide its next strategy. By using sub-neighborhood, the dynamics of the evolution is obtained. Then a method for calculating Nash equilibriums from mixed strategies of multi-players is proposed. The relationship between local Nash equilibriums based on individual neighborhoods and global Nash equilibriums of overall network is revealed. Then a technique is proposed to construct Nash equilibriums of an evolutionary game from its one step static Nash equilibriums. The basic tool of this approach is the semi-tensor product of matrices, which converts strategies into logical matrices and payoffs into pseudo-Boolean functions, then networked evolutionary games become discrete time dynamic systems.      

A pure strategy nash equilibrium for a 3-firm location game on a sphere

There are only two known pure strategy Nash equilibria for multi-firm location games in dimension two. This paper presents an infinite family of pure strategy Nash equilibria for a 3-firm location game on a sphere.