The Impact of Payoff Function and Local Interaction on the N-Player Iterated Prisoner's Dilemma

The N-player iterated prisoner's dilemma (NIPD) game has been widely used to study the evolution of cooperation in social, economic and biological systems. This paper studies the impact of different payoff functions and local interactions on the NIPD game. The evolutionary approach is used to evolve game-playing strategies starting from a population of random strategies. The different payoff functions used in our study describe different behaviors of cooperation and defection among a group of players. Local interaction introduces neighborhoods into the NIPD game. A player does not play against every other player in a group any more. He only interacts with his neighbors. We investigate the impact of neighborhood size on the evolution of cooperation in the NIPD game and the generalization ability of evolved strategies. Received 18 August 1999 / Revised 27 February 2000 / Accepted 15 May 2000     

Evolution of cooperative behavior among heterogeneous agents with different strategy representations in an iterated prisoner’s dilemma game

The iterated prisoner’s dilemma (IPD) game has frequently been used to examine the evolution of cooperative behavior among agents. When the effect of representation schemes of IPD game strategies was examined, the same representation scheme was usually assigned to all agents. That is, in the literature, a population of homogeneous agents was usually used in computational experiments. In this article, we focus on a slightly different situation where every agent does not necessarily use the same representation scheme. That is, a population can be a mixture of heterogeneous agents with different representation schemes. In computational experiments, we used binary strings of different lengths (i.e., three-bit and five-bit strings) to represent IPD game strategies. We examined the evolution of cooperative behavior among heterogeneous agents in comparison with the case of homogeneous ones for the standard IPD game with typical payoff values of 0, 1, 3, and 5. Experimental results showed that the evolution of cooperative behavior was slowed down by the use of heterogeneous agents. It was also demonstrated that a faster evolution of cooperative behavior is achieved among majority agents than among minority ones in a heterogeneous population.     

时间尺度与选择倾向性协同作用下的演化博弈模型

针对合作行为的涌现与维持问题，基于演化博弈理论和网络理论，提出了一种促进合作的演化博弈模型。该模型同时将时间尺度、选择倾向性引入到演化博弈中。在初始化阶段，根据持有策略的时间尺度将个体分为两种类型：一种个体在每个时间步都进行策略更新；另一种个体在每一轮博弈后，以某种概率来决定是否进行策略更新。在策略更新阶段，模型用个体对周围邻居的贡献来表征他的声誉，并假设参与博弈的个体倾向于学习具有较好声誉邻居的策略。仿真实验结果表明，所提出的时间尺度与选择倾向性协同作用下的演化博弈模型中，合作行为能够在群体中维持；惰性个体的存在不利于合作的涌现，但是个体的非理性行为反而能够促进合作。     

Multi-objective design method based on evolution game and its application for suspension

Through research and bionics of biology survival mode, game players with competition, cooperation and self-adaptation capacity are introduced in the multi-objective design. The dynamic behavior and bounded rationality in game processes for players are considered according to Chinese saying “In success, commit oneself to the welfare of the society; in distress, maintain one’s own integrity”. An evolution rule, Poor-Competition-Rich-Cooperation (short for PCRC), is proposed. Then, the corresponding payoff functions of competition and cooperation behavior are established and a multi-objective design method based on evolution game is proposed. The calculation steps are as follows: 1) Taking the design objectives as different game players, and calculating factors of the design variables to objective and fuzzy clustering. The design variables are divided into multiple strategy subsets owned by each game player. 2) According to the evolution rule, each player determines its behavior and payoff function in this game round. 3) In their own strategy subsets, each game player takes their payoff as mono-objective for optimization. It gives the best strategy upon other players. And so the best strategies of all players conform the group strategy in this round. The final equilibrium solution is obtained through multi-round game based on convergence criterion. The validity and reliability of this method are shown by the results of an example of a tri-objective optimization design of passive suspension parameters.     

Evolutionary computation and games

Games provide competitive, dynamic environments that make ideal test beds for computational intelligence theories, architectures, and algorithms. Natural evolution can be considered to be a game in which the rewards for an organism that plays a good game of life are the propagation of its genetic material to its successors and its continued survival. In natural evolution, the fitness of an individual is defined with respect to its competitors and collaborators, as well as to the environment. Within the evolutionary computation (EC) literature, this is known as co-evolution and within this paradigm, expert game-playing strategies have been evolved without the need for human expertise.     

系统直和博弈模型下的合作演化

演化博弈论是生物进化论与博弈论结合产生的理论,已成为研究合作演化行为的有力工具.本文研究了基于系统直和博弈模型下的合作演化行为.首先,利用复制者方程分析了双人双策略及三策略对称博弈的演化动力学过程.然后,以石头剪刀布模型和雪堆模型为基础,采用矩阵直和构建系统直和博弈模型,并将所构造的直和矩阵转化为含参数的系统总支付矩阵.随后,说明了这种方法可推广到n个博弈的情形.最后,利用MATLAB对系统直和博弈模型进行仿真模拟,从系统整体的角度分析合作演化.仿真结果表明,混合之后的系统直和博弈较单一博弈而言,合作策略的占比明显增加,且整个系统稳定性更好.这种合作演化机制呈现了全局互惠.     

Strong coalitional equilibrium in a transportation game

This paper introduces an extension of the vehicle routing problem by involving several decision makers in competition. Each customer is characterized by demand and distance to the warehouse. The problem is described in form of a cooperative transportation game (CTG). We consider customers as players in the game. Their strategies are the routes for a vehicle they should rent in a coalition to deliver goods subject to their demand with minimal transportation costs, under the assumption that transportation costs are allocated between the players according to the Nash arbitration scheme. For each profile in coalitional strategies, we define a coalitional structure of players and the costs of each player. A strong equilibrium is found for the cooperative transportation game. In addition, we develop a procedure to calculate the strong equilibrium. This procedure is illustrated by a numerical example.     

Spatial strategies in a generalized spatial prisoner’s dilemma

Many strategies, such as tit-for-tat, have been proposed in the iterated prisoner’s dilemma (IPD) in which the prisoner’s dilemma (PD) is carried out repeatedly with two players. A spatial version of the iterated prisoner’s dilemma (SPD) has been studied, where a player at each site plays the IPD game with all the players in the neighborhood. However, the strategies studied in the SPD consider the past actions of a single opponent only. We studied spatial strategies that depend on the configuration of actions taken by all neighbors (as opposed to conventional temporal strategies). Since generosity can be considered as a spatial strategy, we first investigate the generosity required when an action error is involved. We also propose several spatial strategies that outperform many others.This work was presented, in part, at the 9th International Symposium on Artificial Life and Robotics, Oita, Japan, January 28–30, 2004     

合作博弈的连续蚁群算法求解

在合作博弈的理论研究中,经典的合作博弈解概念在求解过程中没有体现出局中人的有限理性和互动博弈行为。而在现实博弈环境中,联盟的分配方案更多是通过局中人间理性互动与策略博弈形成的。引入理性因子和控制因子来描述局中人在博弈过程中的决策行为,建立了考虑互动行为的合作博弈模型,并利用连续蚁群算法对合作博弈进行求解。算例表明该解法可以保证分配方案满足有效性和个体理性,并能快速得到联盟的唯一分配方案。这为合作博弈的求解提供了新的思路与工具。     

Risk consideration and cooperation in the iterated prisoner’s dilemma

This paper investigates the cooperative behavior in the two-player iterated prisoner’s dilemma (IPD) game with the consideration of income stream risk. The standard deviation of one-move payoffs for players is defined for measuring the income stream risk, and thus the risk effect on the cooperation in the two-player IPD game is examined. A two-population coevolutionary learning model, embedded with a niching technique, is developed to search optimal strategies for two players to play the IPD game. As experimental results illustrate, risk-averse players perform better than risk-seeking players in cooperating with opponents. In particular, in the case with short game encounters, in which cooperation has been demonstrated to be difficult to achieve in previous work, a high level of cooperation can be obtained in the IPD if both players are risk-averse. The reason is that risk consideration induces players to negotiate for stable gains, which lead to steady mutual cooperation in the IPD. This cooperative pattern is found to be quite robust against low levels of noise. However, with increasingly higher levels of noise, only intermediate levels of cooperation can be achieved in games between two risk-averse players. Games with risk-seeking players get to even lower cooperation levels. By comparing the players’ strategies coevolved with and without a high level of noise, the main reason for the reduction in the extent of cooperation can be explained as the lack of contrition and forgiveness of players in the high-noise interactions. Moreover, although increasing encounter length is helpful in improving cooperation in the noiseless and low-noise IPD, we find that it may enforce the absence of contrition and forgiveness, and thus make cooperation even more difficult in the high-noise games.     

Optimal Strategies of the Iterated Prisoner's Dilemma Problem for Multiple Conflicting Objectives

In this paper, we present a new paradigm of searching optimal strategies in the game of iterated prisoner's dilemma (IPD) using multiple-objective evolutionary algorithms. This method is more useful than the existing approaches, because it not only produces strategies that perform better in the iterated game but also finds a family of nondominated strategies, which can be analyzed to decipher properties a strategy should have to win the game in a more satisfactory manner. We present the results obtained by this new method and discuss sub-strategies found to be common among nondominated strategies. The multiobjective treatment of the IPD problem demonstrated here can be applied to other similar game-playing tasks.      

On Dynamics and Nash Equilibriums of Networked Games

Networked noncooperative games are investigated, where each player (or agent) plays with all other players in its neighborhood. Assume the evolution is based on the fact that each player uses its neighbors' current information to decide its next strategy. By using sub-neighborhood, the dynamics of the evolution is obtained. Then a method for calculating Nash equilibriums from mixed strategies of multi-players is proposed. The relationship between local Nash equilibriums based on individual neighborhoods and global Nash equilibriums of overall network is revealed. Then a technique is proposed to construct Nash equilibriums of an evolutionary game from its one step static Nash equilibriums. The basic tool of this approach is the semi-tensor product of matrices, which converts strategies into logical matrices and payoffs into pseudo-Boolean functions, then networked evolutionary games become discrete time dynamic systems.      

Altruistic behavior in a nonantagonistic positional differential game

We consider a two-person nonantagonistic positional differential game (NPDG) whose dynamics is described by an ordinary nonlinear vector differential equation. Constraints on values of players’ controls are geometric. Final time of the game is fixed. Payoff functionals of both players are terminal. The formalization of positional strategies in an NPDG is based on the formalization and results of the general theory of antagonistic positional differential games (APDGs) (see monographs by N.N. Krasovskii and A.I. Subbotin [3, 4]). Additionally, in the present paper we assume that each player, together with the usual, normal (nor), type of behavior aimed at maximizing his own functional, can use other behavior types introduced in [2, 5]. In particular, these may be altruistic (alt), aggressive (agg), and paradoxical (par) types. It is assumed that in the course of the game players can switch their behavior from one type to another. Using the possibility of such switches in a repeated bimatrix 2 × 2 game in [5, 6] allowed to obtain new solutions of this game. In the present paper, extension of this approach to NPDGs leads to a new formulation of the problem. In particular, of interest is the question of how players’ outcomes at Nash solutions are transformed. An urgent problem is minimizing the time of “abnormal” behavior while achieving a good result. The paper proposes a formalization of an NPDG with behavior types (NPDGwBT). It is assumed that in an NPDGwBT each player, simultaneously with choosing a positional strategy, chooses also his own indicator function defined on the whole game horizon and taking values in the set {normal, altruistic, aggressive, paradoxical}. The indicator function of a player shows the dynamics of changes in the behavior type demonstrated by the player. Thus, in this NPDGwBT each player controls the choice of a pair {positional strategy, indicator function}. We define the notion of a BT-solution of such a game. It is expected that using behavior types in the NPDGwBT which differ from the normal one (so-called abnormal types) in some cases may lead to more favorable outcomes for the players than in the NPDG. We consider two examples of an NPDGwBT with simple dynamics in the plane in each of which one player keeps to altruistic behavior type over some time period. It is shown that in the first example payoffs of both players increase on a BT-solution as compared to the game with the normal behavior type, and in the second example, the sum of players’ payoffs is increased.     

多组对策系统非劣Nash策略的最优均衡解算法

多组对策系统中求解组与组之间的非劣Nash策略至关重要.如何针对一般问题解析求出非劣Nash策略还没有有效的方法.本文阐述了一种利用组与组之间的非劣反应集构造求解非劣Nash策略的迭代算法.为此首先引进多组对策系统组内部合作对策的最优均衡值和最优均衡解的概念,然后通过证明最优均衡解是组内部隐含某一权重向量的合作对策的非劣解,得到求解合作对策的单目标规划问题.进一步说明在组内部该问题的解不仅是非劣解而且对所有局中人都优于不合作时的Nash平衡策略.最后给出了验证该算法有效性的一个实际例子.     

Cooperation in the evolutionary iterated prisoner’s dilemma game with risk attitude adaptation

The Iterated Prisoner’s Dilemma (IPD) game has been commonly used to investigate the cooperation among competitors. However, most previous studies on the IPD focused solely on maximizing players’ average payoffs without considering their risk preferences. By introducing the concept of income stream risk into the IPD game, this paper presents a novel evolutionary IPD model with agents seeking to balance between average payoffs and risks with respect to their own risk attitudes. We build a new IPD model of multiple agents, in which agents interact with one another in the pair-wise IPD game while adapting their risk attitudes according to their received payoffs. Agents become more risk averse after their payoffs exceed their aspirations, or become more risk seeking after their payoffs fall short of their aspirations. The aspiration levels of agents are determined based on their historical self-payoff information or the payoff information of the agent population. Simulations are conducted to investigate the emergence of cooperation under these two comparison methods. Results indicate that agents can sustain a highly cooperative equilibrium when they consider only their own historical payoffs as aspirations (called historical comparison) in adjusting their risk attitudes. This holds true even for the IPD with a short game encounter, for which cooperation was previously demonstrated difficult. However, when agents evaluate their payoffs in comparison with the population average payoff (called social comparison), those agents with payoffs below the population average tend to be dissatisfied with the game outcomes. This dissatisfaction will induce more risk-seeking behavior of agents in the IPD game, which will constitute a strong deterrent to the emergence of mutual cooperation in the population.     

基于博弈遗传算法的组合电路进化设计

为了有效提高组合逻辑电路进化设计的速度和效率,提出了一种基于博弈遗传算法的电路进化设计算法。将组合电路中的每个输出端作为博弈者,组成每个输出端的逻辑门之间的连接和组态作为策略,将电路优化问题转化为博弈优化决策问题,策略的选择通过遗传算法实现,从而建立了组合电路优化设计的博弈模型。最后通过仿真实验验证该算法的有效性。     

Censorship game in optimal choice problem

The authors consider a game of the optimal choice where one of the players seeks to decrease the probability of selection of the best object by the other player by imposing some prohibitions and restrictions on browsing of certain elements. Optimal players’ strategies that form the Nash equilibrium are found. The asymptotic behavior of the strategies in case where the number of objects tends to infinity is studied.     

A multiagent player system composed by expert agents in specific game stages operating in high performance environment

This paper proposes a new approach for the non-supervised learning process of multiagent player systems operating in a high performance environment, being that the cooperative agents are trained so as to be expert in specific stages of a game. This proposal is implemented by means of the Checkers automatic player denominated D-MA-Draughts, which is composed of 26 agents. The first is specialized in initial and intermediary game stages, whereas the remaining are specialists in endgame stages (defined by board-games containing, at most, 12 pieces). Each of these agents consists of a Multilayer Neural Network, trained without human supervision through Temporal Difference Methods. The best move is determined by the distributed search algorithm known as Young Brothers Wait Concept. Each endgame agent is able to choose a move from a determined profile of endgame board. These profiles are defined by a clustering process performed by a Kohonen-SOM network from a database containing endgame boards retrieved from real matches. Once trained, the D-MA-Draughts agents can actuate in a match according to two distinct game dynamics. In fact, the D-MA-Draughts architecture corresponds to an extension of two preliminary versions: MP-Draughts, which is a multiagent system with a serial search algorithm, and D-VisionDraughts, which is a single agent with a distributed search algorithm. The D-MA-Draughts gains are estimated through several tournaments against these preliminary versions. The results show that D-MA-Draughts improves upon its predecessors by significantly reducing training time and the endgame loops, thus beating them in several tournaments.     

Reduced game property of the Egalitarian Non-k-Averaged Contribution (EN k AC-) value and the Shapley value

The Egalitarian Non- k -Averaged Contribution (EN ^k AC-) value for TU-game represents the equal division of the surplus of the total profits, given that each player is already allocated his individual contribution specified by worths of coalitions of size k . This paper deals with the axiomatic characterization of the EN ^k AC-value on the class of cooperative games with a fixed player set as well as a variable player set. The latter axiomatization involves a consistency axiom in terms of the reduced games. The EN ^k AC-value is the unique value on the class of cooperative games with a variable player set which possesses the relative invariance under strategic equivalence, the equal treatment property and the reduced game property for two types of reduced games. We also propose a new reduced game in terms of which the Shapley value is axiomatized.