Artificial societies and generative social science

What is anartificial society? What can such models offer the social sciences in particular? We address these general questions, drawing brief illustrations from the specific artificial society we call “Sugarscape.”     

Reciprocal Altruism-based Cooperation in a Social Network Game

Cooperative behaviors are common in humans and are fundamental to our society. Theoretical and experimental studies have modeled environments in which the behaviors of humans, or agents, have been restricted to analyze their social behavior. However, it is important that such studies are generalized to less restrictive environments to understand human society. Social network games (SNGs) provide a particularly powerful tool for the quantitative study of human behavior. In SNGs, numerous players can behave more freely than in the environments used in previous studies; moreover, their relationships include apparent conflicts of interest and every action can be recorded. We focused on reciprocal altruism, one of the mechanisms that generate cooperative behavior. This study aims to investigate cooperative behavior based on reciprocal altruism in a less restrictive environment. For this purpose, we analyzed the social behavior underlying such cooperative behavior in an SNG. We focused on a game scenario in which the relationship between the players was similar to that in the Leader game. We defined cooperative behaviors by constructing a payoff matrix in the scenario. The results showed that players maintained cooperative behavior based on reciprocal altruism, and cooperators received more advantages than noncooperators. We found that players constructed reciprocal relationships based on two types of interactions, cooperative behavior and unproductive communication.     

Bombalytics: Visualization of Competition and Collaboration Strategies of Players in a Bomb Laying Game

Competition and collaboration form complex interaction patterns between the agents and objects involved. Only by understanding these interaction patterns, we can reveal the strategies the participating parties applied. In this paper, we study such competition and collaboration behavior for a computer game. Serving as a testbed for artificial intelligence, the multiplayer bomb laying game Pommerman provides a rich source of advanced behavior of computer agents. We propose a visualization approach that shows an overview of multiple games, with a detailed timeline-based visualization for exploring the specifics of each game. Since an analyst can only fully understand the data when considering the direct and indirect interactions between agents, we suggest various visual encodings of these interactions. Based on feedback from expert users and an application example, we demonstrate that the approach helps identify central competition strategies and provides insights on collaboration.     

基于囚徒困境策略的改进HK网络上的合作博弈

为模拟现实世界的合作行为,本文在HK网络模型基础上提出了一种具有高聚类幂律可调性质的新的网络模型,并分析了囚徒困境博弈在此网络上的演化。通过仿真实验,研究了该网络的高聚类特性对合作行为的影响。大量实验表明,网络的高聚类特性可以极大促进合作现象的涌现。同时研究也发现,随着诱惑参数的变大,合作水平也会随之下降,但幅度不大。总之,该演化博弈模型可以促进合作现象的涌现并抵御背叛策略的传播。     

The adversarial activity model for bounded rational agents

Multiagent research provides an extensive literature on formal Beliefs-Desires-Intentions (BDI) based models describing the notion of teamwork and cooperation. However, multiagent environments are often not cooperative nor collaborative; in many cases, agents have conflicting interests, leading to adversarial interactions. This form of interaction has not yet been formally defined in terms of the agents mental states, beliefs, desires and intentions. This paper presents the Adversarial Activity model, a formal Beliefs-Desires-Intentions (BDI) based model for bounded rational agents operating in a zero-sum environment. In complex environments, attempts to use classical utility-based search methods with bounded rational agents can raise a variety of difficulties (e.g. implicitly modeling the opponent as an omniscient utility maximizer, rather than leveraging a more nuanced, explicit opponent model). We define the Adversarial Activity by describing the mental states of an agent situated in such environment. We then present behavioral axioms that are intended to serve as design principles for building such adversarial agents. We illustrate the advantages of using the model as an architectural guideline by building agents for two adversarial environments: the Connect Four game and the Risk strategic board game. In addition, we explore the application of our approach by analyzing log files of completed Connect Four games, and gain additional insights on the axioms’ appropriateness.     

Evolution of cooperative behavior among heterogeneous agents with different strategy representations in an iterated prisoner’s dilemma game

The iterated prisoner’s dilemma (IPD) game has frequently been used to examine the evolution of cooperative behavior among agents. When the effect of representation schemes of IPD game strategies was examined, the same representation scheme was usually assigned to all agents. That is, in the literature, a population of homogeneous agents was usually used in computational experiments. In this article, we focus on a slightly different situation where every agent does not necessarily use the same representation scheme. That is, a population can be a mixture of heterogeneous agents with different representation schemes. In computational experiments, we used binary strings of different lengths (i.e., three-bit and five-bit strings) to represent IPD game strategies. We examined the evolution of cooperative behavior among heterogeneous agents in comparison with the case of homogeneous ones for the standard IPD game with typical payoff values of 0, 1, 3, and 5. Experimental results showed that the evolution of cooperative behavior was slowed down by the use of heterogeneous agents. It was also demonstrated that a faster evolution of cooperative behavior is achieved among majority agents than among minority ones in a heterogeneous population.     

资源承载力人工社会模型研究

多主体模型是一种研究生态和社会、经济等复杂系统的动态研究方法,社会经济系统作为由类型多样与数量巨大的经济个体组成的复杂系统,系统中各主体复杂的相互作用表现出单个个体所不具备的特征。基于多Agent人工社会建模,在遗产继承和子辈发展方面对原始Sugarscape模型做了改进,借助REPAST软件平台,仿真研究固定资源下对不同数量人口的支撑作用,获取社会财富、个体生活水平以及社会福利随人口变化的数据,根据其变化曲线,得到经济学启示。     

Hierarchical multi-agent reinforcement learning

In this paper, we investigate the use of hierarchical reinforcement learning (HRL) to speed up the acquisition of cooperative multi-agent tasks. We introduce a hierarchical multi-agent reinforcement learning (RL) framework, and propose a hierarchical multi-agent RL algorithm called Cooperative HRL. In this framework, agents are cooperative and homogeneous (use the same task decomposition). Learning is decentralized, with each agent learning three interrelated skills: how to perform each individual subtask, the order in which to carry them out, and how to coordinate with other agents. We define cooperative subtasks to be those subtasks in which coordination among agents significantly improves the performance of the overall task. Those levels of the hierarchy which include cooperative subtasks are called cooperation levels. A fundamental property of the proposed approach is that it allows agents to learn coordination faster by sharing information at the level of cooperative subtasks, rather than attempting to learn coordination at the level of primitive actions. We study the empirical performance of the Cooperative HRL algorithm using two testbeds: a simulated two-robot trash collection task, and a larger four-agent automated guided vehicle (AGV) scheduling problem. We compare the performance and speed of Cooperative HRL with other learning algorithms, as well as several well-known industrial AGV heuristics. We also address the issue of rational communication behavior among autonomous agents in this paper. The goal is for agents to learn both action and communication policies that together optimize the task given a communication cost. We extend the multi-agent HRL framework to include communication decisions and propose a cooperative multi-agent HRL algorithm called COM-Cooperative HRL. In this algorithm, we add a communication level to the hierarchical decomposition of the problem below each cooperation level. Before an agent makes a decision at a cooperative subtask, it decides if it is worthwhile to perform a communication action. A communication action has a certain cost and provides the agent with the actions selected by the other agents at a cooperation level. We demonstrate the efficiency of the COM-Cooperative HRL algorithm as well as the relation between the communication cost and the learned communication policy using a multi-agent taxi problem.     

Learning to trust in the competence and commitment of agents

For agents to collaborate in open multi-agent systems, each agent must trust in the other agents’ ability to complete tasks and willingness to cooperate. Agents need to decide between cooperative and opportunistic behavior based on their assessment of another agents’ trustworthiness. In particular, an agent can have two beliefs about a potential partner that tend to indicate trustworthiness: that the partner is competent and that the partner expects to engage in future interactions. This paper explores an approach that models competence as an agent’s probability of successfully performing an action, and models belief in future interactions as a discount factor. We evaluate the underlying decision framework’s performance given accurate knowledge of the model’s parameters in an evolutionary game setting. We then introduce a game-theoretic framework in which an agent can learn a model of another agent online, using the Harsanyi transformation. The learning agents evaluate a set of competing hypotheses about another agent during the simulated play of an indefinitely repeated game. The Harsanyi strategy is shown to demonstrate robust and successful online play against a variety of static, classic, and learning strategies in a variable-payoff Iterated Prisoner’s Dilemma setting.     

Cooperation in the evolutionary iterated prisoner’s dilemma game with risk attitude adaptation

The Iterated Prisoner’s Dilemma (IPD) game has been commonly used to investigate the cooperation among competitors. However, most previous studies on the IPD focused solely on maximizing players’ average payoffs without considering their risk preferences. By introducing the concept of income stream risk into the IPD game, this paper presents a novel evolutionary IPD model with agents seeking to balance between average payoffs and risks with respect to their own risk attitudes. We build a new IPD model of multiple agents, in which agents interact with one another in the pair-wise IPD game while adapting their risk attitudes according to their received payoffs. Agents become more risk averse after their payoffs exceed their aspirations, or become more risk seeking after their payoffs fall short of their aspirations. The aspiration levels of agents are determined based on their historical self-payoff information or the payoff information of the agent population. Simulations are conducted to investigate the emergence of cooperation under these two comparison methods. Results indicate that agents can sustain a highly cooperative equilibrium when they consider only their own historical payoffs as aspirations (called historical comparison) in adjusting their risk attitudes. This holds true even for the IPD with a short game encounter, for which cooperation was previously demonstrated difficult. However, when agents evaluate their payoffs in comparison with the population average payoff (called social comparison), those agents with payoffs below the population average tend to be dissatisfied with the game outcomes. This dissatisfaction will induce more risk-seeking behavior of agents in the IPD game, which will constitute a strong deterrent to the emergence of mutual cooperation in the population.     

Synchronization of multi-agent systems without connectivity assumptions

Multi-agent systems arise from diverse fields in natural and artificial systems, such as schooling of fish, flocking of birds, coordination of autonomous agents. In multi-agent systems, a typical and basic situation is the case where each agent has the tendency to behave as other agents do in its neighborhood. Through computer simulations, Vicsek et al. (1995) showed that such a simple local interaction rule can lead to a certain kind of cooperative phenomenon (synchronization) of the overall system, if the initial states are randomly distributed and the size of the system population is large. Since this model is of fundamental importance in understanding the multi-agent systems, it has attracted much research attention in recent years. In this paper, we will present a comprehensive theoretical analysis for this class of multi-agent systems under a random framework with large population, but without imposing any connectivity assumptions as did in almost all of the previous investigations. To be precise, we will show that for any given and fixed model parameters concerning with the interaction radius r and the agents’ moving speed v, the overall system will synchronize as long as the population size n is large enough. Furthermore, to keep the synchronization property as the population size n increases, both r and v can actually be allowed to decrease according to certain scaling rates.     

Evolution of iterated prisoner's dilemma game strategies in structured demes under random pairing in game playing

We discuss the evolution of strategies in a spatial iterated prisoner's dilemma (IPD) game in which each player is located in a cell of a two-dimensional grid-world. Following the concept of structured demes, two neighborhood structures are used. One is for the interaction among players through the IPD game. A player in each cell plays against its neighbors defined by this neighborhood structure. The other is for mating strategies by genetic operations. A new strategy for a player is generated by genetic operations from a pair of parent strings, which are selected from its neighboring cells defined by the second neighborhood structure. After examining the effect of the two neighborhood structures on the evolution of cooperative behavior with standard pairing in game-playing, we introduce a random pairing scheme in which each player plays against a different randomly chosen neighbor at every round (i.e., every iteration) of the game. Through computer simulations, we demonstrate that small neighborhood structures facilitate the evolution of cooperative behavior under random pairing in game-playing.     

Can a rational agent afford to be affectless? a formal approach

In this article, we expose some of the issues raised by the critics of the neoclassical approach to rational agent modeling and we propose a formal approach for the design of artificial rational agents that includes some of the functions of emotions found in the human system. We suggest that emotions and rationality are closely linked in the human mind (and in the body, for that matter) and, therefore, need to be included in architectures for designing rational artificial agents, whether these agents are to interact with humans, to model humans' behaviors and actions, or both. We describe an Affective Knowledge Representation (AKR) scheme to represent emotion schemata, which we developed to guide the design of a variety of socially intelligent artificial agents. Our approach focuses on the notion of "social expertise" of socially intelligent agents in terms of their external behavior and internal motivational goal-based abilities. AKR, which uses probabilistic frames, is derived from combining multiple emotion theories into a hierarchical model of affective phenomena useful for artificial agent design. AKR includes a taxonomy of affect, mood, emotion, and personality, and a framework for emotional state dynamics using probabilistic Markov Models.     

A study of evolution strategy based cooperative behavior in collective agents

The following paper introduces an evolution strategy on the basis of cooperative behaviors in each group of agents. The evolution strategy helps each agent to be self-defendable and self-maintainable. To determine an optimal group behavior strategy under dynamically varying circumstances, agents in same group cooperate with each other. This proposed method use reinforcement learning, enhanced neural network, and artificial life. In the present paper, we apply two different reward models: reward model 1 and reward model 2. Each reward model is designed as considering the reinforcement or constraint of behaviors. In competition environments of agents, the behavior considered to be advantageous is reinforced as adding reward values. On the contrary, the behavior considered to be disadvantageous is constrained as subtracting the values. And we propose an enhanced neural network to add learning behavior of an artificial organism-level to artificial life simulation. In future, the system models and results described in this paper will be applied to the framework of healthcare systems that consists of biosensors, healthcare devices, and healthcare system.     

Soccer server: A tool for research on multiagent systems

This article describes Soccer Server, a simulator of the game of soccer designed as a benchmark for evaluating multiagent systems and cooperative algorithms. In real life, successful soccer teams require many qualities, such as basic ball control skills, the ability to carry out strategies, and teamwork. We believe that simulating such behaviors is a significant challenge for computer science, artificial intelligence, and robotics technologies. It is to promote the development of such technologies, and to help define a new standard problem for research, that we have developed Soccer Server. We demonstrate the potential of Soccer Server by reporting an experiment that uses the system to compare the performance of a neural network architecture and a decision tree algorithm at learning the selection of soccer play plans. Other researchers using Soccer Server to investigate the nature of cooperative behavior in a multiagent environment will have the chance to assess their progress at RoboCup-97, an international competition of robotic soccer to be held in conjunction with IJCAI-97. Soccer Server has been chosen as the official server for this contest.     

Dynamics of rules internalized in dynamic cognitive agents playing a multi-game

Rules such as laws, institutions, and norms can be changed dynamically in our society, because they are shaped by interactions among social members who are affected by them. However, there are also some stable rules enhanced by interactions among rules. In this article, we discuss whether or not rules can be stabilized by interactions among the rules. To investigate this, we propose a multi-game model in which different games are played simultaneously by the dynamic cognitive agents. A minority game (MG) and an n-person iterated prisoners’ dilemma game (NIPDG) are adopted. In our simulation, we found that the agents internalize the complex rules expressed as intricate geometrical shapes like strange attractors on the phase spaces, when the complex macro dynamics emerged. Furthermore, it showed that the macro dynamics shaped by the macro rules in the MG can be stabilized by interaction between the MG and the NIPDG rules internalized in the agents.     

A protocol for multi-agent negotiation in a group-choice decision making process

Negotiation is one of the most important features of agent interactions found in multi-agent systems, because it provides the basis for managing the expectations of the individual negotiating agents, and it enables selecting solutions that satisfy all the agents as much as possible. In order for negotiation to take place between two or more agents there is need for a negotiation protocol that defines the rules of the game; consequently, a variety of agent negotiation protocols have been proposed in literature. However, most of them are inappropriate for Group-Choice Decision Making (GCDM) because they do not explicitly exploit tradeoff to achieve social optimality, and their main focus is solving two-agent negotiation problems such as buyer–seller negotiation. In this paper we present an agent negotiation protocol that facilitates the solving of GCDM problems. The protocol is based on a hybrid of analytic and artificial intelligence techniques. The analytic component of the protocol utilizes a Game Theory model of an n-person general-sum game with complete information to determine the agreement options, while the knowledge-based (artificial intelligence) component of the protocol is similar to the strategic negotiation protocol. Moreover, this paper presents a tradeoff algorithm based on Qualitative Reasoning, which the agents employ to determine the ‘amount’ of tradeoff associated with various agreement options. Finally, the paper presents simulation results that illustrate the operational effectiveness of our agent negotiation protocol.     

Using commercial video games in flipped classrooms to support physical concept construction

Based on the flipped‐classroom model and the potential motivational and instructional benefits of digital games, we describe a flipped game‐based learning (FGBL) strategy focused on preclass and overall learning outcomes. A secondary goal is to determine the effects, if any, of the classroom aspects of the FGBL strategy on learning efficiency. Our experiments involved 2 commercial games featuring physical motion concepts: Ballance (Newton's law of motion) and Angry Birds (mechanical energy conservation). We randomly assigned 87 8th‐grade students to game instruction (digital game before class and lecture‐based instruction in class), FGBL strategy (digital game before class and cooperative learning in the form of group discussion and practice in class), or lecture‐based instruction groups (no gameplay). Results indicate that the digital games exerted a positive effect on preclass learning outcomes and that FGBL‐strategy students achieved better overall learning outcomes than their lecture‐based peers. Our observation of similar overall outcomes between the cooperative learning and lecture‐based groups suggests a need to provide additional teaching materials or technical support when introducing video games to cooperative classroom learning activities.     

Using trust for detecting deceitful agents in artificial societies

Trust is one of the most important concepts guiding decision-making and contracting in human societies. In artificial societies, this concept has been neglected until recently. The inherent benevolence assumption implemented in many multiagent systems can have hazardous consequences when dealing with deceit in open systems. The aim of this paper is to establish a mechanism that helps agents to cope with environments inhabited by both selfish and cooperative entities. This is achieved by enabling agents to evaluate trust in others. A formalization and an algorithm for trust are presented so that agents can autonomously deal with deception and identify trustworthy parties in open systems. The approach is twofold: agents can observe the behavior of others and thus collect information for establishing an initial trust model. In order to adapt quickly to a new or rapidly changing environment, one enables agents to also make use of observations from other agents. The practical relevance of these ideas is demonstrated by means of a direct mapping from a scenario to electronic commerce.     

Faster issue resolution with higher technical quality of software

We performed an empirical study of the relation between technical quality of software products and the issue resolution performance of their maintainers. In particular, we tested the hypothesis that ratings for source code maintainability, as employed by the Software Improvement Group (SIG) quality model, are correlated with ratings for issue resolution speed. We tested the hypothesis for issues of type defect and of type enhancement. This study revealed that all but one of the metrics of the SIG quality model show a significant positive correlation with the resolution speed of defects, enhancements, or both.