Learning dynamics explains human behaviour in Prisoner's Dilemma on networks

Cooperative behaviour lies at the very basis of human societies, yet its evolutionary origin remains a key unsolved puzzle. Whereas reciprocity or conditional cooperation is one of the most prominent mechanisms proposed to explain the emergence of cooperation in social dilemmas, recent experimental findings on networked Prisoner''s Dilemma games suggest that conditional cooperation also depends on the previous action of the player—namely on the ‘mood’ in which the player is currently in. Roughly, a majority of people behave as conditional cooperators if they cooperated in the past, whereas they ignore the context and free ride with high probability if they did not. However, the ultimate origin of this behaviour represents a conundrum itself. Here, we aim specifically to provide an evolutionary explanation of moody conditional cooperation (MCC). To this end, we perform an extensive analysis of different evolutionary dynamics for players'' behavioural traits—ranging from standard processes used in game theory based on pay-off comparison to others that include non-economic or social factors. Our results show that only a dynamic built upon reinforcement learning is able to give rise to evolutionarily stable MCC, and at the end to reproduce the human behaviours observed in the experiments.     

Conformity enhances network reciprocity in evolutionary social dilemmas

The pursuit of highest payoffs in evolutionary social dilemmas is risky and sometimes inferior to conformity. Choosing the most common strategy within the interaction range is safer because it ensures that the payoff of an individual will not be much lower than average. Herding instincts and crowd behaviour in humans and social animals also compel to conformity in their own right. Motivated by these facts, we here study the impact of conformity on the evolution of cooperation in social dilemmas. We show that an appropriate fraction of conformists within the population introduces an effective surface tension around cooperative clusters and ensures smooth interfaces between different strategy domains. Payoff-driven players brake the symmetry in favour of cooperation and enable an expansion of clusters past the boundaries imposed by traditional network reciprocity. This mechanism works even under the most testing conditions, and it is robust against variations of the interaction network as long as degree-normalized payoffs are applied. Conformity may thus be beneficial for the resolution of social dilemmas.     

The role of evolutionary game theory in spatial and non-spatial models of the survival of cooperation in cancer: a review

Evolutionary game theory (EGT) is a branch of mathematics which considers populations of individuals interacting with each other to receive pay-offs. An individual’s pay-off is dependent on the strategy of its opponent(s) as well as on its own, and the higher its pay-off, the higher its reproductive fitness. Its offspring generally inherit its interaction strategy, subject to random mutation. Over time, the composition of the population shifts as different strategies spread or are driven extinct. In the last 25 years there has been a flood of interest in applying EGT to cancer modelling, with the aim of explaining how cancerous mutations spread through healthy tissue and how intercellular cooperation persists in tumour-cell populations. This review traces this body of work from theoretical analyses of well-mixed infinite populations through to more realistic spatial models of the development of cooperation between epithelial cells. We also consider work in which EGT has been used to make experimental predictions about the evolution of cancer, and discuss work that remains to be done before EGT can make large-scale contributions to clinical treatment and patient outcomes.     

The emergence of cooperation from a single mutant during microbial life cycles

Cooperative behaviour is widespread in nature, even though cooperating individuals always run the risk of being exploited by free-riders. Population structure effectively promotes cooperation given that a threshold in the level of cooperation was already reached. However, the question how cooperation can emerge from a single mutant, which cannot rely on a benefit provided by other cooperators, is still puzzling. Here, we investigate this question for a well-defined but generic situation based on typical life cycles of microbial populations where individuals regularly form new colonies followed by growth phases. We analyse two evolutionary mechanisms favouring cooperative behaviour and study their strength depending on the inoculation size and the length of a life cycle. In particular, we find that population bottlenecks followed by exponential growth phases strongly increase the survival and fixation probabilities of a single cooperator in a free-riding population.     

Many ways to stay in the game: individual variability maintains high biodiversity in planktonic microorganisms

In apparent contradiction to competition theory, the number of known, coexisting plankton species far exceeds their explicable biodiversity—a discrepancy termed the Paradox of the Plankton. We introduce a new game-theoretic model for competing microorganisms in which one player consists of all organisms of one species. The stable points for the population dynamics in our model, known as strategic behaviour distributions (SBDs), are probability distributions of behaviours across all organisms which imply a stable population of the species as a whole. We find that intra-specific variability is the key characteristic that ultimately allows coexistence because the outcomes of competitions between individuals with variable competitive abilities are unpredictable. Our simulations based on the theoretical model show that up to 100 species can coexist for at least 10 000 generations, and that even small population sizes or species with inferior competitive ability can survive when there is intra-specific variability. In nature, this variability can be observed as niche differentiation, variability in environmental and ecological factors, and variability of individual behaviours or physiology. Therefore, previous specific explanations of the paradox are consistent with and provide specific examples of our suggestion that individual variability is the mechanism which solves the paradox.     

The organization and control of an evolving interdependent population

Starting with Darwin, biologists have asked how populations evolve from a low fitness state that is evolutionarily stable to a high fitness state that is not. Specifically of interest is the emergence of cooperation and multicellularity where the fitness of individuals often appears in conflict with that of the population. Theories of social evolution and evolutionary game theory have produced a number of fruitful results employing two-state two-body frameworks. In this study, we depart from this tradition and instead consider a multi-player, multi-state evolutionary game, in which the fitness of an agent is determined by its relationship to an arbitrary number of other agents. We show that populations organize themselves in one of four distinct phases of interdependence depending on one parameter, selection strength. Some of these phases involve the formation of specialized large-scale structures. We then describe how the evolution of independence can be manipulated through various external perturbations.     

Group size,individual role differentiation and effectiveness of cooperation in a homogeneous group of hunters

The emergence of cooperation in wolf-pack hunting is studied using a simple, homogeneous, particle-based computational model. Wolves and prey are modelled as particles that interact through attractive and repulsive forces. Realistic patterns of wolf aggregation readily emerge in numerical simulations, even though the model includes no explicit wolf–wolf attractive forces, showing that the form of cooperation needed for wolf-pack hunting can take place even among strangers. Simulations are used to obtain the stationary states and equilibria of the wolves and prey system and to characterize their stability. Different geometric configurations for different pack sizes arise. In small packs, the stable configuration is a regular polygon centred on the prey, while in large packs, individual behavioural differentiation occurs and induces the emergence of complex behavioural patterns between privileged positions. Stable configurations of large wolf-packs include travelling and rotating formations, periodic oscillatory behaviours and chaotic group behaviours. These findings suggest a possible mechanism by which larger pack sizes can trigger collective behaviours that lead to the reduction and loss of group hunting effectiveness, thus explaining the observed tendency of hunting success to peak at small pack sizes. They also explain how seemingly complex collective behaviours can emerge from simple rules, among agents that need not have significant cognitive skills or social organization.     

Collective sensing and collective responses in quorum-sensing bacteria

Bacteria often face fluctuating environments, and in response many species have evolved complex decision-making mechanisms to match their behaviour to the prevailing conditions. Some environmental cues provide direct and reliable information (such as nutrient concentrations) and can be responded to individually. Other environmental parameters are harder to infer and require a collective mechanism of sensing. In addition, some environmental challenges are best faced by a group of cells rather than an individual. In this review, we discuss how bacteria sense and overcome environmental challenges as a group using collective mechanisms of sensing, known as ‘quorum sensing’ (QS). QS is characterized by the release and detection of small molecules, potentially allowing individuals to infer environmental parameters such as density and mass transfer. While a great deal of the molecular mechanisms of QS have been described, there is still controversy over its functional role. We discuss what QS senses and how, what it controls and why, and how social dilemmas shape its evolution. Finally, there is a growing focus on the use of QS inhibitors as antibacterial chemotherapy. We discuss the claim that such a strategy could overcome the evolution of resistance. By linking existing theoretical approaches to data, we hope this review will spur greater collaboration between experimental and theoretical researchers.     

Social network architecture and the maintenance of deleterious cultural traits

How have changes in communications technology affected the way that misinformation spreads through a population and persists? To what extent do differences in the architecture of social networks affect the spread of misinformation, relative to the rates and rules by which individuals transmit or eliminate different pieces of information (cultural traits)? Here, we use analytical models and individual-based simulations to study how a ‘cultural load’ of misinformation can be maintained in a population under a balance between social transmission and selective elimination of cultural traits with low intrinsic value. While considerable research has explored how network architecture affects percolation processes, we find that the relative rates at which individuals transmit or eliminate traits can have much more profound impacts on the cultural load than differences in network architecture. In particular, the cultural load is insensitive to correlations between an individual''s network degree and rate of elimination when these quantities vary among individuals. Taken together, these results suggest that changes in communications technology may have influenced cultural evolution more strongly through changes in the amount of information flow, rather than the details of who is connected to whom.     

First carrot,then stick: how the adaptive hybridization of incentives promotes cooperation

Social institutions often use rewards and penalties to promote cooperation. Providing incentives tends to be costly, so it is important to find effective and efficient policies for the combined use of rewards and penalties. Most studies of cooperation, however, have addressed rewarding and punishing in isolation and have focused on peer-to-peer sanctioning as opposed to institutional sanctioning. Here, we demonstrate that an institutional sanctioning policy we call ‘first carrot, then stick’ is unexpectedly successful in promoting cooperation. The policy switches the incentive from rewarding to punishing when the frequency of cooperators exceeds a threshold. We find that this policy establishes and recovers full cooperation at lower cost and under a wider range of conditions than either rewards or penalties alone, in both well-mixed and spatial populations. In particular, the spatial dynamics of cooperation make it evident how punishment acts as a ‘booster stage’ that capitalizes on and amplifies the pro-social effects of rewarding. Together, our results show that the adaptive hybridization of incentives offers the ‘best of both worlds’ by combining the effectiveness of rewarding in establishing cooperation with the effectiveness of punishing in recovering it, thereby providing a surprisingly inexpensive and widely applicable method of promoting cooperation.     

Predator confusion is sufficient to evolve swarming behaviour

Swarming behaviours in animals have been extensively studied owing to their implications for the evolution of cooperation, social cognition and predator–prey dynamics. An important goal of these studies is discerning which evolutionary pressures favour the formation of swarms. One hypothesis is that swarms arise because the presence of multiple moving prey in swarms causes confusion for attacking predators, but it remains unclear how important this selective force is. Using an evolutionary model of a predator–prey system, we show that predator confusion provides a sufficient selection pressure to evolve swarming behaviour in prey. Furthermore, we demonstrate that the evolutionary effect of predator confusion on prey could in turn exert pressure on the structure of the predator''s visual field, favouring the frontally oriented, high-resolution visual systems commonly observed in predators that feed on swarming animals. Finally, we provide evidence that when prey evolve swarming in response to predator confusion, there is a change in the shape of the functional response curve describing the predator''s consumption rate as prey density increases. Thus, we show that a relatively simple perceptual constraint—predator confusion—could have pervasive evolutionary effects on prey behaviour, predator sensory mechanisms and the ecological interactions between predators and prey.     

Ordering structured populations in multiplayer cooperation games

Spatial structure greatly affects the evolution of cooperation. While in two-player games the condition for cooperation to evolve depends on a single structure coefficient, in multiplayer games the condition might depend on several structure coefficients, making it difficult to compare different population structures. We propose a solution to this issue by introducing two simple ways of ordering population structures: the containment order and the volume order. If population structure is greater than population structure in the containment or the volume order, then can be considered a stronger promoter of cooperation. We provide conditions for establishing the containment order, give general results on the volume order, and illustrate our theory by comparing different models of spatial games and associated update rules. Our results hold for a large class of population structures and can be easily applied to specific cases once the structure coefficients have been calculated or estimated.     

Resistance and relatedness on an evolutionary graph

When investigating evolution in structured populations, it is often convenient to consider the population as an evolutionary graph—individuals as nodes, and whom they may act with as edges. There has, in recent years, been a surge of interest in evolutionary graphs, especially in the study of the evolution of social behaviours. An inclusive fitness framework is best suited for this type of study. A central requirement for an inclusive fitness analysis is an expression for the genetic similarity between individuals residing on the graph. This has been a major hindrance for work in this area as highly technical mathematics are often required. Here, I derive a result that links genetic relatedness between haploid individuals on an evolutionary graph to the resistance between vertices on a corresponding electrical network. An example that demonstrates the potential computational advantage of this result over contemporary approaches is provided. This result offers more, however, to the study of population genetics than strictly computationally efficient methods. By establishing a link between gene transfer and electric circuit theory, conceptualizations of the latter can enhance understanding of the former.     

A sequential-move game for enhancing safety and security cooperation within chemical clusters

The present paper provides a game theoretic analysis of strategic cooperation on safety and security among chemical companies within a chemical industrial cluster. We suggest a two-stage sequential move game between adjacent chemical plants and the so-called Multi-Plant Council (MPC). The MPC is considered in the game as a leader player who makes the first move, and the individual chemical companies are the followers. The MPC's objective is to achieve full cooperation among players through establishing a subsidy system at minimum expense. The rest of the players rationally react to the subsidies proposed by the MPC and play Nash equilibrium. We show that such a case of conflict between safety and security, and social cooperation, belongs to the 'coordination with assurance' class of games, and we explore the role of cluster governance (fulfilled by the MPC) in achieving a full cooperative outcome in domino effects prevention negotiations. The paper proposes an algorithm that can be used by the MPC to develop the subsidy system. Furthermore, a stepwise plan to improve cross-company safety and security management in a chemical industrial cluster is suggested and an illustrative example is provided.     

Mechanistic analysis of the search behaviour of Caenorhabditis elegans

A central question in movement research is how animals use information and movement to promote encounter success. Current random search theory identifies reorientation patterns as key to the compromise between optimizing encounters for both nearby and faraway targets, but how the balance between intrinsic motor programmes and previous environmental experience determines the occurrence of these reorientation behaviours remains unknown. We used high-resolution tracking and imaging data to describe the complete motor behaviour of Caenorhabditis elegans when placed in a novel environment (one in which food is absent). Movement in C. elegans is structured around different reorientation behaviours, and we measured how these contributed to changing search strategies as worms became familiar with their new environment. This behavioural transition shows that different reorientation behaviours are governed by two processes: (i) an environmentally informed ‘extrinsic’ strategy that is influenced by recent experience and that controls for area-restricted search behaviour, and (ii) a time-independent, ‘intrinsic’ strategy that reduces spatial oversampling and improves random encounter success. Our results show how movement strategies arise from a balance between intrinsic and extrinsic mechanisms, that search behaviour in C. elegans is initially determined by expectations developed from previous environmental experiences, and which reorientation behaviours are modified as information is acquired from new environments.     

供应链系统的部分合作博弈研究

针对供应链信息和知识共享过程中存在的由于信任机制欠缺和协议约束松弛而引发的合作障碍,运用合作博弈思想提出了部分合作博弈理论,分析部分合作博弈的生成机理,设计了改善的部分合作博弈策略,转换和优化重复博弈的动态结构.给出了若干建议,力求获取与加强诚信、提高供应链合作程度.     

The principles of pedestrian route choice

Pedestrian route choice, the process by which individuals decide on their walking path between two locations, is a fundamental problem across disciplines. Because this behaviour is investigated from different conceptual and methodological angles, and because it strongly depends on the environmental context, it is challenging to establish a systematic framework for research. Here, by reviewing previous work, we identify four principles for pedestrian route choice that are relevant across disciplines. First, ‘information perception’ deals with how pedestrians can perceive information selectively and purposely, given the limited available information. Second, ‘information integration’ considers how pedestrians subjectively integrate environmental spatial information into mental representations. Third, ‘responding to information’ is concerned with how pedestrians tend to be attracted and repelled by specific attributes individually and how this can lead to positive or negative feedback loops across many individuals. Fourth ‘decision-making mechanisms'' describe how pedestrians trade off the evidence provided by different attributes. How pedestrians perceive, integrate, respond to, and act upon information is not fixed but varies with the context. We give examples for each principle and explain how these principles shape pedestrian choice behaviours. We hope this contribution provides a systematic overview of the field and helps to spark inspiration among specialists.     

Signalling boosts the evolution of cooperation in repeated group interactions

Many biological and social systems show significant levels of collective action. Several cooperation mechanisms have been proposed, yet they have been mostly studied independently. Among these, direct reciprocity supports cooperation on the basis of repeated interactions among individuals. Signals and quorum dynamics may also drive cooperation. Here, we resort to an evolutionary game-theoretical model to jointly analyse these two mechanisms and study the conditions in which evolution selects for direct reciprocity, signalling, or their combination. We show that signalling alone leads to higher levels of cooperation than when combined with reciprocity, while offering additional robustness against errors. Specifically, successful strategies in the realm of direct reciprocity are often not selected in the presence of signalling, and memory of past interactions is only exploited opportunistically in the case of earlier coordination failure. Differently, signalling always evolves, even when costly. In the light of these results, it may be easier to understand why direct reciprocity has been observed only in a limited number of cases among non-humans, whereas signalling is widespread at all levels of complexity.     

Selection methods regulate evolution of cooperation in digital evolution

A key, yet often neglected, component of digital evolution and evolutionary models is the ‘selection method’ which assigns fitness (number of offspring) to individuals based on their performance scores (efficiency in performing tasks). Here, we study with formal analysis and numerical experiments the evolution of cooperation under the five most common selection methods (proportionate, rank, truncation-proportionate, truncation-uniform and tournament). We consider related individuals engaging in a Prisoner''s Dilemma game where individuals can either cooperate or defect. A cooperator pays a cost, whereas its partner receives a benefit, which affect their performance scores. These performance scores are translated into fitness by one of the five selection methods. We show that cooperation is positively associated with the relatedness between individuals under all selection methods. By contrast, the change in the performance benefit of cooperation affects the populations’ average level of cooperation only under the proportionate methods. We also demonstrate that the truncation and tournament methods may introduce negative frequency-dependence and lead to the evolution of polymorphic populations. Using the example of the evolution of cooperation, we show that the choice of selection method, though it is often marginalized, can considerably affect the evolutionary dynamics.     

A statistical method for identifying different rules of interaction between individuals in moving animal groups

The emergent patterns of collective motion are thought to arise from application of individual-level rules that govern how individuals adjust their velocity as a function of the relative position and behaviours of their neighbours. Empirical studies have sought to determine such rules of interaction applied by ‘average’ individuals by aggregating data from multiple individuals across multiple trajectory sets. In reality, some individuals within a group may interact differently from others, and such individual differences can have an effect on overall group movement. However, comparisons of rules of interaction used by individuals in different contexts have been largely qualitative. Here we introduce a set of randomization methods designed to determine statistical differences in the rules of interaction between individuals. We apply these methods to a case study of leaders and followers in pairs of freely exploring eastern mosquitofish (Gambusia holbrooki). We find that each of the randomization methods is reliable in terms of: repeatability of p-values, consistency in identification of significant differences and similarity between distributions of randomization-based test statistics. We observe convergence of the distributions of randomization-based test statistics across repeat calculations, and resolution of any ambiguities regarding significant differences as the number of randomization iterations increases.