Learning dynamics explains human behaviour in Prisoner's Dilemma on networks

Cooperative behaviour lies at the very basis of human societies, yet its evolutionary origin remains a key unsolved puzzle. Whereas reciprocity or conditional cooperation is one of the most prominent mechanisms proposed to explain the emergence of cooperation in social dilemmas, recent experimental findings on networked Prisoner''s Dilemma games suggest that conditional cooperation also depends on the previous action of the player—namely on the ‘mood’ in which the player is currently in. Roughly, a majority of people behave as conditional cooperators if they cooperated in the past, whereas they ignore the context and free ride with high probability if they did not. However, the ultimate origin of this behaviour represents a conundrum itself. Here, we aim specifically to provide an evolutionary explanation of moody conditional cooperation (MCC). To this end, we perform an extensive analysis of different evolutionary dynamics for players'' behavioural traits—ranging from standard processes used in game theory based on pay-off comparison to others that include non-economic or social factors. Our results show that only a dynamic built upon reinforcement learning is able to give rise to evolutionarily stable MCC, and at the end to reproduce the human behaviours observed in the experiments.     

Beyond contact-based transmission networks: the role of spatial coincidence

Animal societies rely on interactions between group members to effectively communicate and coordinate their actions. To date, the transmission properties of interaction networks formed by direct physical contacts have been extensively studied for many animal societies and in all cases found to inhibit spreading. Such direct interactions do not, however, represent the only viable pathways. When spreading agents can persist in the environment, indirect transmission via ‘same-place, different-time’ spatial coincidences becomes possible. Previous studies have neglected these indirect pathways and their role in transmission. Here, we use rock ant colonies, a model social species whose flat nest geometry, coupled with individually tagged workers, allowed us to build temporally and spatially explicit interaction networks in which edges represent either direct physical contacts or indirect spatial coincidences. We show how the addition of indirect pathways allows the network to enhance or inhibit the spreading of different types of agent. This dual-functionality arises from an interplay between the interaction-strength distribution generated by the ants'' movement and environmental decay characteristics of the spreading agent. These findings offer a general mechanism for understanding how interaction patterns might be tuned in animal societies to control the simultaneous transmission of harmful and beneficial agents.     

Contact inhibition of locomotion probabilities drive solitary versus collective cell migration

Contact inhibition of locomotion (CIL) is the process whereby cells collide, cease migrating in the direction of the collision, and repolarize their migration machinery away from the collision. Quantitative analysis of CIL has remained elusive because cell-to-cell collisions are infrequent in traditional cell culture. Moreover, whereas CIL predicts mutual cell repulsion and ‘scattering’ of cells, the same cells in vivo are observed to undergo CIL at some developmental times and collective cell migration at others. It remains unclear whether CIL is simply absent during collective cell migration, or if the two processes coexist and are perhaps even related. Here, we used micropatterned stripes of extracellular matrix to restrict cell migration to linear paths such that cells polarized in one of two directions and collisions between cells occurred frequently and consistently, permitting quantitative and unbiased analysis of CIL. Observing repolarization events in different contexts, including head-to-head collision, head-to-tail collision, collision with an inert barrier, or no collision, and describing polarization as a two-state transition indicated that CIL occurs probabilistically, and most strongly upon head-to-head collisions. In addition to strong CIL, we also observed ‘trains’ of cells moving collectively with high persistence that appeared to emerge from single cells. To reconcile these seemingly conflicting observations of CIL and collective cell migration, we constructed an agent-based model to simulate our experiments. Our model quantitatively predicted the emergence of collective migration, and demonstrated the sensitivity of such emergence to the probability of CIL. Thus CIL and collective migration can coexist, and in fact a shift in CIL probabilities may underlie transitions between solitary cell migration and collective cell migration. Taken together, our data demonstrate the emergence of persistently polarized, collective cell movement arising from CIL between colliding cells.     

Limited interactions in flocks: relating model simulations to empirical data

The mechanism of self-organization resulting in coordinated collective motion has received wide attention from a range of scientists interested in both its technical and biological relevance. Models have been highly influential in highlighting how collective motion can be produced from purely local interactions between individuals. Typical models in this field are termed ‘metric’ because each individual only reacts to conspecifics within a fixed distance. A recent large-scale study has, however, provided evidence that interactions ruling collective behaviour occur between a fixed number of nearest neighbours (‘topological’ framework). Despite their importance in clarifying the nature of the mechanism underlying animal interactions, these findings have yet to be produced by either metric or topological models. Here, we present an original individual-based model of collective animal motion that reproduces the previous findings. Our approach bridges the current gap between previous model analysis and recent evidence, and presents a framework for further study.     

Emergence of collective changes in travel direction of starling flocks from individual birds' fluctuations

One of the most impressive features of moving animal groups is their ability to perform sudden coherent changes in travel direction. While this collective decision can be a response to an external alarm cue, directional switching can also emerge from the intrinsic fluctuations in individual behaviour. However, the cause and the mechanism by which such collective changes of direction occur are not fully understood yet. Here, we present an experimental study of spontaneous collective turns in natural flocks of starlings. We employ a recently developed tracking algorithm to reconstruct three-dimensional trajectories of each individual bird in the flock for the whole duration of a turning event. Our approach enables us to analyse changes in the individual behaviour of every group member and reveal the emergent dynamics of turning. We show that spontaneous turns start from individuals located at the elongated tips of the flocks, and then propagate through the group. We find that birds on the tips deviate from the mean direction of motion much more frequently than other individuals, indicating that persistent localized fluctuations are the crucial ingredient for triggering a collective directional change. Finally, we quantitatively verify that birds follow equal-radius paths during turning, the effects of which are a change of the flock''s orientation and a redistribution of individual locations in the group.     

Crowdsourcing contest dilemma

Crowdsourcing offers unprecedented potential for solving tasks efficiently by tapping into the skills of large groups of people. A salient feature of crowdsourcing—its openness of entry—makes it vulnerable to malicious behaviour. Such behaviour took place in a number of recent popular crowdsourcing competitions. We provide game-theoretic analysis of a fundamental trade-off between the potential for increased productivity and the possibility of being set back by malicious behaviour. Our results show that in crowdsourcing competitions malicious behaviour is the norm, not the anomaly—a result contrary to the conventional wisdom in the area. Counterintuitively, making the attacks more costly does not deter them but leads to a less desirable outcome. These findings have cautionary implications for the design of crowdsourcing competitions.     

The dynamics of audience applause

The study of social identity and crowd psychology looks at how and why individual people change their behaviour in response to others. Within a group, a new behaviour can emerge first in a few individuals before it spreads rapidly to all other members. A number of mathematical models have been hypothesized to describe these social contagion phenomena, but these models remain largely untested against empirical data. We used Bayesian model selection to test between various hypotheses about the spread of a simple social behaviour, applause after an academic presentation. Individuals'' probability of starting clapping increased in proportion to the number of other audience members already ‘infected’ by this social contagion, regardless of their spatial proximity. The cessation of applause is similarly socially mediated, but is to a lesser degree controlled by the reluctance of individuals to clap too many times. We also found consistent differences between individuals in their willingness to start and stop clapping. The social contagion model arising from our analysis predicts that the time the audience spends clapping can vary considerably, even in the absence of any differences in the quality of the presentations they have heard.     

Fishing out collective memory of migratory schools

Animals form groups for many reasons, but there are costs and benefits associated with group formation. One of the benefits is collective memory. In groups on the move, social interactions play a crucial role in the cohesion and the ability to make consensus decisions. When migrating from spawning to feeding areas, fish schools need to retain a collective memory of the destination site over thousands of kilometres, and changes in group formation or individual preference can produce sudden changes in migration pathways. We propose a modelling framework, based on stochastic adaptive networks, that can reproduce this collective behaviour. We assume that three factors control group formation and school migration behaviour: the intensity of social interaction, the relative number of informed individuals and the strength of preference that informed individuals have for a particular migration area. We treat these factors independently and relate the individuals’ preferences to the experience and memory for certain migration sites. We demonstrate that removal of knowledgeable individuals or alteration of individual preference can produce rapid changes in group formation and collective behaviour. For example, intensive fishing targeting the migratory species and also their preferred prey can reduce both terms to a point at which migration to the destination sites is suddenly stopped. The conceptual approaches represented by our modelling framework may therefore be able to explain large-scale changes in fish migration and spatial distribution.     

Algebraic specification of agent computation

Algebraic specification techniques can be applied to many aspects of software agents and multi-agent systems. This paper presents an algebraic framework, the Ψ- calculus, suitable for specifying the computations of complex agents that have a sense–compute–act computation cycle and that employ stored plan execution as the basis of behaviour generation. The framework provides a complete operational semantics of a plan-based programming language for such agents, covering all aspects of computation from the execution of basic plan steps through to the top-level control cycle which determines how external interactions occur. The semantics is specified compactly in process algebraic style by a hierarchy of rewrite rules, and has safety, guarantee and compositionality properties that facilitate reasoning about agent program behaviour.     

The interplay between a self-organized process and an environmental template: corpse clustering under the influence of air currents in ants

Many spatial patterns observed in nature emerge from local processes and their interactions with the local environment. The clustering of objects by social insects represents such a pattern formation process that can be observed at both the individual and the collective level. In this paper, we study the interaction between air currents and clustering behaviour in order to address the coordinating mechanisms at the individual level that underlie the spatial pattern formation process in a heterogeneous environment. We choose the corpse clustering behaviour of the ant Messor sanctus as an experimental paradigm. In a specifically designed experimental set-up with a well-controlled laminar air flow (approx. 1 cm s-1), we first quantify the modulation of the individual corpse aggregation behaviour as a function of corpse density, air flow intensity and the ant's position with respect to corpse piles and air flow direction. We then explore by numerical simulation how the forming corpse piles modify the laminar air flow around them and link this result with the individual behaviour modulation. Finally, we demonstrate on the collective level that this laminar air flow leads to an elongation and a slow displacement of the formed corpse piles in the direction of the air current. Both the individual behaviour modulated by air flow and the air flow modulated by the forming corpse piles can explain the pile patterns observed on the collective level as a stigmergic process. We discuss the generality of this coordinating mechanism to explain the clustering phenomena in heterogeneous environments reported in the literature.     

Predator confusion is sufficient to evolve swarming behaviour

Swarming behaviours in animals have been extensively studied owing to their implications for the evolution of cooperation, social cognition and predator–prey dynamics. An important goal of these studies is discerning which evolutionary pressures favour the formation of swarms. One hypothesis is that swarms arise because the presence of multiple moving prey in swarms causes confusion for attacking predators, but it remains unclear how important this selective force is. Using an evolutionary model of a predator–prey system, we show that predator confusion provides a sufficient selection pressure to evolve swarming behaviour in prey. Furthermore, we demonstrate that the evolutionary effect of predator confusion on prey could in turn exert pressure on the structure of the predator''s visual field, favouring the frontally oriented, high-resolution visual systems commonly observed in predators that feed on swarming animals. Finally, we provide evidence that when prey evolve swarming in response to predator confusion, there is a change in the shape of the functional response curve describing the predator''s consumption rate as prey density increases. Thus, we show that a relatively simple perceptual constraint—predator confusion—could have pervasive evolutionary effects on prey behaviour, predator sensory mechanisms and the ecological interactions between predators and prey.     

Metachronal waves in the flagellar beating of Volvox and their hydrodynamic origin

Groups of eukaryotic cilia and flagella are capable of coordinating their beating over large scales, routinely exhibiting collective dynamics in the form of metachronal waves. The origin of this behaviour—possibly influenced by both mechanical interactions and direct biological regulation—is poorly understood, in large part due to a lack of quantitative experimental studies. Here we characterize in detail flagellar coordination on the surface of the multicellular alga Volvox carteri, an emerging model organism for flagellar dynamics. Our studies reveal for the first time that the average metachronal coordination observed is punctuated by periodic phase defects during which synchrony is partial and limited to specific groups of cells. A minimal model of hydrodynamically coupled oscillators can reproduce semi-quantitatively the characteristics of the average metachronal dynamics, and the emergence of defects. We systematically study the model''s behaviour by assessing the effect of changing intrinsic rotor characteristics, including oscillator stiffness and the nature of their internal driving force, as well as their geometric properties and spatial arrangement. Our results suggest that metachronal coordination follows from deformations in the oscillators'' limit cycles induced by hydrodynamic stresses, and that defects result from sufficiently steep local biases in the oscillators'' intrinsic frequencies. Additionally, we find that random variations in the intrinsic rotor frequencies increase the robustness of the average properties of the emergent metachronal waves.     

Three-dimensional tracking and behaviour monitoring of multiple fruit flies

The increasing interest in the investigation of social behaviours of a group of animals has heightened the need for developing tools that provide robust quantitative data. Drosophila melanogaster has emerged as an attractive model for behavioural analysis; however, there are still limited ways to monitor fly behaviour in a quantitative manner. To study social behaviour of a group of flies, acquiring the position of each individual over time is crucial. There are several studies that have tried to solve this problem and make this data acquisition automated. However, none of these studies has addressed the problem of keeping track of flies for a long period of time in three-dimensional space. Recently, we have developed an approach that enables us to detect and keep track of multiple flies in a three-dimensional arena for a long period of time, using multiple synchronized and calibrated cameras. After detecting flies in each view, correspondence between views is established using a novel approach we call the ‘sequential Hungarian algorithm’. Subsequently, the three-dimensional positions of flies in space are reconstructed. We use the Hungarian algorithm and Kalman filter together for data association and tracking. We evaluated rigorously the system''s performance for tracking and behaviour detection in multiple experiments, using from one to seven flies. Overall, this system presents a powerful new method for studying complex social interactions in a three-dimensional environment.     

Fish and robots swimming together: attraction towards the robot demands biomimetic locomotion

The integration of biomimetic robots in a fish school may enable a better understanding of collective behaviour, offering a new experimental method to test group feedback in response to behavioural modulations of its ‘engineered’ member. Here, we analyse a robotic fish and individual golden shiners (Notemigonus crysoleucas) swimming together in a water tunnel at different flow velocities. We determine the positional preference of fish with respect to the robot, and we study the flow structure using a digital particle image velocimetry system. We find that biomimetic locomotion is a determinant of fish preference as fish are more attracted towards the robot when its tail is beating rather than when it is statically immersed in the water as a ‘dummy’. At specific conditions, the fish hold station behind the robot, which may be due to the hydrodynamic advantage obtained by swimming in the robot''s wake. This work makes a compelling case for the need of biomimetic locomotion in promoting robot–animal interactions and it strengthens the hypothesis that biomimetic robots can be used to study and modulate collective animal behaviour.     

Predictions of tumour morphological stability and evaluation against experimental observations

The hallmark of malignant tumours is their spread into neighbouring tissue and metastasis to distant organs, which can lead to life threatening consequences. One of the defining characteristics of aggressive tumours is an unstable morphology, including the formation of invasive fingers and protrusions observed both in vitro and in vivo. In spite of extensive biological, clinical and modelling study and research at physical scales ranging from the molecular to the tissue, the driving dynamics of tumour invasiveness are not completely understood, partly because it is challenging to observe and study cancer as a multi-scale system. Mathematical modelling has been applied to provide further insights into these complex invasive and metastatic behaviours. Modelling a solid tumour as an incompressible fluid, we consider three possible constitutive relations to describe tumour growth, namely Darcy''s law, Stokes'' law and the combined Darcy–Stokes law. We study the tumour morphological stability described by each model and evaluate the consistency between theoretical model predictions and experimental data from in vitro three-dimensional multicellular tumour spheroids. The analysis reveals that the Stokes model is the most consistent with the experimental observations, and that it predicts our experimental tumour growth is marginally stable. We further show that it is feasible to extract parameter values from a limited set of data and create a self-consistent modelling framework that can be extended to the multi-scale study of cancer.     

Interaction rules underlying group decisions in homing pigeons

Travelling in groups gives animals opportunities to share route information by following cues from each other''s movement. The outcome of group navigation will depend on how individuals respond to each other within a flock, school, swarm or herd. Despite the abundance of modelling studies, only recently have researchers developed techniques to determine the interaction rules among real animals. Here, we use high-resolution GPS (global positioning system) tracking to study these interactions in pairs of pigeons flying home from a familiar site. Momentary changes in velocity indicate alignment with the neighbour''s direction, as well as attraction or avoidance depending on distance. Responses were stronger when the neighbour was in front. From the flocking behaviour, we develop a model to predict features of group navigation. Specifically, we show that the interactions between pigeons stabilize a side-by-side configuration, promoting bidirectional information transfer and reducing the risk of separation. However, if one bird gets in front it will lead directional choices. Our model further predicts, and observations confirm, that a faster bird (as measured from solo flights) will fly slightly in front and thus dominate the choice of homing route. Our results explain how group decisions emerge from individual differences in homing flight behaviour.     

Structural mechanics of filamentous cyanobacteria

Filamentous cyanobacteria, forming long strands of connected cells, are one of the earliest and most successful forms of life on Earth. They exhibit self-organized behaviour, forming large-scale patterns in structures like biomats and stromatolites. The mechanical properties of these rigid structures have contributed to their biological success and are important to applications like algae-based biofuel production. For active polymers like these cyanobacteria, one of the most important mechanical properties is the bending modulus, or flexural rigidity. Here, we quantify the bending stiffness of three species of filamentous cyanobacteria, of order Oscillatoriales, using a microfluidic flow device where single filaments are deflected by fluid flow. This is complemented by measurements of Young’s modulus of the cell wall, via nanoindentation, and the cell wall thickness. We find that the stiffness of the cyanobacteria is well-captured by a simple model of a flexible rod, with most stress carried by a rigid outer wall. Finally, we connect these results to the curved shapes that these cyanobacteria naturally take while gliding, and quantify the forces generated internally to maintain this shape. The measurements can be used to model interactions between cyanobacteria, or with their environment, and how their collective behaviour emerges from such interactions.     

Determining asymptotically large population sizes in insect swarms

Social animals commonly form aggregates that exhibit emergent collective behaviour, with group dynamics that are distinct from the behaviour of individuals. Simple models can qualitatively reproduce such behaviour, but only with large numbers of individuals. But how rapidly do the collective properties of animal aggregations in nature emerge with group size? Here, we study swarms of Chironomus riparius midges and measure how their statistical properties change as a function of the number of participating individuals. Once the swarms contain order 10 individuals, we find that all statistics saturate and the swarms enter an asymptotic regime. The influence of environmental cues on the swarm morphology decays on a similar scale. Our results provide a strong constraint on how rapidly swarm models must produce collective states. But our findings support the feasibility of using swarms as a design template for multi-agent systems, because self-organized states are possible even with few agents.     

A unifying framework for quantifying the nature of animal interactions

Collective phenomena, whereby agent–agent interactions determine spatial patterns, are ubiquitous in the animal kingdom. On the other hand, movement and space use are also greatly influenced by the interactions between animals and their environment. Despite both types of interaction fundamentally influencing animal behaviour, there has hitherto been no unifying framework for the models proposed in both areas. Here, we construct a general method for inferring population-level spatial patterns from underlying individual movement and interaction processes, a key ingredient in building a statistical mechanics for ecological systems. We show that resource selection functions, as well as several examples of collective motion models, arise as special cases of our framework, thus bringing together resource selection analysis and collective animal behaviour into a single theory. In particular, we focus on combining the various mechanistic models of territorial interactions in the literature with step selection functions, by incorporating interactions into the step selection framework and demonstrating how to derive territorial patterns from the resulting models. We demonstrate the efficacy of our model by application to a population of insectivore birds in the Amazon rainforest.     

The diminishing role of hubs in dynamical processes on complex networks

It is notoriously difficult to predict the behaviour of a complex self-organizing system, where the interactions among dynamical units form a heterogeneous topology. Even if the dynamics of each microscopic unit is known, a real understanding of their contributions to the macroscopic system behaviour is still lacking. Here, we develop information-theoretical methods to distinguish the contribution of each individual unit to the collective out-of-equilibrium dynamics. We show that for a system of units connected by a network of interaction potentials with an arbitrary degree distribution, highly connected units have less impact on the system dynamics when compared with intermediately connected units. In an equilibrium setting, the hubs are often found to dictate the long-term behaviour. However, we find both analytically and experimentally that the instantaneous states of these units have a short-lasting effect on the state trajectory of the entire system. We present qualitative evidence of this phenomenon from empirical findings about a social network of product recommendations, a protein–protein interaction network and a neural network, suggesting that it might indeed be a widespread property in nature.