Multi-agent learning for engineers

As suggested by the title of Shoham, Powers, and Grenager's position paper [Y. Shoham, R. Powers, T. Grenager, If multi-agent learning is the answer, what is the question? Artificial Intelligence 171 (7) (2007) 365-377, this issue], the ultimate lens through which the multi-agent learning framework should be assessed is “what is the question?”. In this paper, we address this question by presenting challenges motivated by engineering applications and discussing the potential appeal of multi-agent learning to meet these challenges. Moreover, we highlight various differences in the underlying assumptions and issues of concern that generally distinguish engineering applications from models that are typically considered in the economic game theory literature.     

Distributed best response dynamics for Nash equilibrium seeking in potential games

In this paper, we consider distributed Nash equilibrium (NE) seeking in potential games over a multi-agent network, where each agent can not observe the actions of all its rivals. Based on the best response dynamics, we design a distributed NE seeking algorithm by incorporating the non-smooth finite-time average tracking dynamics, where each agent only needs to know its own action and exchange information with its neighbours through a communication graph. We give a sufficient condition for the Lipschitz continuity of the best response mapping for potential games, and then prove the convergence of the proposed algorithm based on the Lyapunov theory. Numerical simulations are given to verify the result and illustrate the effectiveness of the algorithm.     

Efficient wireless packet scheduling in a non-cooperative environment: Game theoretic analysis and algorithms

In many practical scenarios, wireless devices are autonomous and thus, may exhibit non-cooperative behaviors due to self-interests. For instance, a wireless cellular device may be programmed to report bogus channel information to gain resource allocation advantages. Such non-cooperative behaviors are highly probable as the device’s software can be modified by the user. In this paper, we first analyze the impact of these rationally selfish behaviors on the performance of packet scheduling algorithms in time-slotted wireless networks. Using a mixed strategy game model, we show that the traditional maximum rate packet scheduling algorithm can cause non-cooperative devices to converge to highly inefficient Nash equilibria, in which the wireless channel resources are significantly wasted. By using a repeated game to enforce cooperation, we further propose a novel game theoretic algorithm that can lead to an efficient equilibrium.     

Playing online games against computer- vs. human-controlled opponents: Effects on presence,flow, and enjoyment

The purpose of this study was to examine whether playing online games against other users leads to different experiences in comparison with playing against computer-controlled opponents. Thereby, a one-factorial multivariate design was used (computer-controlled vs. human-controlled opponent). Dependent variables were the participants’ feelings of presence and flow. Additionally, the amount of enjoyment was measured. The findings indicate that the type of opponent influences playing experiences: participants who played against a human-controlled opponent reported more experiences of presence, flow, and enjoyment, whereby the strongest effect refers to the experience of presence. Furthermore, strong relations between presence, flow, and enjoyment were observed. Further analyzes suggest that flow mediates the relationship between presence and enjoyment.     

A Nash equilibrium solution in an oligopoly market: The search for Nash equilibrium solutions with replicator equations derived from the gradient dynamics of a simplex algorithm

The present analysis applies continuous time replicator dynamics to the analysis of oligopoly markets. In the present paper, we discuss continuous game problems in which decision-making variables for each player are bounded on a simplex by equalities and non-negative constraints. Several types of problems are considered under conditions of normalized constraints and non-negative constraints. These problems can be classified into two types based on their constraints. For one type, the simplex constraint applies to the variables for each player independently, such as in a product allocation problem. For the other type, the simplex constraint applies to interference among all players, creating a market share problem. In the present paper, we consider a game problem under the constraints of allocation of product and market share simultaneously. We assume that a Nash equilibrium solution can be applied and derive the gradient system dynamics that attain the Nash equilibrium solution without violating the simplex constraints. Models assume that three or more firms exist in a market. Firms behave to maximize their profits, as defined by the difference between their sales and cost functions with conjectural variations. The effectiveness of the derived dynamics is demonstrated using simple data. The present approach facilitates understanding the process of attaining equilibrium in an oligopoly market.     

Dynamic policy in linear models with rational expectations of future events: A computer package

A computer package is presented, called POREM, for policy optimisation of linear dynamic, continuous-time models with constant coefficients and rational expectations of future events, based on infinite horizons and quadratic preferences. It is possible to calculate cooperative, decentralised Nash and decentralised Stackelberg outcomes and for each outcome it is possible to allow for pre-commitment and for lack of pre-commitment vis-à-vis private sector agents. It is possible to allow for hierarchical games, that is to allow for a group of Stackelberg leaders and a group of Stackelberg followers. The input of the model is very user-friendly and can be done with the aid of mnemonics. The package is programmed in PORTRAN77 and a single-precision version is available for personal computers     

Emissions control policies under uncertainty and rational learning in a linear-state dynamic model

We compare the use of price-based policies or taxes, and quantity-based policies or quotas, for controlling emissions in a dynamic setup when the regulator faces two sources of uncertainty: (i) market-related uncertainty; and (ii) ecological uncertainty. We assume that the regulator is a rational Bayesian learner and the regulator and firms have asymmetric information. In our model the structure of Bayesian learning is general. Our results suggest that the expected level of emissions is the same under taxes and quotas. However, the comparison of the total benefits related to these policies suggests that taxes dominate quotas, that is, they provide a higher social welfare. Even though taxes have some benefits over quotas, neither learning nor ecological uncertainty affect the choice of policy, i.e., the only factor having such an impact is uncertainty in the instantaneous net emissions benefits (market-related uncertainty). Besides, the more volatile is this uncertainty, the more benefits of taxes over quotas. Ecological uncertainty leads to a difference between the emissions rule under the informed and the rational learning assumptions. However, the direction of this difference depends on the beliefs bias with regard to ecological uncertainty. We also find that a change in the regulator’s beliefs toward more optimistic views will increase the emissions.     

Bounded Budget Connection (BBC) games or how to make friends and influence people,on a budget

Motivated by applications in social and peer-to-peer networks, we introduce the Bounded Budget Connection (BBC) game and study its pure Nash equilibria. We have a collection of n   players, each with a budget for purchasing links. Each link has a cost and a length. Each node has a preference weight for each node, and its objective is to purchase outgoing links within its budget to minimize its sum of preference-weighted distances to the nodes. We show that determining if a BBC game has pure Nash equilibria is NP-hard. We study (n,k)$(n,k)$-uniform BBC games, where all link costs, lengths, and preferences are equal and every budget equals k  . We show that pure Nash equilibria exist for all (n,k)$(n,k)$-uniform BBC games and all equilibria are essentially fair. We construct a family of equilibria spanning the spectrum from minimum to maximum social cost. We also analyze best-response walks and alternative node objectives.     

A social scientific framework for social systems in online video games: Building a better looking for raid loot system in World of Warcraft

This paper examines social behavior in the online video game World of Warcraft. Specifically focusing on one element of social design: the behavior of players in the first release of Looking-for-Raid (LFR) loot system of World of Warcraft. It uses lens of economic game theory, combined with Williams (2010) mapping principle and a modern theoretical account of human decision-making, to explore how theory about individual interactions in well-defined contexts (games) can explain collective behavior. It provides some support for this theoretical approach with an examination of data collected as part of an ethnographic study, through focus groups, and a survey distributed to 333 World of Warcraft players. It concludes with a discussion of the results and some guidelines for predicting collective outcomes in certain types of online games using the introduced framework.     

A two-level committee machine: a representation and a learning procedure for general piecewise linear discriminant functions

In order to improve the pattern classification power of the committee machine, a two-level committee machine, which is a committee machine with several lower committees, is proposed and the learning algorithm for it is described. The discriminant function realized by the two-level committee machine can be considered as the general piecewise linear discriminant function which includes Chang's definition.⁽¹⁵⁾ The proposed algorithm is a kind of error-correction procedure, and the learning procedure of the usual committee machine and the perceptron are clearly explained as special cases of the proposed algorithm.     

Game-based Learning and 21st century skills: A review of recent research

Game-based learning and 21st century skills have been gaining an enormous amount of attention from researchers and practitioners. Given numerous studies support the positive effects of games on learning, a growing number of researchers are committed to developing educational games to promote students’ 21st century skill development in schools. However, little is known regarding how games may influence student acquisition of 21st century skills. This paper examines the most recent literature in regard to game-based learning and identified 29 studies which targeted 21st century skills as outcomes. The range of game genres and game design elements as well as learning theories used in these studies are discussed, together with the range of indicators, measures and outcomes for impacts on 21st century skills. The findings suggest that a game-based learning approach might be effective in facilitating students’ 21st century skill development. The paper also provides valuable insights for researchers, game designers, and educators in issues related to educational game design and implementation in general.     

On convergence proofs in system identification – a general principle using ideas from learning theory

It is shown that learning theory offers convergence analysis tools that are useful in system identification problems. They allow analysis in a parameter-free context, which elevates the analysis from parameter sets to model sets and from parameter identification to model identification. When a parameterization is eventually introduced, this leads to alternative assumptions on the parameterization and parameter set. Moreover, structural identification can be analyzed within the same framework. Another advantage is that the proofs are technically and conceptually simple.     

NGI中一种基于食物链算法的柔性QoS组播路由算法

针对下一代互联网（Next Generation Internet,NGI）难以精确测量和用户服务质量（Quality of Service,QoS）需求难以完全表达的特点,设计了一种基于食物链算法（Food Chain Algorithm,FCA）的柔性QoS组播路由算法。给出了QoS组播路由问题模型及其数学描述,针对NGI中QoS参数信息不精确和用户需求柔性的特点,通过博弈分析确定用户和网络方在边上的效用能否达到Nash均衡,基于模糊数学的相关知识并结合FCA的寻优能力,找出在给定条件下用户效用、网络方效用和满足用户QoS需求的可信度同时达到最大的组播路由树。对算法进行了仿真实现与性能评价,结果表明,它是可行和有效的。     

Stable growing neural gas: A topology learning algorithm based on player tracking in video games

Characters in video games usually use a manually-defined topology of the environment to navigate. To evolve in an open, unknown and dynamic world, characters should not have pre-existing representations of their environment. In this paper, characters learn this representation by imitating human players. We here put forward a modified version of the growing neural gas model (GNG) called stable growing neural gas (SGNG). The algorithm is able to learn how to use the environment from one or more teachers (players) by representing it with a graph. Unlike GNG, SGNG learning is in-line, reflecting the dynamic nature of the environment. The evaluation of the quality of the learned representations are detailed.     

Investigating patterns of interaction in networked learning and computer-supported collaborative learning: A role for Social Network Analysis

The focus of this study is to explore the advances that Social Network Analysis (SNA) can bring, in combination with other methods, when studying Networked Learning/Computer-Supported Collaborative Learning (NL/CSCL). We present a general overview of how SNA is applied in NL/CSCL research; we then go on to illustrate how this research method can be integrated with existing studies on NL/CSCL, using an example from our own data, as a way to synthesize and extend our understanding of teaching and learning processes in NLCs. The example study reports empirical work using content analysis (CA), critical event recall (CER) and social network analysis (SNA). The aim is to use these methods to study the nature of the interaction patterns within a networked learning community (NLC), and the way its members share and construct knowledge. The paper also examines some of the current findings of SNA analysis work elsewhere in the literature, and discusses future prospects for SNA. This paper is part of a continuing international study that is investigating NL/CSCL among a community of learners engaged in a master’s program in e-learning.     

Goal setting outcomes: Examining the role of goal interaction in influencing the experience and learning outcomes of video game play for earthquake preparedness

Video game goals are important features of video games. Player’s interaction with goals can not only shape the gaming experience by evoking cognitive and affective reactions in players, but also lead to learning outcomes. However, there are few empirical studies on the effects of interacting with game goals, and no previous research has manipulated goal setting. In two experimental studies, participants were randomly assigned to one of the following five conditions: self-set goal repetitive play, assigned goal repetitive play, no-set goal repetitive play, no-set goal single play, and no play. Results show that playing earthquake preparedness video games generates significant learning outcomes; playing repeatedly with self-set goals yields greater learning compared to playing once with no-set goals or not playing; and cognitive reactions mediates the relation between goal interaction and learning. Implications of the results for the design and evaluation of future video games for learning are explored.     

Digital game-based learning in a Shanghai primary-school mathematics class: A case study

This study examined the perceptions and experiences of a teacher and students in a Shanghai public primary school when digital games were used in a second-grade math class. The participants included one teacher and 45 students. Data collection methods included classroom observation, focus-group and individual interviews, and document analysis. Digital gameplay, when used once daily over a 6-day period, was found to enhance student engagement and interest in learning; many students, however, were concerned about its effect on academic achievement and eyesight. The teacher employed a “making thinking visible” approach using pencil and paper and problem-solving strategies to help students understand the game's mathematical principles and master mathematical knowledge. Obstacles included large class size, difficulty evaluating learning outcomes, balancing fun and learning, and effective game–classroom integration. This study found that exam-oriented education and traditional teacher-centred teaching in China influenced participants' perceptions and experiences of digital game-based learning.     

Scaffolding student-generated questions: Design and development of a customizable online learning system

Despite the fact that the benefits of student question generation are well documented, most students do not take part in question generation exercises during their formal schooling and are not accustomed to authoring questions. Under the premise that student question generation activities should be better supported in a timely, flexible and logistically feasible fashion, a customizable online learning environment that accentuates various scaffolding techniques has been designed and developed. The framework guiding the development of the system, and its associated designs, are described. To assess the various built-in scaffolds used to support students’ learning activities by means of question generation, a study was undertaken to that measured students’ perceived usefulness of each mechanism, as well as the effects of the perceived usefulness of the scaffolds on students’ attitudes toward question generation learning activities in general. The data collected indicated that, by utilizing computers and network technologies, the developed system provided a supportive learning environment for student’s question generation learning activities. Support features not yet included in other similar systems (including access to generic question stems with sample questions, access to model questions, two-way cyclic communication between authors of question and assessors, and the ability to conceal one’s real identity by anonymity or nickname, etc.), were confirmed to provide a high level of support. Recommendations for classroom implementations and future studies are offered.     

A branch and bound algorithm for minimizing makespan on a single machine with unequal release times under learning effect and deteriorating jobs

We present a single-machine problem with the unequal release times under learning effect and deteriorating jobs when the objective is minimizing the makespan. In this study, we introduced a scheduling model with unequal release times in which both job deterioration and learning exist simultaneously. By the effects of learning and deterioration, we mean that the processing time of a job is defined by increasing function of its execution start time and position in the sequence. A branch-and-bound algorithm incorporating with several dominance properties and lower bounds is developed to derive the optimal solution. A heuristic algorithm is proposed to obtain a near-optimal solution. The computational experiments show that the branch-and-bound algorithm can solve instances up to 30 jobs, and the average error percentage of the proposed heuristic is less than 0.16%.