Multi-agent learning for engineers

As suggested by the title of Shoham, Powers, and Grenager's position paper [Y. Shoham, R. Powers, T. Grenager, If multi-agent learning is the answer, what is the question? Artificial Intelligence 171 (7) (2007) 365-377, this issue], the ultimate lens through which the multi-agent learning framework should be assessed is “what is the question?”. In this paper, we address this question by presenting challenges motivated by engineering applications and discussing the potential appeal of multi-agent learning to meet these challenges. Moreover, we highlight various differences in the underlying assumptions and issues of concern that generally distinguish engineering applications from models that are typically considered in the economic game theory literature.     

Multiagent learning is not the answer. It is the question

The article by Shoham, Powers, and Grenager called “If multi-agent learning is the answer, what is the question?” does a great job of laying out the current state of the art and open issues at the intersection of game theory and artificial intelligence (AI). However, from the AI perspective, the term “multiagent learning” applies more broadly than can be usefully framed in game theoretic terms. In this larger context, how (and perhaps whether) multiagent learning can be usefully applied in complex domains is still a large open question.     

No regrets about no-regret

No-regret is described as one framework that game theorists and computer scientists have converged upon for designing and evaluating multi-agent learning algorithms. However, Shoham, Powers, and Grenager also point out that the framework has serious deficiencies, such as behaving sub-optimally against certain reactive opponents. But all is not lost. With some simple modifications, regret-minimizing algorithms can perform in many of the ways we wish multi-agent learning algorithms to perform, providing safety and adaptability against reactive opponents. We argue that the research community should have no regrets about no-regret methods.     

An economist's perspective on multi-agent learning

We comment on the Shoham, Powers, and Grenager survey of multi-agent learning and game theory, emphasizing that some of their categories are important for economics and others are not. We also try to correct some minor imprecisions in their discussion of the economics literature on learning in games.     

基于多Agent的网上教学模型的研究

Agent是人工智能和计算机软件领域的一种新兴的技术,智能Agent技术近年来有了很大的发展,本文基于多Agent技术构建了一个网上教学系统模型,为学生和教师提供虚拟教学环境,实现学生之间的协作化学习。     

基于多Agent的网上教学模型的研究

Agent是人工智能和计算机软件领域的一种新兴的技术,智能Agent技术近年来有了很大的发展．本文基于多Agent技术构建了一个网上教学系统模型,为学生和教师提供虚拟教学环境,实现学生之间的协作化学习。     

An online game approach for improving students’ learning performance in web-based problem-solving activities

In this paper, an online game was developed in the form of a competitive board game for conducting web-based problem-solving activities. The participants of the game determined their move by throwing a dice. Each location of the game board corresponds to a gaming task, which could be a web-based information-searching question or a mini-game; the former was used to guide the participants to search for information to answer a series of questions related to the target learning issue, while the latter was used to provide supplementary materials during the gaming process. To evaluate the performance of the proposed approach, an experiment was conducted on an elementary school natural science course. The experimental results showed that the proposed approach not only significantly promoted the flow experience, learning attitudes, learning interest and technology acceptance degree of the students, but also improved their learning achievements in the web-based problem-solving activity.     

From symbolic to sub-symbolic information in question classification

Question Answering (QA) is undoubtedly a growing field of current research in Artificial Intelligence. Question classification, a QA subtask, aims to associate a category to each question, typically representing the semantic class of its answer. This step is of major importance in the QA process, since it is the basis of several key decisions. For instance, classification helps reducing the number of possible answer candidates, as only answers matching the question category should be taken into account. This paper presents and evaluates a rule-based question classifier that partially founds its performance in the detection of the question headword and in its mapping into the target category through the use of WordNet. Moreover, we use the rule-based classifier as a features’ provider of a machine learning-based question classifier. A detailed analysis of the rule-base contribution is presented. Despite using a very compact feature space, state of the art results are obtained.     

Serious games and learning effectiveness: The case of It’s a Deal!

Although the value of serious games in education is undeniable and the potential benefits of using video games as ideal companions to classroom instruction is unquestionable, there is still little consensus on the game features supporting learning effectiveness, the process by which games engage learners, and the types of learning outcomes that can be achieved through game play. Our aim in this discussion is precisely to advance in this direction by providing evidence of some of the factors influencing the learning effectiveness of a serious game called It’s a Deal! This serious game was created for the purpose of teaching intercultural business communication between Spaniards and Britons in business settings in which English is used as the lingua franca. This paper hypothesizes that the immersive, all-embracing and interactive learning environment provided by the video game to its users may contribute to develop and enhance their intercultural communicative competence. The study attempts to answer three main research questions: (a) after playing It’s a Deal!, did the students sampled improve their intercultural awareness, intercultural knowledge and intercultural communicative competence in business English? (b) If they improved their intercultural learning, what are the factors influencing such improvement? And (c) if they did not improve their intercultural learning, what are the factors influencing such failure? The game participants who volunteered to take part in the study were all students of English Studies at the University of Alicante in the academic year 2010-2011. One hundred and six students completed both the pre-test and the post-test questionnaires, and played It’s a Deal! A sample of fifty students was selected randomly for the empirical study. The results obtained in the tests performed were compared and contrasted intra-group, both qualitatively and quantitatively, for the purpose of finding any statistically significant difference that may confirm whether or not there was an improvement in the students’ intercultural communicative competence in business English as a result of the implementation of the It’s a Deal! serious game. Findings of this study demonstrate that the video game is an effective learning tool for the teaching of intercultural communication between Spaniards and Britons in business settings in which English is used as the lingua franca. In particular, whereas the game had a small learning effect on intercultural awareness and a medium learning effect on intercultural knowledge, it had a large learning effect on intercultural communicative competence. The study also documents correlating factors that make serious games effective, since it shows that the learning effectiveness of It’s a Deal! stems from the correct balance of the different dimensions involved in the creation of serious games, specifically instructional content, game dimensions, game cycle, debriefing, perceived educational value, transfer of learnt skills and intrinsic motivation.     

随机博弈框架下的多agent强化学习方法综述

多agent学习是在随机博弈的框架下,研究多个智能体间通过自学习掌握交互技巧的问题.单agent强化学习方法研究的成功,对策论本身牢固的数学基础以及在复杂任务环境中广阔的应用前景,使得多agent强化学习成为目前机器学习研究领域的一个重要课题.首先介绍了多agent系统随机博弈中基本概念的形式定义;然后介绍了随机博弈和重复博弈中学习算法的研究以及其他相关工作;最后结合近年来的发展,综述了多agent学习在电子商务、机器人以及军事等方面的应用研究,并介绍了仍存在的问题和未来的研究方向.     

A multi-agent system model of a diesel engine using rule induction

The proliferation of a multi-agent system (MAS) and ideas from Artificial Intelligence (AI)/distributed AI have changed the way systems, in general are controlled, and operation of a system (diesel engine) in particular is automated. In this paper a distributed multi-agent architecture for a diesel engine and the knowledge sources that handle electricity generation is developed. Electronic devices and components used for data handling are described. The sensed data are presented in fuzzy logic and calculated in entropy values and depicted in a decision hierarchy. A comparative performance assessment of the proposed multi-agent based system with an existing system is presented and discussed.     

强化学习算法与应用综述

强化学习是机器学习领域的研究热点, 是考察智能体与环境的相互作用, 做出序列决策、优化策略并最大化累积回报的过程. 强化学习具有巨大的研究价值和应用潜力, 是实现通用人工智能的关键步骤. 本文综述了强化学习算法与应用的研究进展和发展动态, 首先介绍强化学习的基本原理, 包括马尔可夫决策过程、价值函数、探索-利用问题. 其次, 回顾强化学习经典算法, 包括基于价值函数的强化学习算法、基于策略搜索的强化学习算法、结合价值函数和策略搜索的强化学习算法, 以及综述强化学习前沿研究, 主要介绍多智能体强化学习和元强化学习方向. 最后综述强化学习在游戏对抗、机器人控制、城市交通和商业等领域的成功应用, 以及总结与展望.     

A Perspective View and Survey of Meta-Learning

Different researchers hold different views of what the term meta-learning exactlymeans. The first part of this paper provides our own perspective view in which the goal isto build self-adaptive learners (i.e. learning algorithms that improve their bias dynamicallythrough experience by accumulating meta-knowledge). The second part provides a survey ofmeta-learning as reported by the machine-learning literature. We find that, despite differentviews and research lines, a question remains constant: how can we exploit knowledge aboutlearning (i.e. meta-knowledge) to improve the performance of learning algorithms? Clearlythe answer to this question is key to the advancement of the field and continues being thesubject of intensive research.     

Prototypes construction from partial rankings to characterize the attractiveness of companies in Belgium

What are the most relevant factors to be considered by employees when searching for an employer? The answer to this question poses valuable knowledge from the Business Intelligence viewpoint since it allows companies to retain personnel and attract competent employees. It leads to an increase in sales of their products or services, therefore remaining competitive across similar companies in the market. In this paper we assess the attractiveness of companies in Belgium by using a new two-stage methodology based on Artificial Intelligence techniques. The proposed method allows constructing high-quality prototypes from partial rankings indicating experts’ preferences. Being more explicit, in the first step we propose a fuzzy clustering algorithm for partial rankings called fuzzy c-aggregation. This algorithm is based on the well-known fuzzy c-means procedure and uses the Hausdorff distance as dissimilarity functional and a counting strategy for updating the center of each cluster. However, we cannot ensure the optimality of such prototypes, and therefore more accurate prototypes must be derived. That is why the second step is focused on solving the extended Kemeny ranking problem for each discovered cluster taking into account the estimated membership matrix. To accomplish that, we adopt an optimization method based on Swarm Intelligence that exploits a colony of artificial ants. Several simulations show the effectiveness of the proposal for the real-world problem under investigation.     

Pre-run-time scheduling in real-time systems: Current researches and Artificial Intelligence perspectives

This paper presents the taxonomy of real-time systems with special emphasize on pre-run-time scheduling problem. Firstly, we present real-time systems, real-time tasks, timing, precedence and exclusion constraints. Then, we describe the problem of pre-run-time scheduling of tasks under constraints. After that, we present the most existing efficient techniques to deal with the latter problem. We summarize the discussion of existing techniques and possible research perspectives after surveying the Artificial Intelligence’s point of view about the problem of pre-run-time scheduling of real-time tasks. The Artificial Intelligence survey includes Constraint Satisfaction Problems class since pre-run-time scheduling belongs to the latter class. The Artificial Intelligence survey includes also Path-finding Problems from which intelligent algorithms could be observed such as Learning-Real-Time-A1(LRTA1) thanks to its important properties (optimality, linear space complexity and determinism). The development of an algorithm like LRTA1 to solve Constraints Satisfaction Problems and particularly the pre-run-time scheduling of real-time tasks problem is one clear research direction to deal with large-scale real-time systems. The overall objective of this paper is to show what are the perspectives to Artificial Intelligence literature that could be beneficial firstly to Artificial Intelligence community itself and secondly to real-time systems community.     

Concept learning games

In this paper, we intend to have a game theoretic study on the concept learning problem in a multi-agent system. Concept learning is a very essential and well-studied domain of machine learning when it is studied under the characteristics of a multi-agent system. The most important reasons are the partiality of the environment perception for any agent and also the communication holdbacks, resulting into a deep need for a collaborative protocol in favor of multi-agent transactions. Here we wish to investigate multi-agent concept learning with the help of its components, thoroughly with a game theoretic taste, esp. on the pre-learning processes. Based on two standard notations, we address the non-unanimity of concepts, classification of objects, voting and communicating protocol, and also the learning itself. In such a game of concept learning, we consider a group of agents, communicating and consulting to upgrade their ontologies based on their conceptualizations of the environment. For this purpose, we investigate the problem in two separate and standard distinctions of game theory study, cooperation and competition. Several solution concepts and innovative ideas from the multi-agent realm are used to produce an approach that contains the reasoning process of the agents in this system. Some experimentations come at the end to show the functionality of our approach. These experimentations come distinctly for both cooperative and competitive views.     

Editorial: Special issue on multiagent-based system modeling and control practices

Dealing with a large-scaled system as multi-agent system is not a new methodology in system control field. The root of multi-agent system is longer than half century, however, it is recently spotlighted again due to the needs of theoretical tools for dealing with large-scale system such as smart city, global behavior of traffic system, networked systems, and machine learning. Common characteristic of this kind of systems is that the whole system consists of a larger number of autonomous agents which are coupled and connected each other, and the individual agent is under local control but it accomplishes a task as the whole system. This collection of research papers is from the submissions to the special issue with title ``multiagent-based system modeling and control practices''. The aim of the special issue is to provide a stage for both of theorists and practitioners in this field to exchange challenging results in new issues, especially biological, bio-inspired systems, mean-field game, and connected network systems. The collection is divided as three groups: The survey paper by K. Hou et al. presented an overview on recent advances in control and communication of multi-robot swarms. A biomolecular control scheme is proposed in the paper by P. Rong and T. Nakakuki which demonstrated how the dynamic DNA nanotechnology can be used in system control field. The last brief paper of this group is from S. Azuma which discussed the network structure for Boolean network systems. The second group includes three case studies from the view of creating new theoretical tools. The paper by T. Wang et al. proposed an online iterative algorithm for solving multi-agent dynamic graphical games which showd the possibility of online solution of Bellman equation. Y. Du et al. presented in their paper a distributed scheme for ensemble learning under a diffusion strategy which aimed to the classification problem on big data. The paper by Z. Lu et al. addressed a mixed-triggered finite-time non-fragile filtering problem for interval type-2 T-S fuzzy network control systems. The last group of this collection includes three papers with practical background. The paper by J. Zhang and F. Xu investigated the problem of minimizing energy consumption for connected hybrid electric vehicles, and the paper from Q. Fu et al. presented a challenge of applying mean-field game theory to achieve speed consensus for the vehicle driving in a large-scale traffic flow. Targeted on the alternative transportation devices, high-speed train with connected real-time traffic information is addressed in the paper by D. He et al. with energy-efficient receding horizon trajectory planning method.     

Creating knowledge maps by exploiting dependent relationships

Knowledge is an interesting concept that has attracted the attention of philosophers for thousands of years. In more recent times, researchers have investigated knowledge in a more applied way with the chief aim of bringing knowledge to life in machines. Artificial Intelligence has provided some degree of rigour to the study of knowledge and Expert Systems are able to use knowledge to solve problems and answer questions.Current business, social, political and technological pressures have forced organisations to take greater control of the knowledge asset. Software suppliers and others offering valuable solutions in this area have unfortunately clouded the issue of knowledge. Information and data control are seen as implicit knowledge management tools and many have abandoned the search for explicit knowledge management methods.Knowledge representation schemes help to identify knowledge. They allow for human understanding and machine application and they can support the automated use of knowledge in problem solving. Some of these representation methods also employ spatial techniques that add an extra dimension to human understanding.Knowledge mapping defined in this work uses learning dependency to organise the map and draws on the ideas of what knowledge is and on spatial representation structures. Knowledge maps can support metrics that provide information about the knowledge asset. Knowledge maps create a visible knowledge framework that supports the explicit management of knowledge by organisation managers and directors. Knowledge maps also offer other advantages to the organisation, the individual and to educational institutions.     

Computing machinery and morality

Artificial Intelligence (AI) is a technology widely used to support human decision-making. Current areas of application include financial services, engineering, and management. A number of attempts to introduce AI decision support systems into areas which more obviously include moral judgement have been made. These include systems that give advice on patient care, on social benefit entitlement, and even ethical advice for medical professionals. Responding to these developments raises a complex set of moral questions. This paper proposes a clearer replacement question to them. The replacement question asks under what circumstances, if any, people would accept a moral judgement made by some sort of machine. Since, it is argued, the answer to this replacement question is positive, urgent practical moral problems are raised.

---