Machine learning in digital games: a survey

Artificial intelligence for digital games constitutes the implementation of a set of algorithms and techniques from both traditional and modern artificial intelligence in order to provide solutions to a range of game dependent problems. However, the majority of current approaches lead to predefined, static and predictable game agent responses, with no ability to adjust during game-play to the behaviour or playing style of the player. Machine learning techniques provide a way to improve the behavioural dynamics of computer controlled game agents by facilitating the automated generation and selection of behaviours, thus enhancing the capabilities of digital game artificial intelligence and providing the opportunity to create more engaging and entertaining game-play experiences. This paper provides a survey of the current state of academic machine learning research for digital game environments, with respect to the use of techniques from neural networks, evolutionary computation and reinforcement learning for game agent control.     

计算机博弈中序贯不完美信息博弈求解研究进展

计算机博弈是人工智能的果蝇和通用测试基准.近年来,序贯不完美信息博弈求解一直是计算机博弈研究领域的前沿课题.围绕计算机博弈中不完美信息博弈求解问题展开综述分析.首先,梳理计算机博弈领域标志性突破的里程碑事件,简要介绍4类新评估基准,归纳3种研究范式,提出序贯不完美信息博弈求解研究框架;然后,着重对序贯不完美信息博弈的博弈模型和解概念进行调研,从博弈构建、子博弈和元博弈、解概念以及评估3方面进行简要介绍;接着,围绕离线策略求解,系统梳理算法博弈论、优化理论和博弈学习3大类方法,围绕在线策略求解,系统梳理对手近似式学习、对手判别式适变和对手生成式搜索3大类方法;最后,从环境、智能体(对手)和策略求解3个角度分析面临的挑战,从博弈动力学和策略空间理论、多模态对抗博弈和序贯建模、通用策略学习和离线预训练、对手建模(剥削)和反剥削、临机组队和零样本协调5方面展望未来研究前沿课题.对于当前不完美信息博弈求解问题进行全面概述,期望能够为人工智能和博弈论领域相关研究带来启发.     

实时格斗游戏的智能决策方法

格斗游戏作为实时双人零和对抗博弈的代表性问题,具有实时对抗和快速响应的重要研究特性.相应针对性方法的提出有效反映了游戏人工智能领域的重要研究进展及发展方向.本文以格斗游戏人工智能竞赛作为研究背景,将智能决策方法分为启发式规则型、统计前向规划型与深度强化学习型三大类型,介绍相应的智能决策方法在实时格斗游戏中的研究进展.为分析格斗游戏智能决策方法的表现性能,本文提出了胜率、剩余血量、执行速率、优势性和伤害性的5个性能因子,系统分析智能决策方法的性能优势及不足.最后,对未来的在格斗游戏中研究发展趋势进行展望.     

深度强化学习中状态注意力机制的研究

虽然在深度学习与强化学习结合后,人工智能在棋类游戏和视频游戏等领域取得了超越人类水平的重大成就,但是实时策略性游戏星际争霸由于其巨大的状态空间和动作空间,对于人工智能研究者来说是一个巨大的挑战平台,针对Deepmind在星际争霸II迷你游戏中利用经典的深度强化学习算法A3C训练出来的基线智能体的水平和普通业余玩家的水平相比还存在较大的差距的问题。通过采用更简化的网络结构以及把注意力机制与强化学习中的奖励结合起来的方法,提出基于状态注意力的A3C算法,所训练出来的智能体在个别星际迷你游戏中利用更少的特征图层取得的成绩最高,高于Deepmind的基线智能体71分。     

严肃游戏中虚拟角色行为建模综述

严肃游戏是计算机游戏一个新的发展方向,可以提供形象互动的模拟教学环境,已经广泛应用于科学教育、康复医疗、应急管理、军事训练等领域。虚拟角色是严肃游戏中模拟具有生命特征的图形实体,行为可信的虚拟角色能够提升用户使用严肃游戏的体验感。严肃游戏中的图形渲染技术已经逐步成熟,而虚拟角色行为建模的研究尚在初级阶段。可信的虚拟角色必须能够具有感知、情绪和行为能力。本文分别从游戏剧情与行为、行为建模方法、行为学习和行为建模评价等4个方面来分析虚拟角色行为建模研究。分析了有限状态机和行为树的特点,讨论了虚拟角色的行为学习方法。指出了强化学习的关键要素,探讨了深度强化学习的应用途径。综合已有研究,归纳了虚拟角色行为框架,该框架主要包括感觉输入、知觉分析、行为决策和动作4大模块。从情感计算的融入、游戏剧情和场景设计、智能手机平台和多通道交互4个角度讨论需要进一步研究的问题。虚拟角色的行为建模需要综合地考虑游戏剧情、机器学习和人机交互技术,构建具有自主感知、情绪、行为、学习能力、多通道交互的虚拟角色能够极大地提升严肃游戏的感染力,更好地体现寓教于乐。     

Developing serious games for cultural heritage: a state-of-the-art review

Although the widespread use of gaming for leisure purposes has been well documented, the use of games to support cultural heritage purposes, such as historical teaching and learning, or for enhancing museum visits, has been less well considered. The state-of-the-art in serious game technology is identical to that of the state-of-the-art in entertainment games technology. As a result, the field of serious heritage games concerns itself with recent advances in computer games, real-time computer graphics, virtual and augmented reality and artificial intelligence. On the other hand, the main strengths of serious gaming applications may be generalised as being in the areas of communication, visual expression of information, collaboration mechanisms, interactivity and entertainment. In this report, we will focus on the state-of-the-art with respect to the theories, methods and technologies used in serious heritage games. We provide an overview of existing literature of relevance to the domain, discuss the strengths and weaknesses of the described methods and point out unsolved problems and challenges. In addition, several case studies illustrating the application of methods and technologies used in cultural heritage are presented.     

Serious games and learning effectiveness: The case of It’s a Deal!

Although the value of serious games in education is undeniable and the potential benefits of using video games as ideal companions to classroom instruction is unquestionable, there is still little consensus on the game features supporting learning effectiveness, the process by which games engage learners, and the types of learning outcomes that can be achieved through game play. Our aim in this discussion is precisely to advance in this direction by providing evidence of some of the factors influencing the learning effectiveness of a serious game called It’s a Deal! This serious game was created for the purpose of teaching intercultural business communication between Spaniards and Britons in business settings in which English is used as the lingua franca. This paper hypothesizes that the immersive, all-embracing and interactive learning environment provided by the video game to its users may contribute to develop and enhance their intercultural communicative competence. The study attempts to answer three main research questions: (a) after playing It’s a Deal!, did the students sampled improve their intercultural awareness, intercultural knowledge and intercultural communicative competence in business English? (b) If they improved their intercultural learning, what are the factors influencing such improvement? And (c) if they did not improve their intercultural learning, what are the factors influencing such failure? The game participants who volunteered to take part in the study were all students of English Studies at the University of Alicante in the academic year 2010-2011. One hundred and six students completed both the pre-test and the post-test questionnaires, and played It’s a Deal! A sample of fifty students was selected randomly for the empirical study. The results obtained in the tests performed were compared and contrasted intra-group, both qualitatively and quantitatively, for the purpose of finding any statistically significant difference that may confirm whether or not there was an improvement in the students’ intercultural communicative competence in business English as a result of the implementation of the It’s a Deal! serious game. Findings of this study demonstrate that the video game is an effective learning tool for the teaching of intercultural communication between Spaniards and Britons in business settings in which English is used as the lingua franca. In particular, whereas the game had a small learning effect on intercultural awareness and a medium learning effect on intercultural knowledge, it had a large learning effect on intercultural communicative competence. The study also documents correlating factors that make serious games effective, since it shows that the learning effectiveness of It’s a Deal! stems from the correct balance of the different dimensions involved in the creation of serious games, specifically instructional content, game dimensions, game cycle, debriefing, perceived educational value, transfer of learnt skills and intrinsic motivation.     

深度学习在游戏中的应用

综述了近年来发展迅速的深度学习技术及其在游戏(或博弈)中的应用. 深度学习通过多层神经网络来构建端对端的从输入到输出的非线性映射, 相比传统的机器学习模型有显见的优势. 最近, 深度学习被成功地用于解决强化学习中的策略评估和策略优化的问题, 并于多种游戏的人工智能取得了突破性的提高. 本文详述了深度学习在常见游戏中的应用.     

Implementing digital game-based learning in schools: augmented learning environment of ‘Europe 2045’

It is widely agreed that the traditional process of schooling can benefit from the usage of computers as supportive tools. Of various approaches using computers in education over the last decade, e-learning and edutainment have become the most prominent. Recently, a number of authors have criticised these approaches arguing that they conserve traditional ‘drill and practice’ behaviouristic methods of teaching instead of enhancing and augmenting them. It has been proposed that a ‘paradigm shift’ is needed and that this shift may come through utilizing all the advantages of full-fledged video games, so-called digital game-based learning (DGBL). However, several case studies reported serious problems with the DGBL. Among the most notable issues are the lack of acceptance of games as an educational tool, problems with integration of games into formal schooling environments, and the so-called transfer problem, which is the problem of the inherent tension between game play and learning objectives, the tension that mitigates the ability of students to transfer knowledge gained in the video game to the real-world context. Here, we present a framework for an augmented learning environment (ALE), which verbalises one way of how these problems can be challenged. The ALE framework has been constructed based on our experience with the educational game, Europe 2045, which we developed and which has been implemented in a number of secondary schools in the Czech Republic during 2008. The key feature of this game is that it combines principles of on-line multi-player computer games with social, role-playing games. The evaluation which we present in this paper indicates the successful integration of the game and its acceptance by teachers and students. The ALE framework isolates key principles of the game contributing to this success, abstracts them into theoretical entities we call action-based spaces and causal and grounding links, and condenses them in a coherent methodological structure, which paves the way for further exploitation of the DGBL by educational game researchers and designers.     

Interactive story authoring: A viable form of creative expression for the classroom

The unprecedented growth in numbers of children playing computer games has stimulated discussion and research regarding what, if any, educational value these games have for teaching and learning. The research on this topic has primarily focused on children as players of computer games rather than builders/constructors of computer games. Recently, several game companies, such as BioWare Corp. and Bethesda Softworks, have released game story creation tools to the public, along with their games. However, a major obstacle to using these commercial tools is the level of programming experience required to create interactive game stories. In this paper, we demonstrate that a commercial game story construction tool, BioWare Corp.’s Aurora Toolset, can be augmented by our new tool, ScriptEase, to enable students in two grade ten English classes to successfully construct interactive game stories. We present evidence that describes the relationship between interactive story authoring and traditional story authoring, along with a series of factors that can potentially affect success at these activities: gender, creativity, intellectual ability, previous experiences with programming, time playing computer games, and time spent online. Results indicate that students can successfully construct sophisticated interactive stories with very little training. The results also show no gender differences in the quality of these interactive stories, regardless of programming experience or the amount of time per week playing computer games or participating in general online activities, although a subset of female students did show a slightly higher level of performance on interactive story authoring. In the educational context of this study, we show that ScriptEase provides an easy-to-use tool for interactive story authoring in a constructionist learning environment.     

Modeling outcomes of violent video game play: Applying mental models and model matching to explain the relationship between user differences,game characteristics,enjoyment, and aggressive intentions

While much research on video games has focused on aggressive outcomes, the extant research has not come to consensus on the role of other factors, notably enjoyment, frustration and individual differences amongst players, in mediating the link between violent game play and aggression. This experimental design uses a mental models approach to examine game features (e.g., controller naturalness), player characteristics (e.g., first person shooter experience, sex, degree of competitiveness) and the game play experience (e.g., perceived game realism, immersion, game failure, frustration and enjoyment) to explain aggressive outcomes. Results from a revised path model suggest perceived realism, immersion, and experience with other first person shooters all positively predicted enjoyment while frustration negatively predicted game enjoyment. Frustration and enjoyment both predicted state aggression. The findings of this study support the application of mental models as a theoretical approach to unify thinking about violent games, contextual features of violent games, individual difference variables and differences in player perceptions and game experiences with regards to outcome aggression. Results are discussed in terms of the model matching hypothesis and implications for the comprehensive study of violent game play, including the importance of enjoyment and frustration, are also discussed.     

面向人工智能专业课程的棋类博弈教学辅助平台

针对人工智能类课程博弈算法理论性太强、算法较复杂和抽象等问题,设计并实现功能较为完整、用户界面友好的棋类博弈教学辅助平台,平台允许用户上传编写好的博弈程序,实现博弈程序间对弈、人与博弈程序对弈以及人人对弈。本文阐述了棋类博弈教学辅助平台的总体设计、主要功能模块以及搭建平台采用的核心技术。棋类博弈教学平台的设计有益于培养学生依据所学理论知识来解决实际问题的能力,促进学生对计算机博弈程序和人工智能方法的不断探索改进,以达到更好的教学效果。本文进一步评估了棋类博弈平台在人工智能课程的实际使用情况与实践效果。     

基于神经网络求解游戏关卡问题

随着网络游戏的流行,玩家越来越关注游戏的智能化。本文基于人工神经网络中的BP网络算法建立了一个自动过关模型,此模型根据游戏中的不同环境输入神经元,经过隐含神经的不断学习,输出正确的路径,实验表明此模型能够有效地解决游戏关卡问题,并和遗传算法,情境法进行了对比。     

Applying simulation experience design methods to creating serious game-based adaptive training systems

The purpose of the present paper is to briefly introduce adaptive training systems, and describe the Simulation Experience Design Method. Adaptive training systems are serious games whose goal it is to engender communication opportunities for players to learn about their strengths and weaknesses, receive real-time in-game performance feedback, and share diverse solutions and strategies during, between, and after game play in order to update, or adapt, player understanding. The Simulation Experience Design Method extends HCI approaches to create engaging multiplayer learning experiences by focusing on how dynamic game content, roles, scenarios, and assessment feedback contribute to emergent culture. Preliminary evaluations of an earlier instantiation of serious game-based adaptive training systems employing this design method have been positive. The contribution of the present paper lies in describing how designers create rich systems of experiences for serious games and adaptive training systems by employing HCI principles and the Simulation Experience Design Method.     

What makes good games go viral? The role of technology use,efficacy, emotion and enjoyment in players’ decision to share a prosocial digital game

Serious digital games may be an effective tool for prosocial message dissemination because they offer technology and experiences that encourage players to share them with others, and spread virally. But little is known about the factors that predict players’ willingness to share games with others in their social network. This panel study explores how several factors, including sharing technology use, emotional responses, and game enjoyment, contribute to players’ decision to share the game Darfur is Dying, with others. College students played the game and completed questionnaires that assessed whether they had shared the games at two different time points: during game play and after game play. Positive emotions predicted sharing while students played the game, but negative emotions predicted whether the game was shared after initial game play. Game enjoyment predicted players’ intentions to share the game, but it did not predict actual sharing behavior. Neither players’ general use of sharing technologies nor their satisfaction related to sharing digital content predicted sharing intentions or behavior. These findings have implications for the study of viral social marketing campaigns, and serious game design and theory.     

Comparing the retention and flow experience in playing Solitary and Heart Attack games of San Zi Jing: A perspective of Dual Process Theory

Most educational games require high speed process; they require fast recognitions and accurate hand-eye coordination to play. Under this constraint, Dual Process theory (DPT) indicated players have different levels of cognitive process over different games which affect their mood. Csikszentmihalyi highlighted that a flow experience is the essential psychological effect on mood and cognitive process. According to DPT, how students engage in different game design will affect their learning retention effects. The flow experiences were examined in this study in relation to Solitary and Heart Attack games of San Zi Jing (a Chinese educational poem). The present study was implemented with a sample of 209 5th and 6th grade elementary school students. The students were assigned with one of the games randomly. The results indicated that in terms of flow experiences, Heart Attack group was significantly higher than that of the Solitary group; secondly, solitary game showed a better retention rate than Heart Attack game did after one month; even though the retention rate of both type games have slight loose after one month. The results of this study suggest if the game design emphasizes on learning retention, solitary game is recommended for higher grade students. Whereas, game design for hedonic values, then Heart Attack type is recommended. Future studies may focus on additional contents and establish more trials to examine the effects of flow experiences and engagement effects.     

Children’s choices and strategies in video games

It is important to develop an understanding of children’s engagement and choices in learning experiences outside of school as this has implications for their development and orientations to other learning environments. This mixed-methods study examines relationships between the genres of video games children choose to play and the learning strategies they employ to improve at these games. It also explores students’ motivations for playing the games they choose to play. One hundred eighteen fourth- and fifth-grade students participated in this study. Qualitative analyses of student responses resulted in a model for classifying motivation for game choices. Children primarily cite reasons that can be classified as psychological or cognitive reasons for choosing to play certain video games, and are motivated by the challenge and thinking required in the games. Analyses using Chi-square tests of association demonstrated significant relationships between video game genre and learning strategy used for two of the six learning strategies (p < .05). Children playing action games are more likely to use repetition to learn the game and children playing adventure games are more likely to use their imaginations to take on the role of the character in the game and think the way the character would to make decisions in the game. There were also several gender differences in learning preferences.     

Teaching computer science through game design

Playing computer games is a popular recreational activity for young people. Not surprisingly, many of these enthusiasts dream that one day they will develop computer games themselves. Developing computer games involves many aspects of computing, including computer graphics, artificial intelligence, human-computer interaction, security, distributed programming, simulation, and software engineering. Users who can't code a single line can use game maker to build games and learn object-oriented design.     

通用对弈游戏:一个探索机器游戏智能的领域

通用对弈游戏（general game playing,简称GGP）是致力于提高机器的通用游戏智能的研究领域.与专用游戏智能程序不同,GGP玩家直到游戏开始时才获得游戏规则,从而避免依赖于人类关于特定游戏的经验.GGP研究发展至今,已在游戏表示、搜索算法、状态估值等方面做了深入探索,并在知识迁移等方面做出了尝试.GGP研究的进展在一定程度上代表了通用人工智能的发展,因而是值得关注的.     

计算机扑克智能博弈研究综述

计算机博弈是人工智能领域的“果蝇”,备受人工智能领域研究者的关注,已然成为研究认知智能的有利平台。扑克类博弈对抗问题可建模成边界确定、规则固定的不完美信息动态博弈,计算机扑克 AI 需要具备不完全信息动态决策、对手误导欺诈行为识别以及多回合筹码和风险管理等能力。首先梳理了以德州扑克为代表的计算机扑克智能博弈的发展历程,其次针对计算机扑克智能博弈典型模型算法、关键技术以及存在的主要问题进行了综述分析,最后探讨了计算机扑克智能博弈的未来发展趋势和应用前景。