The interplay of game elements with psychometric qualities,learning, and enjoyment in game-based assessment

Educators today are increasingly interested in using game-based assessment to assess and support students' learning. In the present study, we investigated how changing a game design element, linearity in gameplay sequences, influenced the effectiveness of game-based assessment in terms of validity, reliability, fairness, learning, and enjoyment. Two versions of a computer game, Physics Playground (formerly Newton's Playground), with different degrees of linearity in gameplay sequences were compared. Investigation of the assessment qualities—validity, reliability, and fairness—suggested that changing one game element (e.g., linearity) could significantly influence how players interacted with the game, thus changing the evidentiary structure of in-game measures. Although there was no significant group difference in terms of learning between the two conditions, participants who played the nonlinear version of the game showed significant improvement on qualitative physics understanding measured by the pre- and posttests while the participants in the linear condition did not. There was also no significant group difference in terms of enjoyment. Implications of the findings for future researchers and game-based assessment designers are discussed.     

An application of adaptive games-based learning based on learning style to teach SQL

The fact that each student has a different way of learning and processing information has long been recognised by educationalists. In the classroom, the benefits derived from delivering learning content in ways that match the student's learning style have also been identified. As new modes of delivery of learning content such as computer-assisted learning systems (e.g. eLearning) have become increasingly popular, research into these has also identified the benefits of tailoring learning content to learning styles. However, in games-based learning (GBL), the adaptation based on learning style to enhance the educational experience has not been well researched. For the purpose of this research, a game with three game modes has been developed: 1) non-adaptivity mode; 2) a mode that customises the game according to the student's learning style identified by using a learning style questionnaire; and 3) a mode that has an in-game adaptive system that dynamically and continuously adapts its content according to the student's interactions in the game.This paper discusses the term adaptivity in a GBL context and presents the results of an experimental study investigating the differences in learning effectiveness of the different game modes compared to a paper-based learning. The study was performed with 120 Higher Education students learning the database language SQL (Structured Query Language). The results show that the game developed, regardless of mode, produced better learning outcomes than those who learned from a textbook while adaptive GBL was better in terms of allowing learners to complete the tasks faster than the other two game versions.     

Modeling how incoming knowledge,persistence, affective states,and in-game progress influence student learning from an educational game

This study investigated the relationships among incoming knowledge, persistence, affective states, in-game progress, and consequently learning outcomes for students using the game Physics Playground. We used structural equation modeling to examine these relations. We tested three models, obtaining a model with good fit to the data. We found evidence that both the pretest and the in-game measure of student performance significantly predicted learning outcome, while the in-game measure of performance was predicted by pretest data, frustration, and engaged concentration. Moreover, we found evidence for two indirect paths from engaged concentration and frustration to learning, via the in-game progress measure. We discuss the importance of these findings, and consider viable next steps concerning the design of effective learning supports within game environments.     

The format of problem representation for in‐game learning supports

A promising method to support game‐based learning is to facilitate learners' externalization of cognitive and metacognitive processes. Externalizing Problem Representation (EPR) refers to a cognitive behaviour in which a learner constructs her own representations overtly. The purpose of this study is to investigate whether learning supports promoting EPR enhance qualitative understanding and quantitative proficiency in ratios and proportional relationships in a learning game (i.e., E‐Rebuild) context. Specifically, this study investigated the effects of representation format in problem representation on qualitative understanding and quantitative proficiency in a learning game context. The results of this study indicate that (a) symbolic learning supports better facilitate comprehension of math concepts and their relations than iconic learning supports in video game contexts, (b) symbolic learning supports better facilitate players' reflection for implicit understanding and promote their math problem‐solving skills, (c) participants in the symbolic learning support group increased significantly in qualitative understanding but not in quantitative proficiency after gameplay, and (d) participants in the iconic learning support group experienced significant growth in quantitative proficiency but not in qualitative understanding after gameplay.     

Learning through playing Virtual Age: Exploring the interactions among student concept learning,gaming performance,in-game behaviors,and the use of in-game characters

Video games possess many unique features that facilitate learning. Meanwhile, teaching about evolution is never an easy task due to the existence of some barriers to its learning. Virtual Age, therefore, has been developed in an attempt to harness the power of gaming to increase student understanding of biological evolution. The aim of this study was to examine whether Virtual Age is effective for learning about evolution and to further explore the interplay of student concept learning, gaming performance, and in-game behaviors. A total of 62 7th graders took part in the study, and significant findings were revealed. The students did learn by playing Virtual Age, and their long-term knowledge retention was promising. The in-game behaviors, such as times and duration of viewing the relevant information embedded in Virtual Age, were significantly related to gaming performance (game score), which subsequently influenced learning outcomes. Moreover, the results of cluster analysis indicated that three clusters of low learning outcomes/low gaming performance, high learning outcomes, and high gaming performance emerged. Overall, Virtual Age is an effective game for learning about evolution based on its sound and sophisticated design. Implications derived from the study and suggestions for future work are proposed.     

Implicit learning as a design strategy for learning games: Alert Hockey

Concussion education and prevention for youth hockey players has been an issue of recent concern amongst sport medicine practitioners and hockey’s administrative bodies. This article details the assessment of a sports-action hockey video game that aims to reduce the aggressive and negligent behaviours that can lead to concussions. The game, termed Alert Hockey, was designed to modify game playing behaviour by embedding an implicit teaching mechanism within the gameplay. In Alert Hockey, participants were expected to learn by simply playing to win, in contrast to playing to learn. We studied learning in an experimental simulated environment where the possibility to win the game was exaggerated as a consequence of desirable safety behaviours (positive learning group) and effectively reduced as a consequence of undesirable (negative learning group) behaviour. The positive learning group significantly improved their mean score on a composite behavioural indicator compared with no significant change amongst control group participants. The results demonstrate that implicit learning embedded in a sports-action game can lead to changes in game-play behaviour.     

Predicting students' knowledge after playing a serious game based on learning analytics data: A case study

Serious games have proven to be a powerful tool in education to engage, motivate, and help students learn. However, the change in student knowledge after playing games is usually measured with traditional (paper) prequestionnaires–postquestionnaires. We propose a combination of game learning analytics and data mining techniques to predict knowledge change based on in-game student interactions. We have tested this approach in a case study for which we have conducted preexperiments–postexperiments with 227 students playing a previously validated serious game on first aid techniques. We collected student interaction data while students played, using a game learning analytics infrastructure and the standard data format Experience API for Serious Games. After data collection, we developed and tested prediction models to determine whether knowledge, given as posttest results, can be accurately predicted. Additionally, we compared models both with and without pretest information to determine the importance of previous knowledge when predicting postgame knowledge. The high accuracy of the obtained prediction models suggests that serious games can be used not only to teach but also to measure knowledge acquisition after playing. This will simplify serious games application for educational settings and especially in the classroom easing teachers' evaluation tasks.     

Assessing implicit science learning in digital games

Building on the promise shown in game-based learning research, this paper explores methods for Game-Based Learning Assessments (GBLA) using a variety of educational data mining techniques (EDM). GBLA research examines patterns of behaviors evident in game data logs for the measurement of implicit learning—the development of unarticulated knowledge that is not yet expressible on a test or formal assessment. This paper reports on the study of two digital games showing how the combination of human coding with EDM has enabled researchers to measure implicit learning of Physics. In the game Impulse, researchers combined human coding of video with educational data mining to create a set of automated detectors of students' implicit understanding of Newtonian mechanics. For Quantum Spectre, an optics puzzle game, human coding of Interaction Networks was used to identify common student errors. Findings show that several of our measures of student implicit learning within these games were significantly correlated with improvements in external postassessments. Methods and detailed findings were different for each type of game. These results suggest GBLA shows promise for future work such as adaptive games and in-class, data-driven formative assessments, but design of the assessment mechanics must be carefully crafted for each game.     

Responsiveness to a game-based intervention to enhance reading efficiency in first graders

In the present study, a 5-week tablet-based word reading efficiency game intervention (Reading Turbo) was integrated in a comprehensive phonics-based reading curriculum. The aims of the study were to examine whether the game would advance children's word reading efficiency, and to determine the extent to which pre-reading capacities and in-game mechanisms could explain individual variation in responsiveness to the game. To do so, word reading efficiency scores of first graders in an intervention group (n = 132) and a control group that continued with the usual reading curriculum (n = 118) were compared prior to, directly after, and 2 months after the 5-week intervention period. Individual variation in responsiveness was examined by relating reading precursors (i.e., phonological awareness, letter efficiency, rapid automatized naming (RAN), and verbal working memory) measured at the onset of the intervention, and in-game accuracy and efficiency to word reading efficiency outcomes in untrained words. Results indicated that playing the word reading game significantly enhanced first graders' word reading efficiency directly after the intervention, but the effect was no longer significant 2 months after the intervention. With respect to individual variation in game responsiveness, it was found that phonological awareness, letter efficiency, and verbal working memory predicted responsiveness to the game via in-game accuracy. In addition, phonological awareness and RAN predicted responsiveness to the game via in-game efficiency, and letter efficiency and verbal working memory were directly related to responsiveness. The effectiveness of the word reading game thus differentiated between children with higher and lower pre-reading capacities, and operated via in-game child affordances.     

To solve or to observe? The case of problem-solving interactivity within child learning games

We explored whether problem-solving interactivity within an instructional game fosters learning for children aged 8–10 years. Participants (N = 139) studied a biological topic either through a game-based learning environment (in which they solved assigned problems by interacting with a plant model) or from a standard learning environment (in which they observed how the problems were solved in an animation) (i.e., between-subject design). The treatments were equal with regard to learning content and guidance. No between-group differences in learning outcomes were detected (comprehension: d = 0.16; transfer: d = −0.01). Self-rating of enjoyment tended to be higher in the game group (d = 0.32), and when the children could choose between the treatments in a free-choice period, they strongly preferred the interactive game (87.5%). The results suggest that both the interactive and the non-interactive treatments are useful, but their applicability may differ depending on the learning context (e.g., school vs. home).     

The evaluation of different gaming modes and feedback types on game-based formative assessment in an online learning environment

This study proposed an online learning system for energy education, modifying the typical rules of tic-tac-toe and incorporating multiple choice tests into the game in order to develop a game-based formative assessment tool for an online learning course. In order to explore how different gaming modes and feedback types in this game-based formative assessment affect knowledge acquisition effectiveness and participation perceptions, a tic-tac-toe quiz game (TRIS-Q) with two gaming modes: single-player online game (SOG) and multi-player online game (MOG), and two feedback types: immediate elaborated feedback (IEF) and no immediate elaborated feedback (no IEF), were developed. A 2(SOG vs. MOG) × 2(IEF vs. no IEF) between-subject experiment was also conducted to investigate the effects on 109 ninth-grade students from four junior high school classes. The research findings indicated that different gaming modes of TRIS-Q did not affect the effectiveness of knowledge acquisition; providing IEF for each question answered in the game facilitated the enhancement of both energy knowledge acquisition and student tic-tac-toe ability when comparing it with the no IEF type. Additionally, the different gaming modes and feedback types did not affect participation perceptions.     

Triple scheme of learning support design for scientific discovery learning based on computer simulation: experimental research

Abstract Learning support studies involving simulation‐based scientific discovery learning have tended to adopt an ad hoc strategies‐oriented approach in which the support strategies are typically pre‐specified according to learners' difficulties in particular activities. This article proposes a more integrated approach, a triple scheme for learning support design on the basis of the systematic analysis of the internal conditions of scientific discovery learning. The triple learning support scheme involves: (a) interpretative support that helps learners with knowledge access and the generation of meaningful and integrative understandings; (b) experimental support that scaffolds learners in systematic and valid experimental activities; and (c) reflective support that increases learners' self‐awareness of the discovery processes and prompts their reflective abstraction and integration. Two experiments were conducted with eighth graders (13‐year‐olds) to examine the effects of these learning supports embedded into a simulation program on floating and sinking. The overall results support the main hypotheses that learning supports in a simulation environment should be directed towards the three perspectives to invite meaningful, systematic, and reflective discovery learning.     

Scaffolding game-based learning: Impact on learning achievements,perceived learning,and game experiences

One of the central challenges of integrating game-based learning in school settings is helping learners make the connections between the knowledge learned in the game and the knowledge learned at school, while maintaining a high level of engagement with game narrative and gameplay. The current study evaluated the effect of supplementing a business simulation game with an external conceptual scaffold, which introduces formal knowledge representations, on learners' ability to solve financial-mathematical word problems following the game, and on learners' perceptions regarding learning, flow, and enjoyment in the game. Participants (M_age = 10.10 years) were randomly assigned to three experimental conditions: a “study and play” condition that presented the scaffold first and then the game, a “play and study” condition, and a “play only” condition. Although no significant gains in problem-solving were found following the intervention, learners who studied with the external scaffold before the game performed significantly better in the post-game problem-solving assessment. Adding the external scaffold before the game reduced learners' perceived learning. However, the scaffold did not have a negative impact on reported flow and enjoyment. Flow was found to significantly predict perceived learning and enjoyment. Yet, perceived learning and enjoyment did not predict problem-solving and flow directly predicted problem solving only in the “play and study” condition. We suggest that presenting the scaffold may have “problematized” learners' understandings of the game by connecting them to disciplinary knowledge. Implications for the design of scaffolds for game-based learning are discussed.     

The role of executive control in young children's serious gaming behavior

The present study examined (1) how executive control contributed to in-game behaviors in young children while playing a serious game, (2) whether the levels of control changed when the game was played repeatedly, and (3) how the first experience with the game mediated the role of executive control to in-game behaviors when the game was repeated. Attentional and action control were directly assessed in 106 kindergartners, who played a single-leveled serious game twice. During their gameplay, the following behaviors were registered: time, number of scaffolds needed, mistakes, verbal expressions, questions, irrelevant game activities (drawings), and off-task behavior. The results for the first game round showed that time, expressions, and the need for scaffolds were predicted by attentional control. In the second round, a strong role for action control was found to overcome off-task behavior and irrelevant drawings. Verbal expressiveness was again influenced by attentional control. Moreover, mediation effects of attentional control to efficient in-game behaviors in the second gameplay were evidenced via scaffolding and expressiveness in the first gameplay. It is concluded that in new games children's attentional control contributes to formulating strategies and problem-solving, while their action control underlies sustained and goal-directed learning over time.     

Learning,goals, and learning goals: A perspective on goal-driven learning

In cognitive science, artificial intelligence, psychology, and education, a growing body of research supports the view that the learning process is strongly influenced by the learner's goals. The fundamental tenet ofgoal-driven learning is that learning is largely an active and strategic process in which the learner, human or machine, attempts to identify and satisfy its information needs in the context of its tasks and goals, its prior knowledge, its capabilities, and environmental opportunities for learning. This article examines the motivations for adopting a goal-driven model of learning, the relationship between task goals and learning goals, the influences goals can have on learning, and the pragmatic implications of the goal-driven learning model. It presents a new integrative framework for understanding the goal-driven learning process and applies this framework to characterizing research on goal-driven learning.     

Learning style analysis in adaptive GBL application to teach SQL

The benefits derived from delivering learning content in ways that match the student's learning style have been identified in classroom learning and eLearning. Although there is limited empirical evidence in adaptive Games-Based Learning (GBL), adaptivity has been identified to have the potential to improve learning effectiveness. This paper presents the results of a study to investigate the use of learning styles in GBL particularly in identifying the learning style in GBL including the learning style's fluctuation during the learning process. For the purposes of this study, a game with two game modes was developed: 1) non-adaptivity mode and 2) a mode that had an in-game adaptive system that dynamically and continuously adapted its contents according to the student's interactions in the game. In both modes, the interactions between the participants and the game were recorded in a database. The study was performed with 60 students in Higher Education. The results show that the learning style identified by using a learning style questionnaire is not always consistent with the learning style identified in the game. The results also show the learning style fluctuates during the learning process in GBL although there is a tendency for participants to choose the same learning style as the learning style identified outside the game in the first mission of the game. The number of mistakes committed by participants has been identified to have a strong correlation to the fluctuation. The results contribute to the body of empirical evidence in adaptive GBL particularly in identifying the learning style fluctuation in GBL and the paper provides recommendations on the use of adaptivity in GBL to accommodate this fluctuation.     

Reinforcement learning via approximation of the Q-function

Relational reinforcement learning (RRL) combines traditional reinforcement learning (RL) with a strong emphasis on a relational (rather than attribute-value) representation. Earlier work used RRL on a learning version of the classic Blocks World planning problem (a version where the learner does not know what the result of taking an action will be) and the Tetris game. Learning results based on the structure of training examples were obtained, such as learning in a mixed 3–5 block environment and being able to perform in a 3 or 10 block environment. Here, we instead take a function approximation approach to RL for the Blocks World problem. We obtain similar learning accuracies, with better running times, allowing us to consider much larger problem sizes. For instance, we can train on 15 blocks and then perform well on worlds with 100–800 blocks–using less running time than the relational method required to perform well for 3–10 blocks.     

Re-thinking physics teaching with web-based learning

There is extensive uptake of ICT in the teaching of science but more evidence is needed on how ICT impacts on the learning practice and the learning outcomes at the classroom level. In this study, a physics website (Getsmart) was developed using the cognitive apprenticeship framework for students at a high school in Australia. This website was designed to enhance students’ knowledge of concepts in physics. Reflexive pedagogies were used in the delivery learning materials in a blended learning environment. The students in the treatment group accessed the website over a 10 week period. Pre and post-test results of the treatment (N = 48) and comparison group (N = 32) were compared. The MANCOVA analysis showed that the web-based learning experience benefitted the students in the treatment group. It not only impacted on the learning outcomes, but qualitative data from the students suggested that it had a positive impact on their attitudes towards studying physics in a blended environment.     

Challenging games help students learn: An empirical study on engagement,flow and immersion in game-based learning

In this paper, we investigate the impact of flow (operationalized as heightened challenge and skill), engagement, and immersion on learning in game-based learning environments. The data was gathered through a survey from players (N = 173) of two learning games (Quantum Spectre: N = 134 and Spumone: N = 40). The results show that engagement in the game has a clear positive effect on learning, however, we did not find a significant effect between immersion in the game and learning. Challenge of the game had a positive effect on learning both directly and via the increased engagement. Being skilled in the game did not affect learning directly but by increasing engagement in the game. Both the challenge of the game and being skilled in the game had a positive effect on both being engaged and immersed in the game. The challenge in the game was an especially strong predictor of learning outcomes. For the design of educational games, the results suggest that the challenge of the game should be able to keep up with the learners growing abilities and learning in order to endorse continued learning in game-based learning environments.     

Game immersion experience: its hierarchical structure and impact on game‐based science learning

Many studies have shown the positive impact of serious educational games (SEGs) on learning outcomes. However, there still exists insufficient research that delves into the impact of immersive experience in the process of gaming on SEG‐based science learning. The dual purpose of this study was to further explore this impact. One purpose was to develop and validate an innovative measurement, the Game Immersion Questionnaire (GIQ), and to further verify the hierarchical structure of game immersion by construct validity approaches, including exploratory factor analysis (EFA) (n = 257) and confirmatory factor analysis (CFA) (n = 1044). The second purpose was to investigate the impact of game immersion on science learning through SEG play (n = 260). Overall, the results supported the internal structure of the GIQ with good reliability and validity, and the inter factor bivariate correlations for each construct indicated a high internal consistency. Players did learn from playing an SEG, and game immersion experience did lead to higher gaming performance. Moreover, players' gaming performance plays a role in mediating the effect of immersion on science learning outcomes through SEG play. However, as players became more emotionally and subjectively attached to the game, the science learning outcomes were not definitively reliable.