The interplay of game elements with psychometric qualities,learning, and enjoyment in game-based assessment

Educators today are increasingly interested in using game-based assessment to assess and support students' learning. In the present study, we investigated how changing a game design element, linearity in gameplay sequences, influenced the effectiveness of game-based assessment in terms of validity, reliability, fairness, learning, and enjoyment. Two versions of a computer game, Physics Playground (formerly Newton's Playground), with different degrees of linearity in gameplay sequences were compared. Investigation of the assessment qualities—validity, reliability, and fairness—suggested that changing one game element (e.g., linearity) could significantly influence how players interacted with the game, thus changing the evidentiary structure of in-game measures. Although there was no significant group difference in terms of learning between the two conditions, participants who played the nonlinear version of the game showed significant improvement on qualitative physics understanding measured by the pre- and posttests while the participants in the linear condition did not. There was also no significant group difference in terms of enjoyment. Implications of the findings for future researchers and game-based assessment designers are discussed.     

Maximizing learning without sacrificing the fun: Stealth assessment,adaptivity and learning supports in educational games

In this study, we investigated the validity of a stealth assessment of physics understanding in an educational game, as well as the effectiveness of different game-level delivery methods and various in-game supports on learning. Using a game called Physics Playground, we randomly assigned 263 ninth- to eleventh-grade students into four groups: adaptive, linear, free choice and no-treatment control. Each condition had access to the same in-game learning supports during gameplay. Results showed that: (a) the stealth assessment estimates of physics understanding were valid—significantly correlating with the external physics test scores; (b) there was no significant effect of game-level delivery method on students' learning; and (c) physics animations were the most effective (among eight supports tested) in predicting both learning outcome and in-game performance (e.g. number of game levels solved). We included student enjoyment, gender and ethnicity in our analyses as moderators to further investigate the research questions.     

Supporting conceptual understanding of the associative and distributive properties through digital gameplay

There has been a steady rise in the support for games as learning environments. This support is largely based on the strong levels of engagement and motivation observed during gameplay. What has proven difficult is the ability to consistently design and develop learning games that are both engaging and educationally viable. Those in the game‐based learning community contend the solution is to tightly integrate instructional objective(s) within the game's mechanics. In order to investigate whether or not intrinsic integration would aid in game design, an experimental study was conducted with 111 fourth and fifth graders in which three versions of a game were designed in order to determine which approach would have a more significant impact on conceptual understanding of the associative and distributive properties. It was found that those who played the intrinsic version of the game (properties mapped to game mechanics) had significant learning gains compared with those who played the extrinsic version. Perceptions of the game are also discussed, along with future directions.     

Using commercial video games in flipped classrooms to support physical concept construction

Based on the flipped‐classroom model and the potential motivational and instructional benefits of digital games, we describe a flipped game‐based learning (FGBL) strategy focused on preclass and overall learning outcomes. A secondary goal is to determine the effects, if any, of the classroom aspects of the FGBL strategy on learning efficiency. Our experiments involved 2 commercial games featuring physical motion concepts: Ballance (Newton's law of motion) and Angry Birds (mechanical energy conservation). We randomly assigned 87 8th‐grade students to game instruction (digital game before class and lecture‐based instruction in class), FGBL strategy (digital game before class and cooperative learning in the form of group discussion and practice in class), or lecture‐based instruction groups (no gameplay). Results indicate that the digital games exerted a positive effect on preclass learning outcomes and that FGBL‐strategy students achieved better overall learning outcomes than their lecture‐based peers. Our observation of similar overall outcomes between the cooperative learning and lecture‐based groups suggests a need to provide additional teaching materials or technical support when introducing video games to cooperative classroom learning activities.     

Does individual or collaborative self-debriefing better enhance learning from games?

The primary aim of this study is to find out whether use of different self-debriefing modes affects learning from a game. In self-debriefing participants are led to reflect upon their game experiences by a set of debriefing questions. Two conditions were compared: Individual and Collaborative self-debriefing. The 45 participants first played the game of Lemonade Tycoon Deluxe, were tested for knowledge and self-debriefed in pairs or alone. Then they played the game once more and were tested again. Game scores increased significantly from the first to the second round of gameplay to an equal degree in both conditions. Knowledge scores of participants in individual self-debriefing increased significantly more than those of participants in the Collaborative condition. The study shows that game-based learning can be effectively scaffolded with self-debriefing. Future studies might investigate whether the type of self-debriefing differentially affects game motivation. In addition, attention to the role of feedback is called for.     

Performance,cognitive load,and behaviour of technology‐assisted English listening learning: From CALL to MALL

This study examines differences in English listening comprehension, cognitive load, and learning behaviour between outdoor ubiquitous learning and indoor computer‐assisted learning. An experimental design, employing a pretest‐posttest control group is employed. Randomly assigned foreign language university majors joined either the experimental group (outdoor ubiquitous learning), with 80 participants (26 males and 54 females), or a control group (indoor computer‐assisted learning), with 80 participants (27 males and 53 females). The experiment lasted 3 weeks. Both groups were administered a test of English listening proficiency before and after the experiment along with a questionnaire of cognitive load postexperiment. Prior English listening proficiency forms a covariate for the multivariate analysis of covariance. Results show (a) students in the experimental group exhibit significantly better English listening comprehension after the experiment compared to the control group; (b) students in the experimental group reported significantly lower cognitive load than the control group; (c) English listening comprehension and cognitive load exhibited a statistically significant negative relationship; and (d) outdoor ubiquitous learning enhanced self‐reported learning interests and interactions more than indoor computer‐assisted learning. Contributions and significances of this study are presented based on these results. Finally, implications for teaching practices are proposed.     

Persistence temperament associated with children playing math games between touch panel and embodied interaction

The different designs of various interactive gaming controllers affect posture and muscle loading of the body. It is assumed that prolonged exposure to the interactive gaming controllers can affect the effectiveness in using the game for the purposes of learning. This study explores the differences in behavioural responses among students with different temperaments regarding mathematic gameplay by comparing the touch‐based and gesture‐based interactive devices. The experiment was designed for 119 5‐year‐old kindergarten children, and valid data from 99 individuals were analysed for this study. The results indicated that participants in the touch‐based interaction (TBI) groups performed better than the participants in the gesture‐based interaction (GBI) groups with respect to numerical counting in both games. The results also showed that among all dimensions of temperaments, only persistence had a positive correlation to TBI. This implied that TBI was preferred over GBI for kindergarten children. Accordingly, e‐learning designers should place more emphasis on TBI.     

Integrating self-explanation functionality into a complex game environment: Keeping gaming in motion

Previous research has shown that either asking students to explain their answers or providing explanatory feedback can be effective ways to increase learning from an educational game. This study focused on an educational physics game about Newton's 3 Laws of Motion called SURGE: The Fuzzy Chronicles. Eighty-six middle school students played one of three versions of the game: (1) the base version with no tips or questions, (2) the self-explanation version with self-explanation questions prompts, and (3) the explanatory feedback version with gameplay tips. There were no significant overall learning differences between the three groups, but students in the base version successfully answered more questions about Newton's second law than students in the self-explanation group. This may have been due to students in the base condition progressing significantly further through the game than students in the self-explanation group. The results suggest that the cognitive load for gameplay as well as game flow must be managed in order for students to take advantage of explanation functionality in educational tools designed to increase deeper, germane processing.     

Implicit learning as a design strategy for learning games: Alert Hockey

Concussion education and prevention for youth hockey players has been an issue of recent concern amongst sport medicine practitioners and hockey’s administrative bodies. This article details the assessment of a sports-action hockey video game that aims to reduce the aggressive and negligent behaviours that can lead to concussions. The game, termed Alert Hockey, was designed to modify game playing behaviour by embedding an implicit teaching mechanism within the gameplay. In Alert Hockey, participants were expected to learn by simply playing to win, in contrast to playing to learn. We studied learning in an experimental simulated environment where the possibility to win the game was exaggerated as a consequence of desirable safety behaviours (positive learning group) and effectively reduced as a consequence of undesirable (negative learning group) behaviour. The positive learning group significantly improved their mean score on a composite behavioural indicator compared with no significant change amongst control group participants. The results demonstrate that implicit learning embedded in a sports-action game can lead to changes in game-play behaviour.     

A case study of computer gaming for math: Engaged learning from gameplay?

Employing mixed-method approach, this case study examined the in situ use of educational computer games in a summer math program to facilitate 4th and 5th graders’ cognitive math achievement, metacognitive awareness, and positive attitudes toward math learning. The results indicated that students developed more positive attitudes toward math learning through five-week computer math gaming, but there was no significant effect of computer gaming on students’ cognitive test performance or metacognitive awareness development. The in-field observation and students’ think-aloud protocol informed that not every computer math drill game would engage children in committed learning. The study findings have highlighted the value of situating learning activities within the game story, making games pleasantly challenging, scaffolding reflections, and designing suitable off-computer activities.     

How competition and heterogeneous collaboration interact in prevocational game-based mathematics education

The present study addresses the effectiveness of an educational mathematics game for improving proportional reasoning in students from prevocational education. Though in theory game-based learning is promising, research shows that results are ambiguous and that we should look into ways to support game-based learning. The current study explored two factors (i.e., collaboration and competition) that have been associated with motivational and cognitive effects, and have potential to optimize game-based learning. In a fully crossed design, four conditions were examined: collaboration and competition, collaboration control, competition control, and control. It was found that, over all, gameplay did improve students' proportional reasoning skills but that learning effects did not differ between conditions. However, when students’ ability levels were taken into account, an interaction between collaboration and competition was found. For below-average students, the effect of collaboration was modified by competition, showing a negative effect of competition on domain knowledge gains in a collaborative learning situation. In contrast, for above-average students, the data demonstrated a trend that suggests a positive effect of competition on domain knowledge gain in a collaborative learning situation.     

An implementation of design-based learning through creating educational computer games: A case study on mathematics learning during design and computing

This mixed-method case study examined the potential of computer-assisted, math game making activities in facilitating design-based math learning for school children. Sixty-four middle school children participated in Scratch-based, math game making activities. Data were collected via activity and conversation observation, artifact analysis, interviewing, and survey. The study findings indicated that participants developed significantly more positive dispositions toward mathematics after computer game making. The study also found that experience-driven game design processes helped to activate children's reflection on everyday mathematical experiences. Mathematical thinking and content experience were intertwined within the process of computer game authoring. On the other hand, children designers were involved in game-world and story crafting more than mathematical representation. And it was still challenging for them to perform computer game coding with abstract reasoning.     

The validity of a game-based assessment of persistence

In this study, 154 students individually played a challenging physics video game for roughly 4 h. Based on time data for both solved and unsolved problems derived from log files, we created a game-based assessment of persistence that was validated against an existing measure of persistence. We found that the game-based assessment of persistence predicted learning of qualitative physics after controlling for gender, video game experience, pretest knowledge and enjoyment of the game. These findings support the implementation of a real-time formative assessment of persistence to be used to dynamically change gameplay.     

Comparing animated and static modes in educational gameplay on user interest,performance and gameplay anxiety

To compare the perception of animated to static mode of gameplay and how it affects the emotional state and performance of playing, this study designed two games, Chinese idiom string up game (CISUG) and Chinese idiom fishing game (CIFG). Seven classes with a total of 238 students were invited to participate in CIFG, and six classes with a total of 211 students were recruited for CISUG. All participants were 5th and 6th grade elementary school students. Performance achievement, interest level and gameplay anxiety were statistically examined using paired sample t-tests. The results revealed that the static mode fostered better performance. In addition, interest was found to be better maintained in the static mode than in the animated mode, but there was no difference between the two types of game modes in respect of gameplay anxiety. However, for both modes, gameplay anxiety did decrease slightly upon subsequent playing, though it did not reach a level of statistical significance. The results suggest that in educational game design, the use of static rather than animated modes should be considered when the objective is to help students achieve better performance in a competitive setting.     

Is single or dual channel with different English proficiencies better for English listening comprehension,cognitive load and attitude in ubiquitous learning environment?

The purpose of the present study was to examine the effects of English proficiency (low vs. high) and material presentation mode (single channel vs. dual channel) on English listening comprehension, cognitive load and learning attitude in a ubiquitous learning environment. An experimental learning activity was implemented using PDA as a learning tool to facilitate learning. A quasi-experimental design was employed with university students participating in the experiment. The results revealed that (a) high English proficiency learners had significantly better English listening comprehension and lower intrinsic and extraneous load than low English proficiency learners; (b) both high and low English proficiency learners learning with dual channel had significantly better English listening comprehension and held more positive attitude toward the ubiquitous learning environment than learners learning with single channel; (c) for learners learning with single channel, low English proficiency learners had significantly higher extraneous load than high English proficiency learners; and (d) for low English proficiency learners, learners who learned with dual channel possessed significantly lower extraneous load than learners who learned with single channel. The suggestions for educators and instructional designers were also discussed in the present study.     

Multiphase method for analysing online discussions

Several studies have analysed and assessed online performance and discourse using quantitative and qualitative methods. Quantitative measures have typically included the analysis of participation rates and learning outcomes in terms of grades. Qualitative measures of postings, discussions and context features aim to give insights into the nature of learning activities taking place in the online learning environment. No matter what the particular method, one of the critical aspects in these analyses is the coherence between theory and methodology as well as the explicit focus on the unit(s) of analysis. Technology‐enhanced learning (TEL) settings typically involve processes (e.g. cognitive or social) at different levels, and these processes are often intertwined with each other in a way that is not reducible to any one level only. In this article, a short review on the challenges of research‐based evaluation of TEL, and computer‐supported collaborative learning in particular, is made. With these challenges in the background, the integration of different levels of analysis is demonstrated with the aid of the method developed for analysing the level and reciprocity of discussions in a Web‐based learning setting.     

Triple scheme of learning support design for scientific discovery learning based on computer simulation: experimental research

Abstract Learning support studies involving simulation‐based scientific discovery learning have tended to adopt an ad hoc strategies‐oriented approach in which the support strategies are typically pre‐specified according to learners' difficulties in particular activities. This article proposes a more integrated approach, a triple scheme for learning support design on the basis of the systematic analysis of the internal conditions of scientific discovery learning. The triple learning support scheme involves: (a) interpretative support that helps learners with knowledge access and the generation of meaningful and integrative understandings; (b) experimental support that scaffolds learners in systematic and valid experimental activities; and (c) reflective support that increases learners' self‐awareness of the discovery processes and prompts their reflective abstraction and integration. Two experiments were conducted with eighth graders (13‐year‐olds) to examine the effects of these learning supports embedded into a simulation program on floating and sinking. The overall results support the main hypotheses that learning supports in a simulation environment should be directed towards the three perspectives to invite meaningful, systematic, and reflective discovery learning.     

Analyzing collaborative knowledge construction: multiple methods for integrated understanding

Documenting collaborative knowledge construction is critical for research in computer-supported collaborative learning. Because this is a multifaceted phenomenon, mixed methods are necessary to construct a good understanding of collaborative interactions, otherwise there is a risk of being overly reductionistic. In this paper I use quantitative methods of verbal data analysis, qualitative analysis, and techniques of data representation to characterize two successful knowledge building interactions from a sociocultural perspective. In the first study, a computer simulation helped mediate the interaction and in the second, a student-constructed representation was an important mediator. A fine-grained turn-by-turn analysis of the group discussions was supplemented with qualitative analysis of larger units of dialogue. In addition, chronological representations of discourse features and tool-related activity were used in study 2 to gain an integrated understanding of how a student-generated representation mediated collaborative knowledge construction. It is only by mixing methods that collaborative knowledge construction can be well characterized.     

Homogeneous and heterogeneous multiple representations in equation-solving problems: An eye-tracking study

Multiple external representations (MERs) play an important role in the learning field of mathematics. Whereas the cognitive theory of multimedia learning and the integrative text and picture comprehension model assume that the heterogeneous combination of symbolic and analogous representations fosters learning; the design, functions, and tasks framework holds that learning benefits depend on the specific functions of MERs. The current paper describes a conceptual replication study of one of the few studies comparing single representations, heterogeneous, and homogeneous MERs in the context of mathematics learning. In a balanced incomplete block design, the participants were provided single representations (a graphic, text, or formula) or a heterogeneous (e.g., text + graphic) or homogeneous (text + formula) combination of these to solve linear system of equations problems. In accordance with previous research, performance was superior in conditions providing MERs compared to single-representation conditions. Moreover, heterogeneous MERs led to time savings over homogeneous MERs which triggered an increase in cognitive load. Contrary to previous research, text was the least fixated representation whereas the graphical representation proved to be most beneficial. With regard to practical implications, experts should be fostered through more challenging homogeneous MERs whereas novices should be supported through the accessible graphic contained in heterogeneous MERs.