The interplay of game elements with psychometric qualities,learning, and enjoyment in game-based assessment

Educators today are increasingly interested in using game-based assessment to assess and support students' learning. In the present study, we investigated how changing a game design element, linearity in gameplay sequences, influenced the effectiveness of game-based assessment in terms of validity, reliability, fairness, learning, and enjoyment. Two versions of a computer game, Physics Playground (formerly Newton's Playground), with different degrees of linearity in gameplay sequences were compared. Investigation of the assessment qualities—validity, reliability, and fairness—suggested that changing one game element (e.g., linearity) could significantly influence how players interacted with the game, thus changing the evidentiary structure of in-game measures. Although there was no significant group difference in terms of learning between the two conditions, participants who played the nonlinear version of the game showed significant improvement on qualitative physics understanding measured by the pre- and posttests while the participants in the linear condition did not. There was also no significant group difference in terms of enjoyment. Implications of the findings for future researchers and game-based assessment designers are discussed.     

Using commercial video games in flipped classrooms to support physical concept construction

Based on the flipped‐classroom model and the potential motivational and instructional benefits of digital games, we describe a flipped game‐based learning (FGBL) strategy focused on preclass and overall learning outcomes. A secondary goal is to determine the effects, if any, of the classroom aspects of the FGBL strategy on learning efficiency. Our experiments involved 2 commercial games featuring physical motion concepts: Ballance (Newton's law of motion) and Angry Birds (mechanical energy conservation). We randomly assigned 87 8th‐grade students to game instruction (digital game before class and lecture‐based instruction in class), FGBL strategy (digital game before class and cooperative learning in the form of group discussion and practice in class), or lecture‐based instruction groups (no gameplay). Results indicate that the digital games exerted a positive effect on preclass learning outcomes and that FGBL‐strategy students achieved better overall learning outcomes than their lecture‐based peers. Our observation of similar overall outcomes between the cooperative learning and lecture‐based groups suggests a need to provide additional teaching materials or technical support when introducing video games to cooperative classroom learning activities.     

Applying the self-explanation principle to multimedia learning in a computer-based game-like environment

What is the most effective way to incorporate self-explanation into an educational game? In Experiment 1, students who played a 10-level computer game about electrical circuits performed better on an embedded transfer test (i.e., level 10) if they were required to select the reason for each move from a list on levels 1–9 (selection self-explanation) than if they were not required to engage in self-explanation (d = 1.20). In Experiment 2, the same pattern of results was replicated (d = 0.71), but students who were required to type in their reason for each move on levels 1–9 (generation self-explanation) did not perform any better than those who were not required to engage in self-explanation (d = −0.06). Overall, asking students to select a reason from a list fosters some degree of reflection while not overly disrupting the flow of the game.     

Maximizing learning without sacrificing the fun: Stealth assessment,adaptivity and learning supports in educational games

In this study, we investigated the validity of a stealth assessment of physics understanding in an educational game, as well as the effectiveness of different game-level delivery methods and various in-game supports on learning. Using a game called Physics Playground, we randomly assigned 263 ninth- to eleventh-grade students into four groups: adaptive, linear, free choice and no-treatment control. Each condition had access to the same in-game learning supports during gameplay. Results showed that: (a) the stealth assessment estimates of physics understanding were valid—significantly correlating with the external physics test scores; (b) there was no significant effect of game-level delivery method on students' learning; and (c) physics animations were the most effective (among eight supports tested) in predicting both learning outcome and in-game performance (e.g. number of game levels solved). We included student enjoyment, gender and ethnicity in our analyses as moderators to further investigate the research questions.     

Supporting conceptual understanding of the associative and distributive properties through digital gameplay

There has been a steady rise in the support for games as learning environments. This support is largely based on the strong levels of engagement and motivation observed during gameplay. What has proven difficult is the ability to consistently design and develop learning games that are both engaging and educationally viable. Those in the game‐based learning community contend the solution is to tightly integrate instructional objective(s) within the game's mechanics. In order to investigate whether or not intrinsic integration would aid in game design, an experimental study was conducted with 111 fourth and fifth graders in which three versions of a game were designed in order to determine which approach would have a more significant impact on conceptual understanding of the associative and distributive properties. It was found that those who played the intrinsic version of the game (properties mapped to game mechanics) had significant learning gains compared with those who played the extrinsic version. Perceptions of the game are also discussed, along with future directions.     

The effectiveness of adaptive difficulty adjustments on students' motivation and learning in an educational computer game

Computer games that adaptively adjust difficulty are used to continuously challenge players according to their abilities. The adjustment of difficulty occurs automatically in response to a game's ongoing assessment of a player's performance. This approach to difficulty adjustment is likely to be of value in educational computer games as a means of scaffolding learning for students. However, there is limited research evaluating the effectiveness of educational computer games with adaptive difficulty adjustment when compared to non-adaptive difficulty adjustment. To expand on this research a quasi-experimental study was designed to isolate the impact of the difficulty adjustment game element on motivation and learning. A total of 234 secondary school students were allocated to one of three activities involving learning about Spanish cognates: an adaptive difficulty adjustment game, an incremental difficulty adjustment game that was non-adaptive, and a written activity. The three learning activities were designed following the same learning and motivation theories. The two games were identical apart from the difficulty adjustment mechanism. The results for motivation indicated that all students experienced high levels and there was no significant difference between the three learning activities. The pre- and post-tests results for learning indicated that significantly higher learning outcomes were achieved by students who played the adaptive game. Analysis of a game log recording the correctness of students' responses indicated that the adaptive difficulty adjustment game, in contrast to the non-adaptive incremental difficulty adjustment game, provided a scaffolding structure to enhance student learning.     

Learning immunology in a game: Learning outcomes,the use of player characters,immersion experiences and visual attention distributions

This study contributed to the current body of literature on game-based learning by investigating the way playing an educational game, Humunology, affected learning about the immune system and examining further the association between game immersion and visual attention distribution. A total of 79 undergraduate and graduate students participated, and data were collected both in situ and ex situ. The results showed that the students learned through playing Humunology, and the analyses of the use of player characters indicated that the game design facilitated gameplay behaviours that are consistent with the science content. The use of the eye-tracking method also revealed that students who were more immersed in playing Humunology paid more attention to areas related to player characters. The interpretations and limitations are discussed further.     

A case study of the in-class use of a video game for teaching high school history

This study examines the case of a sophomore high school history class where Making History, a video game designed with educational purposes in mind, is used in the classroom to teach about World War II. Data was gathered using observation, focus group and individual interviews, and document analysis. The high school was a rural school located in a small town in the Midwestern United States. The teacher had been teaching with the game for several years and spent one school week teaching World War II, with students playing the game in class for three days of that week. The purpose of this study was to understand teacher and student experiences with and perspectives on the in-class use of an educational video game. Results showed that the use of the video game resulted in a shift from a traditional teacher-centered learning environment to a student-centered environment where the students were much more active and engaged. Also, the teacher had evolved implementation strategies based on his past experiences using the game to maximize the focus on learning.     

Digital game-based learning in a Shanghai primary-school mathematics class: A case study

This study examined the perceptions and experiences of a teacher and students in a Shanghai public primary school when digital games were used in a second-grade math class. The participants included one teacher and 45 students. Data collection methods included classroom observation, focus-group and individual interviews, and document analysis. Digital gameplay, when used once daily over a 6-day period, was found to enhance student engagement and interest in learning; many students, however, were concerned about its effect on academic achievement and eyesight. The teacher employed a “making thinking visible” approach using pencil and paper and problem-solving strategies to help students understand the game's mathematical principles and master mathematical knowledge. Obstacles included large class size, difficulty evaluating learning outcomes, balancing fun and learning, and effective game–classroom integration. This study found that exam-oriented education and traditional teacher-centred teaching in China influenced participants' perceptions and experiences of digital game-based learning.     

The role of executive control in young children's serious gaming behavior

The present study examined (1) how executive control contributed to in-game behaviors in young children while playing a serious game, (2) whether the levels of control changed when the game was played repeatedly, and (3) how the first experience with the game mediated the role of executive control to in-game behaviors when the game was repeated. Attentional and action control were directly assessed in 106 kindergartners, who played a single-leveled serious game twice. During their gameplay, the following behaviors were registered: time, number of scaffolds needed, mistakes, verbal expressions, questions, irrelevant game activities (drawings), and off-task behavior. The results for the first game round showed that time, expressions, and the need for scaffolds were predicted by attentional control. In the second round, a strong role for action control was found to overcome off-task behavior and irrelevant drawings. Verbal expressiveness was again influenced by attentional control. Moreover, mediation effects of attentional control to efficient in-game behaviors in the second gameplay were evidenced via scaffolding and expressiveness in the first gameplay. It is concluded that in new games children's attentional control contributes to formulating strategies and problem-solving, while their action control underlies sustained and goal-directed learning over time.     

Evaluating user experience of adaptive digital educational games with Activity Theory

Adaptive digital educational games (DEGs) providing players with relevant interventions can enhance gameplay experience. This advance in game design, however, renders the user experience (UX) evaluation of DEGs even more challenging. To tackle this challenge, we developed a four-dimension evaluation framework (i.e., gaming experience, learning experience, adaptivity, and usability) and applied it to an empirical study with a DEG on teaching geography. Mixed-method approaches were adopted to collect data with 16 boys aged 10–11. Specifically, a so-called Dyadic User Experience Tests (DUxT) was employed; participants were paired up to assume different roles during gameplay. Learning efficacy was evaluated with a pre-post intervention measurement using a domain-specific questionnaire. Learning experience, gaming experiences and usability were evaluated with intensive in situ observations and interviews guided by a multidimensional scheme; content analysis of these transcribed audio data was supplemented by video analysis. Effectiveness of adaptivity algorithms was planned to be evaluated with automatic logfiles, which, unfortunately, could not be realised due to some technical problem. Nonetheless, the user-based data could offer some insights into this issue. Furthermore, we attempted to bridge the existing gap in UX research – the lack of theoretical frameworks in understanding user experience – by adopting Engeström's (1987) extended framework of Activity Theory (AT) that provides contextual information essential for understanding contradictions and breakdowns observed in the interactions between the game players. The dyadic gameplay setting allows us to explore the issue of group UX. Implications for further applications of the AT framework in the UX research, especially the interplay between evaluation and redesign (i.e., downstream utility of UX evaluation methods), are discussed.     

The format of problem representation for in‐game learning supports

A promising method to support game‐based learning is to facilitate learners' externalization of cognitive and metacognitive processes. Externalizing Problem Representation (EPR) refers to a cognitive behaviour in which a learner constructs her own representations overtly. The purpose of this study is to investigate whether learning supports promoting EPR enhance qualitative understanding and quantitative proficiency in ratios and proportional relationships in a learning game (i.e., E‐Rebuild) context. Specifically, this study investigated the effects of representation format in problem representation on qualitative understanding and quantitative proficiency in a learning game context. The results of this study indicate that (a) symbolic learning supports better facilitate comprehension of math concepts and their relations than iconic learning supports in video game contexts, (b) symbolic learning supports better facilitate players' reflection for implicit understanding and promote their math problem‐solving skills, (c) participants in the symbolic learning support group increased significantly in qualitative understanding but not in quantitative proficiency after gameplay, and (d) participants in the iconic learning support group experienced significant growth in quantitative proficiency but not in qualitative understanding after gameplay.     

Scaffolding game-based learning: Impact on learning achievements,perceived learning,and game experiences

One of the central challenges of integrating game-based learning in school settings is helping learners make the connections between the knowledge learned in the game and the knowledge learned at school, while maintaining a high level of engagement with game narrative and gameplay. The current study evaluated the effect of supplementing a business simulation game with an external conceptual scaffold, which introduces formal knowledge representations, on learners' ability to solve financial-mathematical word problems following the game, and on learners' perceptions regarding learning, flow, and enjoyment in the game. Participants (M_age = 10.10 years) were randomly assigned to three experimental conditions: a “study and play” condition that presented the scaffold first and then the game, a “play and study” condition, and a “play only” condition. Although no significant gains in problem-solving were found following the intervention, learners who studied with the external scaffold before the game performed significantly better in the post-game problem-solving assessment. Adding the external scaffold before the game reduced learners' perceived learning. However, the scaffold did not have a negative impact on reported flow and enjoyment. Flow was found to significantly predict perceived learning and enjoyment. Yet, perceived learning and enjoyment did not predict problem-solving and flow directly predicted problem solving only in the “play and study” condition. We suggest that presenting the scaffold may have “problematized” learners' understandings of the game by connecting them to disciplinary knowledge. Implications for the design of scaffolds for game-based learning are discussed.     

Effects of Interactivity in Educational Games: A Mediating Role of Social Presence on Learning Outcomes

Networked interactivity is one of the essential factors that differentiate recent online educational games from traditional stand-alone CD-based games. Despite the growing popularity of online educational games, empirical studies about the effects of networked interactivity are relatively rare. The current study tests the effects of networked interactivity on game users' learning outcomes by comparing three groups (online educational quiz game vs. off-line educational quiz game vs. traditional classroom lecture). In addition, the study examines the mediating role of social presence in the context of educational games. Results indicate that networked interactivity in the online educational quiz game condition enhances game users' positive evaluation of learning, test performance, and feelings of social presence. However, there was no significant difference between the off-line educational quiz game and the lecture-based conditions in terms of learning outcomes. Further analyses indicate that feelings of social presence mediate the effect of networked interactivity on various learning outcomes. Theoretical and practical implications are discussed.     

Serious games and learning effectiveness: The case of It’s a Deal!

Although the value of serious games in education is undeniable and the potential benefits of using video games as ideal companions to classroom instruction is unquestionable, there is still little consensus on the game features supporting learning effectiveness, the process by which games engage learners, and the types of learning outcomes that can be achieved through game play. Our aim in this discussion is precisely to advance in this direction by providing evidence of some of the factors influencing the learning effectiveness of a serious game called It’s a Deal! This serious game was created for the purpose of teaching intercultural business communication between Spaniards and Britons in business settings in which English is used as the lingua franca. This paper hypothesizes that the immersive, all-embracing and interactive learning environment provided by the video game to its users may contribute to develop and enhance their intercultural communicative competence. The study attempts to answer three main research questions: (a) after playing It’s a Deal!, did the students sampled improve their intercultural awareness, intercultural knowledge and intercultural communicative competence in business English? (b) If they improved their intercultural learning, what are the factors influencing such improvement? And (c) if they did not improve their intercultural learning, what are the factors influencing such failure? The game participants who volunteered to take part in the study were all students of English Studies at the University of Alicante in the academic year 2010-2011. One hundred and six students completed both the pre-test and the post-test questionnaires, and played It’s a Deal! A sample of fifty students was selected randomly for the empirical study. The results obtained in the tests performed were compared and contrasted intra-group, both qualitatively and quantitatively, for the purpose of finding any statistically significant difference that may confirm whether or not there was an improvement in the students’ intercultural communicative competence in business English as a result of the implementation of the It’s a Deal! serious game. Findings of this study demonstrate that the video game is an effective learning tool for the teaching of intercultural communication between Spaniards and Britons in business settings in which English is used as the lingua franca. In particular, whereas the game had a small learning effect on intercultural awareness and a medium learning effect on intercultural knowledge, it had a large learning effect on intercultural communicative competence. The study also documents correlating factors that make serious games effective, since it shows that the learning effectiveness of It’s a Deal! stems from the correct balance of the different dimensions involved in the creation of serious games, specifically instructional content, game dimensions, game cycle, debriefing, perceived educational value, transfer of learnt skills and intrinsic motivation.     

Does individual or collaborative self-debriefing better enhance learning from games?

The primary aim of this study is to find out whether use of different self-debriefing modes affects learning from a game. In self-debriefing participants are led to reflect upon their game experiences by a set of debriefing questions. Two conditions were compared: Individual and Collaborative self-debriefing. The 45 participants first played the game of Lemonade Tycoon Deluxe, were tested for knowledge and self-debriefed in pairs or alone. Then they played the game once more and were tested again. Game scores increased significantly from the first to the second round of gameplay to an equal degree in both conditions. Knowledge scores of participants in individual self-debriefing increased significantly more than those of participants in the Collaborative condition. The study shows that game-based learning can be effectively scaffolded with self-debriefing. Future studies might investigate whether the type of self-debriefing differentially affects game motivation. In addition, attention to the role of feedback is called for.     

An old game,new experience: exploring the effect of players’ personal gameplay history on game experience

This experimental study explores how game experience differs between players with different gameplay histories within the same game universe. We are interested in how prolonged engagement with a game series affects the gameplay experience in relation to the most recent game version in the series. A total of 54 participants were divided into four groups depending on their gaming experience, namely non-gamers, new-gamers, old-gamers and core-gamers. They played the mobile version of Super Mario Run, and questionnaire data was collected after the gameplay session. The results of the study showed that not only the players’ personal gameplay history but also the length of experience or degree of familiarity with the game universe affected the experience of playing a new game in the same game universe. Additionally, familiarity with the game universe had a positive impact on the feeling of competence, immersion, emotions and flow.

     

How competition and heterogeneous collaboration interact in prevocational game-based mathematics education

The present study addresses the effectiveness of an educational mathematics game for improving proportional reasoning in students from prevocational education. Though in theory game-based learning is promising, research shows that results are ambiguous and that we should look into ways to support game-based learning. The current study explored two factors (i.e., collaboration and competition) that have been associated with motivational and cognitive effects, and have potential to optimize game-based learning. In a fully crossed design, four conditions were examined: collaboration and competition, collaboration control, competition control, and control. It was found that, over all, gameplay did improve students' proportional reasoning skills but that learning effects did not differ between conditions. However, when students’ ability levels were taken into account, an interaction between collaboration and competition was found. For below-average students, the effect of collaboration was modified by competition, showing a negative effect of competition on domain knowledge gains in a collaborative learning situation. In contrast, for above-average students, the data demonstrated a trend that suggests a positive effect of competition on domain knowledge gain in a collaborative learning situation.     

Does the emotional design of scaffolds enhance learning and motivational outcomes in game-based learning?

Background

In recent years, the importance of emotions in learning has been increasingly recognized. Applying emotional design to induce positive emotions has been considered a means to enhance the instructional effectiveness of digital learning environments. However, only a few studies have examined the specific effects of emotional design in game-based learning.

Objectives

This quasi-experimental study utilized a value-added research approach to investigate whether emotional design applied to scaffolding in a game-based learning environment improves learning and motivational outcomes more than emotionally neutral scaffolding.

Methods

A total of 138 participants, mean age of 11.5 (SD = 0.73) participated in the study. A total of 68 participants played the base version of a fraction learning game (Number Trace), where scaffolding was provided with emotionally neutral mathematical notations, and 70 participants played the value-added version of the game using emotionally designed animated scaffolding agents. Pre-and post-tests were used to measure conceptual fraction knowledge and self-reported measures of situational interest and situational self-efficacy to evaluate motivational outcomes.

Results and Conclusions

Our results indicate that the emotional design applied to scaffolds can improve the educational value of a game-based learning environment by enhancing players' situational interest and situational self-efficacy. However, although the intervention improved the participants' conceptual fraction knowledge, there was no significant difference between the scaffolding conditions in participants' learning outcomes.

Takeaways

The results suggest that emotional design can increase the educational impact of game-based learning by promoting the development of interest, as well as improving self-efficacy.     

User-adaptive explanatory program visualization: evaluation and insights from eye movements

User-adaptive visualization and explanatory visualization have been suggested to increase educational effectiveness of program visualization. This paper presents an attempt to assess the value of these two approaches. The results of a controlled experiment indicate that explanatory visualization allows students to substantially increase the understanding of a new programming topic. Furthermore, an educational application that features explanatory visualization and employs a user model to track users’ progress allows students to interact with a larger amount of material than an application which does not follow users’ activity. However, no support for the difference in short-term knowledge gain between the two applications is found. Nevertheless, students admit that they prefer the version that estimates and visualizes their progress and adapts the learning content to their level of understanding. They also use the application’s estimation to pace their work. The differences in eye movement patterns between the applications employing adaptive and non-adaptive explanatory visualizations are investigated as well. Gaze-based measures show that adaptive visualization captivates attention more than its non-personalized counterpart and is more interesting to students. Natural language explanations also accumulate a big portion of students’ attention. Furthermore, the results indicate that working memory span can mediate the perception of adaptation. It is possible that user-adaptation in an educational context provides a different service to people with different mental processing capabilities.