The evaluation of different gaming modes and feedback types on game-based formative assessment in an online learning environment

This study proposed an online learning system for energy education, modifying the typical rules of tic-tac-toe and incorporating multiple choice tests into the game in order to develop a game-based formative assessment tool for an online learning course. In order to explore how different gaming modes and feedback types in this game-based formative assessment affect knowledge acquisition effectiveness and participation perceptions, a tic-tac-toe quiz game (TRIS-Q) with two gaming modes: single-player online game (SOG) and multi-player online game (MOG), and two feedback types: immediate elaborated feedback (IEF) and no immediate elaborated feedback (no IEF), were developed. A 2(SOG vs. MOG) × 2(IEF vs. no IEF) between-subject experiment was also conducted to investigate the effects on 109 ninth-grade students from four junior high school classes. The research findings indicated that different gaming modes of TRIS-Q did not affect the effectiveness of knowledge acquisition; providing IEF for each question answered in the game facilitated the enhancement of both energy knowledge acquisition and student tic-tac-toe ability when comparing it with the no IEF type. Additionally, the different gaming modes and feedback types did not affect participation perceptions.     

Incorporating customization and personalization into game-based learning: A cognitive style perspective

In recent years, customization and personalization were widely applied to accommodate the needs of different cognitive style groups. Such two approaches have different advantages and disadvantages but there is a lack of studies to compare these two approaches from the perspective of game-based learning, which is currently popular in educational settings. To this end, we developed a customized game-based learning system and a personalized game-based learning system and conducted two empirical studies to examine how cognitive styles affected learner's reactions to these two game-based learning systems. The results from the customized game-based learning system showed that Holists might not always favor to listen to music because they frequently switched on/off music. On the other hand, Serialists did not prefer to use hints. In addition, learners with the customized game-based learning system had more positive perceptions while learners with the personalized game-based learning system had more negative perceptions though both systems were useful to enhance learners' learning performance, regardless of their cognitive styles.     

Note-taking while learning hypermedia: Cognitive and motivational considerations

The non-linear format of hypermedia requires the use of self-regulated learning (SRL) processes, including making decisions about which representation to access. The design of hypermedia environments can make these decisions difficult, and may result in extraneous cognitive load. Note-taking, a SRL strategy may offload extraneous cognitive load while learning with hypermedia. This study examined factors related to undergraduate students’ note-taking while learning with hypermedia. Think-aloud, self-report, pretest, and posttest data were collected from 53 undergraduates while they learned about a challenging science topic for 30 min with hypermedia. Results indicated that participants’ prior domain knowledge, as measured by the pretest, did not significantly predict the content of their notes. However, motivation, as measured by a self-report questionnaire, had a significant relationship with the content of the participants’ notes. Specifically, intrinsic motivation had a significant negative relationship while extrinsic motivation had a significant positive relationship with the content of notes. Additionally, there was a significant interaction between the content of the participants’ notes and their use of self-regulatory processes while learning with hypermedia.     

The influence of virtual presence: Effects on experienced cognitive load and learning outcomes in educational computer games

Does the immersive design of an educational gaming environment affect learners’ virtual presence and how much do they learn? Does virtual presence affect learning? This study tries to answer these questions by examining the differences in virtual presence and learning outcomes in two different computer-based multimedia environments: a gaming environment with high immersive design vs. hypertext learning environment with low immersive design. As the main focus, the effect of virtual presence on learning is also explained and tested. By identifying virtual presence as a variable that may determine learning, it is argued that computer gaming environments present a new challenge for researchers to investigate, particularly, the effects of virtual presence on the immersive design of games in order to help designers to predict which instructional configurations will maximize learning performance. In general, results revealed that the high-immersive gaming environment leads to the strongest form of virtual presence but also decreased learning. Although regression analyses indicate that virtual presence positively influences trivial- and non-trivial learning outcomes, learners who learned in a low-immersive environment outperformed the gaming group. A mediation analysis showed that the relation between virtual presence and non-trivial learning outcomes is partly mediated through increased cognitive load.     

The concept of flow in collaborative game-based learning

Generally, high-school students have been characterized as bored and disengaged from the learning process. However, certain educational designs promote excitement and engagement. Game-based learning is assumed to be such a design. In this study, the concept of flow is used as a framework to investigate student engagement in the process of gaming and to explain effects on game performance and student learning outcome. Frequency 1550, a game about medieval Amsterdam merging digital and urban play spaces, has been examined as an exemplar of game-based learning. This 1-day game was played in teams by 216 students of three schools for secondary education in Amsterdam. Generally, these students show flow with their game activities, although they were distracted by solving problems in technology and navigation. Flow was shown to have an effect on their game performance, but not on their learning outcome. Distractive activities and being occupied with competition between teams did show an effect on the learning outcome of students: the fewer students were distracted from the game and the more they were engaged in group competition, the more students learned about the medieval history of Amsterdam. Consequences for the design of game-based learning in secondary education are discussed.     

FARMTASIA: an online game-based learning environment based on the VISOLE pedagogy

Virtual interactive student-oriented learning environment (VISOLE) is a game-based constructivist pedagogical approach that encompasses the creation of an online interactive world modeled upon a set of interdisciplinary domains, in which students participate as “citizens” to take part cooperatively and competitively in shaping the development of the virtual world as a means to construct their knowledge and skills. FARMTASIA is the first online game designed using the VISOLE philosophy, encompassing the subject areas of biology, government, economics, technology, production system and natural environment. The “virtual world” deployed is a farming system covering the domains of cultivation, horticulture and pasturage, situated in a competitive economy governed by good public policies. The design and implementation of FARMTASIA pursue three vital principles. The first one is to make the game as realistic as possible so that students can learn in a near-real life environment; the second one is to inject motivational elements so that students can sustain to learn and acquire various knowledge and skills with the game; and the third one is to make easy for teachers to conduct various VISOLE facilitation tasks. According to our exploratory educational study, we show evidentially that positive perceptions and an advancement of subject-specific and interdisciplinary knowledge appeared among the students who participated in VISOLE learning with FARMTASIA.     

The effects of time-compressed instruction and redundancy on learning and learners’ perceptions of cognitive load

Can increasing the speed of audio narration in multimedia instruction decrease training time and still maintain learning? The purpose of this study was to examine the effects of time-compressed instruction and redundancy on learning and learners’ perceptions of cognitive load. 154 university students were placed into conditions that consisted of time-compression (0%, 25%, or 50%) and redundancy (redundant text and narration or narration only). Participants were presented with multimedia instruction on the human heart and its parts then given factual and problem solving knowledge tests, a cognitive load measure, and a review behavior (back and replay buttons) measure. Results of the study indicated that participants who were presented 0% and 25% compression obtained similar scores on both the factual and problem solving measures. Additionally, they indicated similar levels of cognitive load. Participants who were presented redundant instruction were not able to perform as well as participants presented non-redundant instruction.     

Examining different types of collaborative learning in a complex computer-based environment: A cognitive load approach

This study compared the effects of two collaborative learning strategies (Open-ended and Task-based) with an individualized learning strategy on individual learning in a computer-based environment. The experiment sought ecological validity by conducting it under real teaching and homework conditions. Ninety-four students from grade 9 participated in a webpage design task. Cognitive load theory was used to predict that the collaborative approaches would outperform the individualized approach due to reduced cognitive load. This hypothesis was confirmed by performance scores and cognitive load only in the case of the Open-ended collaborative learning condition. Evidence was also found that the Open-ended collaborative learning condition outperformed the Task-based collaborative one. It was concluded that in collaborative learning a more Open-ended task design together with moderate independent sub-task requirements leads to more effective learning.     

The interplay of game elements with psychometric qualities,learning, and enjoyment in game-based assessment

Educators today are increasingly interested in using game-based assessment to assess and support students' learning. In the present study, we investigated how changing a game design element, linearity in gameplay sequences, influenced the effectiveness of game-based assessment in terms of validity, reliability, fairness, learning, and enjoyment. Two versions of a computer game, Physics Playground (formerly Newton's Playground), with different degrees of linearity in gameplay sequences were compared. Investigation of the assessment qualities—validity, reliability, and fairness—suggested that changing one game element (e.g., linearity) could significantly influence how players interacted with the game, thus changing the evidentiary structure of in-game measures. Although there was no significant group difference in terms of learning between the two conditions, participants who played the nonlinear version of the game showed significant improvement on qualitative physics understanding measured by the pre- and posttests while the participants in the linear condition did not. There was also no significant group difference in terms of enjoyment. Implications of the findings for future researchers and game-based assessment designers are discussed.     

Does the emotional design of scaffolds enhance learning and motivational outcomes in game-based learning?

Background

In recent years, the importance of emotions in learning has been increasingly recognized. Applying emotional design to induce positive emotions has been considered a means to enhance the instructional effectiveness of digital learning environments. However, only a few studies have examined the specific effects of emotional design in game-based learning.

Objectives

This quasi-experimental study utilized a value-added research approach to investigate whether emotional design applied to scaffolding in a game-based learning environment improves learning and motivational outcomes more than emotionally neutral scaffolding.

Methods

A total of 138 participants, mean age of 11.5 (SD = 0.73) participated in the study. A total of 68 participants played the base version of a fraction learning game (Number Trace), where scaffolding was provided with emotionally neutral mathematical notations, and 70 participants played the value-added version of the game using emotionally designed animated scaffolding agents. Pre-and post-tests were used to measure conceptual fraction knowledge and self-reported measures of situational interest and situational self-efficacy to evaluate motivational outcomes.

Results and Conclusions

Our results indicate that the emotional design applied to scaffolds can improve the educational value of a game-based learning environment by enhancing players' situational interest and situational self-efficacy. However, although the intervention improved the participants' conceptual fraction knowledge, there was no significant difference between the scaffolding conditions in participants' learning outcomes.

Takeaways

The results suggest that emotional design can increase the educational impact of game-based learning by promoting the development of interest, as well as improving self-efficacy.     

Analytics of learning tactics and strategies in an online learnersourcing environment

Background

The use of crowdsourcing in a pedagogically supported form to partner with learners in developing novel content is emerging as a viable approach for engaging students in higher-order learning at scale. However, how students behave in this form of crowdsourcing, referred to as learnersourcing, is still insufficiently explored.

Objectives

To contribute to filling this gap, this study explores how students engage with learnersourcing tasks across a range of course and assessment designs.

Methods

We conducted an exploratory study on trace data of 1279 students across three courses, originating from the use of a learnersourcing environment under different assessment designs. We employed a new methodology from the learning analytics (LA) field that aims to represent students' behaviour through two theoretically-derived latent constructs: learning tactics and the learning strategies built upon them.

Results

The study's results demonstrate students use different tactics and strategies, highlight the association of learnersourcing contexts with the identified learning tactics and strategies, indicate a significant association between the strategies and performance and contribute to the employed method's generalisability by applying it to a new context.

Implications

This study provides an example of how learning analytics methods can be employed towards the development of effective learnersourcing systems and, more broadly, technological educational solutions that support learner-centred and data-driven learning at scale. Findings should inform best practices for integrating learnersourcing activities into course design and shed light on the relevance of tactics and strategies to support teachers in making informed pedagogical decisions.     

Challenging games help students learn: An empirical study on engagement,flow and immersion in game-based learning

In this paper, we investigate the impact of flow (operationalized as heightened challenge and skill), engagement, and immersion on learning in game-based learning environments. The data was gathered through a survey from players (N = 173) of two learning games (Quantum Spectre: N = 134 and Spumone: N = 40). The results show that engagement in the game has a clear positive effect on learning, however, we did not find a significant effect between immersion in the game and learning. Challenge of the game had a positive effect on learning both directly and via the increased engagement. Being skilled in the game did not affect learning directly but by increasing engagement in the game. Both the challenge of the game and being skilled in the game had a positive effect on both being engaged and immersed in the game. The challenge in the game was an especially strong predictor of learning outcomes. For the design of educational games, the results suggest that the challenge of the game should be able to keep up with the learners growing abilities and learning in order to endorse continued learning in game-based learning environments.     

Recognizing and measuring self-regulated learning in a mobile learning environment

With the realization that more research is needed to explore external factors (e.g., pedagogy, parental involvement in the context of K-12 learning) and internal factors (e.g., prior knowledge, motivation) underlying student-centered mobile learning, the present study conceptually and empirically explores how the theories and methodologies of self-regulated learning (SRL) can help us analyze and understand the processes of mobile learning. The empirical data collected from two elementary science classes in Singapore indicates that the analytical SRL model of mobile learning proposed in this study can illuminate the relationships between three aspects of mobile learning: students’ self-reports of psychological processes, patterns of online learning behavior in the mobile learning environment (MLE), and learning achievement. Statistical analyses produce three main findings. First, student motivation in this case can account for whether and to what degree the students can actively engage in mobile learning activities metacognitively, motivationally, and behaviorally. Second, the effect of students’ self-reported motivation on their learning achievement is mediated by their behavioral engagement in a pre-designed activity in the MLE. Third, students’ perception of parental autonomy support is not only associated with their motivation in school learning, but also associated with their actual behaviors in self-regulating their learning.     

How competition and heterogeneous collaboration interact in prevocational game-based mathematics education

The present study addresses the effectiveness of an educational mathematics game for improving proportional reasoning in students from prevocational education. Though in theory game-based learning is promising, research shows that results are ambiguous and that we should look into ways to support game-based learning. The current study explored two factors (i.e., collaboration and competition) that have been associated with motivational and cognitive effects, and have potential to optimize game-based learning. In a fully crossed design, four conditions were examined: collaboration and competition, collaboration control, competition control, and control. It was found that, over all, gameplay did improve students' proportional reasoning skills but that learning effects did not differ between conditions. However, when students’ ability levels were taken into account, an interaction between collaboration and competition was found. For below-average students, the effect of collaboration was modified by competition, showing a negative effect of competition on domain knowledge gains in a collaborative learning situation. In contrast, for above-average students, the data demonstrated a trend that suggests a positive effect of competition on domain knowledge gain in a collaborative learning situation.     

The interplay between cognitive and motivational variables in a supportive online learning system for secondary physical education

A path model was used to test the unique and interactive effects of cognitive and motivational variables when learning in a supportive online learning system, Collaborative Inquiry System (CIS). In this student-centered learning environment, students interact with computer simulations and are assisted by online scaffolds intended to help them learn complex scientific concepts. The present study also explored the relationships between students’ motivational, cognitive, and metacognitive strategy use and online performance. In total, 178 tenth-grade students participated in the study. The statistical analyses revealed that students’ learning goals and cognitive preferences predicted metacognitive strategy use and later influenced their performance. Prior knowledge is a predictor of neither metacognitive strategy use nor learning goals and need for cognition, but is nevertheless an important determinant of online performance. Discussions on how to accommodate the different needs of students with varying levels of prior knowledge, goal orientation, and cognitive preference in a supportive learning system are provided based on the results.     

Micro-persistence and difficulty in a game-based learning environment for computational thinking acquisition

Persistence has been identified as a crucial quality of learning. However, it is hard to attain in online game-based environments as the drive to progress in the game may influence the ability to achieve the learning goals. This study aimed to examine the associations between micro-persistence, that is, the tendency to complete an individual task successfully, and task difficulty while acquiring computational thinking (CT). We further explored whether contextual or personal attributes better explain micro-persistence. We analysed data of 111 school students who used the CodeMonkey platform. We took a learning analytics approach for analysing the platform's log files. We found that micro-persistence is associated with task difficulty and that students who demonstrated an aptitude to learn new material are motivated to achieve the best solution. We also found that contextual variables better-explained micro-persistence than personal attributes. Encouraging micro-persistence can improve CT acquisition and the learning processes involved.     

An examination of two mental workload measurement approaches to understanding multimedia learning

This study reports on an examination of two measures of mental workload: the NASA-TLX and Paas’ Subjective Cognitive Load (SCL) measure. The goal was to assess the relative efficacy of the measures in the design and research of multimedia learning environments. Benchmarks based on the literature as to the goals for mental workload measurement in learning research are established. A multifaceted study was conducted which manipulated various aspects of mental workload in order to study the utility of these two measures in detecting changes in load and their relationship to learning outcomes. The results indicate that the weighted version of the NASA-TLX provided little additional value over the unweighted version of the measure. While both the NASA-TLX and SCL measures were sensitive to changes in both intrinsic and extraneous load, the study revealed differences in the measures based on levels of each of these load factors. The study also concludes that a better understanding of the third factor, germane load, will be needed to both expand the theoretical framework about mental workload in instructional settings and further understand the utility of these two measures.     

The impact of goal specificity and goal type on learning outcome and cognitive load

Two hundred and thirty three 15-year old students conducted experiments within a computer-based learning environment. They were provided with different goals according to an experimental 2 × 2 design with goal specificity (nonspecific goals versus specific goals) and goal type (problem solving goals versus learning goals) as factors. We replicated the findings of other researchers that nonspecific problem solving goals lead to lower cognitive load and better learning than specific problem solving goals. For learning goals, however, we observed this goal specificity effect only on cognitive load but not on learning outcome. Results indicate that the goal specificity affects the element interactivity of a task and cognitive load with both, problem solving goals or learning goals. But differences in overall cognitive load are not sufficient for explaining differences in learning outcome. Additionally, differences in strategy use come into play. Specific problem solving goals seem to restrict students to use a problem solving strategy whereas nonspecific problem solving goals or learning goals allow students to use a learning strategy. We conclude that in order to foster learning, students must be provided with goals that allow them to use a learning strategy. Additionally, providing them with nonspecific goals decreases cognitive load and, thus, enables students to learn with less effort.     

The cognitive impact of interactive design features for learning complex materials in medical education

To identify the most effective way for medical students to interact with a browser-based learning module on the symptoms and neurological underpinnings of stroke syndromes, this study manipulated the way in which subjects interacted with a graphical model of the brain and examined the impact of functional changes on learning outcomes. It was hypothesized that behavioral interactions that were behaviorally more engaging and which required deeper consideration of the model would result in heightened cognitive interaction and better learning than those whose manipulation required less deliberate behavioral and cognitive processing. One hundred forty four students were randomly assigned to four conditions whose model controls incorporated features that required different levels of behavioral and cognitive interaction: Movie (low behavioral/low cognitive, n = 40), Slider (high behavioral/low cognitive, n = 36), Click (low behavioral/high cognitive, n = 30), and Drag (high behavioral/high cognitive, n = 38). Analysis of Covariates (ANCOVA) showed that students who received the treatments associated with lower cognitive interactivity (Movie and Slider) performed better on a transfer task than those receiving the module associated with high cognitive interactivity (Click and Drag, partial eta squared = .03). In addition, the students in the high cognitive interactivity conditions spent significantly more time on the stroke locator activity than other conditions (partial eta squared = .36). The results suggest that interaction with controls that were tightly coupled with the model and whose manipulation required deliberate consideration of the model's features may have overtaxed subjects' cognitive resources. Cognitive effort that facilitated manipulation of content, though directed at the model, may have resulted in extraneous cognitive load, impeding subjects in recognizing the deeper, global relationships in the materials. Instructional designers must, therefore, keep in mind that the way in which functional affordances are integrated with the content can shape both behavioral and cognitive processing, and has significant cognitive load implications.     

Productive use of learning resources in an online problem-based learning environment

This study examined students’ use of learning resources in a technologically-mediated online learning environment. Undergraduate student groups were engaged in an online problem-based learning (PBL) environment, rich with pre-selected video and knowledge resources. Quantitative and qualitative analyses showed that students accessed resources fairly frequently and benefited from them. Resources helped students construct a rich understanding of the problem and provided ideas for problem solutions. Detailed analyses of resource exploration along with contrasting case analyses between high-achieving and low-achieving student groups suggested that for learning to be effective in resource-rich environments, students first need to develop an understanding of the resources and learn how to access them efficiently. Second, students need to learn to process the contents of resources in meaningful ways so that they can integrate diverse resources to form a coherent understanding and apply them to solve problems. Finally, students need to develop knowledge and skills to use resources collaboratively, such as sharing and relating to each other’s resources. The results indicated that students, especially low-achieving students, need guidance to use resources effectively in resource-rich learning environments.