The effect of the modality principle to support learning with virtual reality: An eye-tracking and electrodermal activity study

Background

Virtual reality (VR) is considered a promising approach to support learning. An instructional design is essential to optimize cognitive processes. Studies show that VR has unique instructional and pedagogical requirements.

Objectives

To evaluate the effectiveness and applicability of the modality principle, which was previously validated in 2D classic multimedia, for learning with VR. The modality principle states that multimedia information presented as spoken narration is superior to on-screen text.

Methods

A prospective experimental study with two compared conditions of instruction: VR-based learning guided by on-screen text (n = 34) versus spoken narration (n = 28). Students' cognitive learning experiences were captured by eye-tracking and electrodermal activity (EDA). In addition, students' knowledge was evaluated using a pre–post knowledge test.

Results and Conclusions

Overall, there was no significant difference in knowledge retention between the participants who learned with on-screen text compared to spoken narration. However, results from the eye-tracking analysis showed that students who learned with the on-screen text devoted longer visual attention toward important learning activity areas of interest, suggesting a better ability to discern between relevant and irrelevant information. Conversely, students who learned with the spoken narration expressed significantly more EDA peak responses, proposing a higher cognitive load.

Implications

This study outlines that while learning with VR was effective, the modality principle might not apply to learning with VR. Moreover, the analysis of the learning process suggests even an inverse effect, favouring the provision of instructional scaffolds as on-screen text. Future research should evaluate this effect on long-term knowledge retention.     

Extending multimedia research: How do prerequisite knowledge and reading comprehension affect learning from text and pictures

The present study aimed at extending research on multimedia design principles by investigating their validity as a function of learners’ reading comprehension and scientific literacy. Students (N = 125; age: M = 15.11 years) learned about cell reproduction during their regular Biology lessons in one of six conditions resulting from cross-varying multimedia (text only vs. text plus animations) and text modality (spoken vs. written vs. spoken and written). Recall and transfer were assessed immediately after learning and again 1 week later. Overall, adding animations to text as well as using spoken rather than written text improved only immediate recall; in addition, a multimedia effect for delayed recall was observed for learners with higher levels of scientific literacy. A redundant presentation of text proved harmful especially for delayed performance measures. Reading comprehension did not moderate multimedia design effects. Students with more suitable cognitive prerequisites were better able to maintain performance from the immediate to the delayed tests. Future multimedia research should further investigate the boundary conditions that moderate multimedia effectiveness.     

Embedded blended learning within an Algebra classroom: a multimedia capture experiment

This two‐group, pretest‐posttest, quasi‐experimental study compared secondary students' learning of Algebra II materials over a 4‐week period when identical instruction by the same teacher was delivered through either embedded blended learning (treatment group; n = 32) or a live‐lecture classroom (control group; n = 24). For both groups, instruction was delivered in a normal classroom setting. A math test and a student survey were used to measure students' learning of Algebra II and satisfaction with the instruction. Students in the treatment group showed significantly greater gains in Algebra II test scores and evaluated their learning experiences significantly more positively than did the control group. The great majority (80%) of students in the treatment group preferred the embedded blended learning over traditional live lectures for future learning of math. Students' responses to open‐ended survey questions suggested that students in the treatment group appreciated the: (a) ability to control the pace of instruction; (b) new role of the classroom teacher; (c) lack of distraction in the blended learning environment; and (d) accessibility of the embedded multimedia lessons outside the classroom. This study suggests that screen‐capture instructional technology can be used towards establishing a teacher‐based, embedded blended learning environment within a secondary algebraic classroom.     

A comparison of temporal speech and text cueing in educational multimedia

This research focused on the prediction that children in their school setting would learn more from educational multimedia when critical information was presented as spoken instead of textual cues. Analyses of a study (n= 42) showed that 12‐year‐olds did not learn any more from temporal speech cueing than from temporal text cueing. The findings suggest that multimedia learning for children is a different kind of learning experience than for adults or older adolescents. The results indicate underdeveloped executive control of the referential connections in the children's working memory between reading screen text while listening to spoken cues, and between watching on‐screen animations play while listening to spoken cues. Further study is warranted. Implications may be derived for educational multimedia research in school settings.     

Effects of traditional,blended and e‐learning on students' achievement in higher education

The study investigates the effect of e‐learning, blended learning and classroom learning on students’ achievement. Two experimental groups together with a control group from Umm Al‐Qura University in Saudi Arabia were identified randomly. To assess students’ achievement in the different groups, pre‐ and post‐achievement tests were used. The results of the study (N = 148) show that there was a statistically significant difference between the three methods in terms of students’ achievement favouring the blended learning method (n = 55) with a substantial effect size of 1.34 (Hedges’ g). No significant difference was found between the e‐learning (n = 43) and traditional learning groups (n = 50) in terms of students’ achievement and with a negligible effect size of 0.02.     

The use of a computer tutorial as a replacement for human tuition in a mastery learning strategy

Computer Assisted Learning (CAL) is a traditional activity in tertiary education. Few available CAL programs exhibit user friendliness and most are complex simulation or modelling programs not primarily designed as instructional programs. Authoring languages are designed to enable teachers to write instructional programs without the need to learn computer languages. This investigation reports upon the use of a tutorial program written in the authoring language “STAF2” to replace human tuition as part of a mastery learning experience in undergraduate biology. A similar mastery learning experience, taught identically to both student groups served as a control. All learning was assessed cognitively in terms of Bloom's Taxonomy of Educational Objectives and the results compared between the human-taught and STAF2-taught groups. All learning experiences resulted in the majority of students attaining mastery. The students taught by the STAF2 computer program attained significantly higher scores for retention of knowledge, its application and evaluation when compared with those taught by a human teacher. Whilst there was no significant difference between the groups in terms of comprehension, analysis and synthesis, the mean scores and the percentage of students attaining mastery after computer tuition was higher than the human taught group. The implications of these results are discussed.     

Developing a web‐based multimedia assessment system for facilitating science laboratory instruction

This study developed a web‐based multimedia assessment system (WMA system) and applied it to science laboratory instruction. The goal was to improve students' knowledge acquisition under science laboratory instruction. The developed system enabled learners to perform self‐assessments by responding to multimedia technology test items online. The system recorded each learner's complete answer history and provided the students with personalized learning resources. This study adopted a quasi‐experimental research design. The learning content was an “experiment on separating mixtures.” Students participating in the research were divided into a typical science laboratory instruction group (TI group; n = 25) and a group that received instruction through the WMA system (WMA group; n = 26). Before instruction, all the students completed the conceptual knowledge and experimental knowledge pretests. During instruction, the TI group watched the teacher's demonstration experiment, and the students then performed the experiment in a real laboratory. In the WMA group, after learning through the WMA system, the students also performed the experiment in a real laboratory. After instruction, all the students completed the conceptual knowledge and experimental knowledge posttests. The findings indicated that the students in the WMA group showed significantly higher improvements in their scientific conceptual knowledge and experimental knowledge.     

Effect of blended learning environment on transfer of learning: an experimental study

The main purpose of the study is to investigate the effect of the blended learning and face‐to‐face learning on students' academic achievements and transfer of learning. The first aim of the study is to analyse the students' academic achievements to determine the change of theoretical knowledge in terms of the instructional method applied in this study. However, students' performances during the learning process are not always a good indicator. Therefore, the second aim of the study is to examine transferring students' theoretical knowledge to their educational multimedia projects. For this purpose, 14 weeks of pre‐ and post‐test experimental study is designed. The study is administered through an educational technology course in which students learn how to develop educational multimedia materials. The participants are undergraduate pre‐service teachers; they are assigned randomly to the experimental and control group. While the experimental group has both online and classroom sessions, the control group has only face‐to‐face classroom sessions. Although there is no significant difference between the groups in terms of academic achievement, the students in the blended group are more successful in transferring their knowledge to their projects than the ones in the face‐to‐face group. In conclusion, the results show that the blended learning approach has a positive effect on the transfer of learning.     

Adjunct aids and signals support online learning from multiple representations

To achieve deeper understanding when learning from multiple representations, learners should actively select, organize and integrate the relevant information from text and graphics within a coherent mental representation. However, as learners often fail to select and integrate all relevant information, especially from graphics, they need specific instructional support. The current study investigated the effects of instructional support in the form of adjunct aids (i.e. fill-in-the blank tasks) with references to the graphics (e.g. “see Figure X”) on retention and understanding. In our study, 106 learners (N) received multimedia instructional materials about the formation of auroras either with or without adjunct aids – the former with references to graphics (signals), or with none. In line with our hypotheses, adjunct aids with signals led to deeper understanding, as reflected by higher scores in the comprehension test. In contrast, adjunct aids with signals did not lead to higher scores in the retention test. Thus, our results are in line with previous research, showing that instructional support for integrating text and graphics specifically fosters deeper understanding. Possible boundary conditions and implications for future research are discussed.     

Cognitive load and science text comprehension: Effects of drawing and mentally imagining text content

One hundred and eleven 10th graders read an expository science text on the dipole character of water molecules (ca. 1600 words). Reading instruction was varied according to a 2 × 2 experimental design with factors ‘drawing pictures of text content on paper’ (yes, no) and ‘mentally imagining text content while reading’ (yes, no). The results indicate that drawing pictures, mediated through increased cognitive load, decreased text comprehension and, thus, learning (d = −0.37), whereas mental imagery, although decreasing cognitive load, increased comprehension only when students did not have to draw pictures simultaneously (d = 0.72). No evidence was found that the effects were moderated by domain-specific prior knowledge, verbal ability, or spatial ability. The results are in line with cognitive theories of multimedia learning, self-regulated learning, and mental imagery as well as conceptions of science learning that focus on promoting mental model construction by actively visualizing the content to be learned. Constructing mental images seems to reduce cognitive load and to increase comprehension and learning outcome when the mental visualization processes are not disturbed by externally drawing pictures on paper, whereas drawing pictures seems to increase cognitive load resulting in reduced comprehension and learning outcome.     

Effects of self-generated graphic organizers on learning depend on in-task guidance

We investigated the effects of readymade versus self-generated graphic organizers (GOs) on learning, comparing the performances of undergraduates (N = 81) tasked with learning a multimedia document. This document was either presented on its own (control group), with a readymade GO, or with a blank GO that students had to fill in either before or during the learning of the document. In line with previous research, adding a readymade GO increased students' memorization and transfer scores, compared with controls. By displaying the main ideas in the text and their hierarchical relations, GOs act as visual aids to learning. Results showed that self-generating a GO was no more beneficial than viewing a readymade GO when students were placed in a dual-task situation (generation + learning). However, when the students' information processing was guided by sequencing these tasks (generation then learning), they outperformed the control and readymade groups on memorization and comprehension.     

Effects of providing partial hypotheses as a support for simulation-based inquiry learning

Hypothesis generation is an important but difficult process for students. This study investigated the effects of providing students with support for hypothesis generation, with regard to the testability and complexity of the generated hypotheses, the quality of the subsequent inquiry learning processes and knowledge acquisition. Fifty-two secondary school students completed three prior knowledge tests and worked on an inquiry task in the domain of force and motion, concerning the topic of Newton's first law of motion. They received either a set of terms (variables, conditions and relations) to help them generate hypotheses (T condition, n = 23) or the same set of terms plus a partial hypothesis to start from (T + PHy condition, n = 29). Results showed that students in the T + PHy condition generated more complex hypotheses, performed better at data collection and acquired more domain knowledge than students in the T condition. No effects of prior knowledge were found.     

Improving summary writing through formative feedback in a technology-enhanced learning environment

Summary writing is a useful instructional tool for learning. However, summary writing is a challenge to many students. This mixed-method study examined the potential of the Student Mental Model Analyzer for Research and Teaching (SMART) system to help students produce summaries that reflect key concepts and relations in a text. SMART uses the students' summary to generate a multi-dimensional 3S (surface, structure, semantic) evaluation of the students' mental model. This model is then used to drive feedback to help students revise their summary. The current study is an initial investigation examining whether writing and revising in SMART improves students' summary quality. Students (n = 38) in a graduate-level online course used SMART for seven reading assignments. The 38 students submitted a total of 357 summaries in response to the seven readings. In 47 cases, students produced both an initial draft and a modified revision. These 47 cases were selected for analysis. In the quantitative phase, MANOVA results indicated that students' summaries improved along the 3S dimensions from initial draft to revision. In the qualitative phase, inspection of exemplar cases revealed how students' mental models changed towards more robust and cohesive knowledge structure for texts.     

3D visualization types in multimedia applications for science learning: A case study for 8th grade students in Greece

This research aims to determine whether the use of specific types of visualization (3D illustration, 3D animation, and interactive 3D animation) combined with narration and text, contributes to the learning process of 13- and 14- years-old students in science courses. The study was carried out with 212 8th grade students in Greece. This exploratory study utilizes three different versions of an interactive multimedia application called “Methods of separation of mixtures”, each one differing from the other two in a type of visuals. The results indicate that multimedia applications with interactive 3D animations as well as with 3D animations do in fact increase the interest of students and make the material more appealing to them. The findings also suggest that the most obvious and essential benefit of static visuals (3D illustrations) is that they leave the time control of learning to the students and decrease the cognitive load.     

Cognitive load and instructionally supported learning with provided and learner-generated visualizations

This study investigated, whether learning from science texts can be enhanced by providing learners with different forms of visualizations (pictures) in addition to text. One-hundred-two 9th and 10th graders read a computer-based text on chemical processes of washing and answered questions on cognitive load (mental effort, perceived difficulty) and comprehension (retention, transfer, drawing). Instruction varied according to a 2 × 2-factorial design with ‘learner-generated pictures’ (yes, no) and ‘provided pictures’ (yes, no) as factors. Results indicate positive main effects of provided pictures on all three comprehension measures and negative main effects on both cognitive load measures. Additional analyses revealed a mediation effect of perceived difficulty on retention and transfer, that is learning with provided pictures decreased cognitive load and enhanced comprehension. Furthermore, results show a positive main effect of learner-generated pictures on drawing and mental effort, but no mediation effect. Taken together, computer-based learning with provided pictures enhances comprehension as it seems to promote active processing while reducing extraneous cognitive processing. Learners, generating pictures, however, seem to have less cognitive resources available for essential and generative processing, resulting in reduced comprehension. These results are in line with cognitive load theory, cognitive theories of multimedia learning, and generative theories of learning.     

The effects of conceptual change texts accompanied with animations on overcoming 11th grade students’ alternative conceptions of chemical bonding

This paper aims to determine the effect of conceptual change texts accompanied with computer animations on 11th grade students’ understanding and alternative conceptions related to chemical bonding. One experimental group (EG; N = 28) and one comparison group (CG; N = 30) were used in the study. While the comparison group taught traditional instruction, the experimental group received conceptual change text accompanied with computer animations instruction. Chemical bonding achievement test was applied as pre-test, post-test and delayed test to collect data. The results of the study indicated that while there is no statistically significant difference between groups in pre-test, performance of EG students is greater than the CG ones in post-test and delayed test. And also, the EG students are better in remediating their alternative conceptions related to chemical bonding. Based on the study, it is concluded that conceptual change texts combined with computer animations can be effective instructional tools to improve students’ conceptual understanding of chemical concepts.     

Inventing motivates and prepares student teachers for computer‐based learning

A brief, problem‐oriented phase such as an inventing activity is one potential instructional method for preparing learners not only cognitively but also motivationally for learning. Student teachers often need to overcome motivational barriers in order to use computer‐based learning opportunities. In a preliminary experiment, we found that student teachers who were given paper‐based course material spent more time on follow‐up coursework than teachers who were given a well‐developed computer‐based learning environment (CBLE), leading to higher learning outcomes. Thus, we tested inventing as an instructional method that may help overcome motivational barriers of teachers' use of computer‐supported tools or learning environments in our main experimental study (N = 44). As a computer‐based environment, we used the ‘Assessment of Learning strategies in Learning journals’. The inventing group produced ideas about criteria to evaluate learning strategies based on student cases prior to the learning phase. The control condition read a text containing possible answers to the inventing problem. The inventing activity enhanced motivation prior to the learning phase and assessment skills as assessed by transfer problems. Hence, the inventing activity prepared student teachers to learn from a CBLE in a motivational as well as cognitive way.     

Using pop-up windows to improve multimedia learning

Abstract The aim of the present study is to evaluate the effects on learning of the spatial integration of textual information incorporated into illustrations in the form of pop‐up windows that are opened by the user. Three groups of students viewed illustrated texts depicting the functioning of the heart and the replication of the AIDS virus either with textual information presented below the picture, with textual information integrated within the picture, or with textual information integrated within the picture with pop‐up windows. The results showed that the integrated formats were more effective than the separated format when we tested the retention of textual information, comprehension and the matching of textual elements to the appropriate illustrated elements. Furthermore, they indicated that the group working with pop‐up windows performed better than the integrated groups on the retention test for illustrated information as well as when asked to find correct solutions to problems in a comprehension test. Consequently, these findings support the use of pop‐up windows in learning with this kind of illustrated explanatory text.     

Instructional effectiveness of a computer-supported program for teaching reading comprehension strategies

This article examines the effectiveness of a computer-based instructional program (e-PELS) aimed at direct instruction in a collection of reading comprehension strategies. In e-PELS, students learn to highlight and outline expository passages based on various types of text structures (such as comparison or cause-and-effect) as well as to paraphrase, self-question, and summarize. The study involved 1041 fourth-grade elementary students from 21 schools distributed in three regions in central Chile. Participant teachers integrated this program into the Spanish language curriculum, instructing their students during thirty sessions of 90 min each during one school semester. Pretest-to-posttest gains in reading comprehension scores were significantly greater for students instructed with this program than for students who received traditional instruction (d = .5), with particularly strong effects for lower-achieving students (d = .7). The findings support the efficacy of direct instruction in specific learning strategies in a computer-based environment.