Showing a model's eye movements in examples does not improve learning of problem-solving tasks

Eye movement modeling examples (EMME) are demonstrations of a computer-based task by a human model (e.g., a teacher), with the model's eye movements superimposed on the task to guide learners' attention. EMME have been shown to enhance learning of perceptual classification tasks; however, it is an open question whether EMME would also improve learning of procedural problem-solving tasks. We investigated this question in two experiments. In Experiment 1 (72 university students, M_age = 19.94), the effectiveness of EMME for learning simple geometry problems was addressed, in which the eye movements cued the underlying principle for calculating an angle. The only significant difference between the EMME and a no eye movement control condition was that participants in the EMME condition required less time for solving the transfer test problems. In Experiment 2 (68 university students, M_age = 21.12), we investigated the effectiveness of EMME for more complex geometry problems. Again, we found no significant effects on performance except for time spent on transfer test problems, although it was now in the opposite direction: participants who had studied EMME took longer to solve those items. These findings suggest that EMME may not be more effective than regular video examples for teaching procedural problem-solving skills.     

Using eye-movement modelling examples to improve critical reading of multiple webpages on a conflicting topic

Interventions to promote students' source evaluations have used various methods designed for the classroom context. In the present study, we tested an approach that is easily adaptable to online courses, based on eye movement modelling examples (EMME), that is, short videos displaying an expert student's eye gaze while s/he reads multiple pages on the Internet to learn about a conflicting topic. Using an eye-tracking methodology in a pre–post design, we analysed how an intervention using EMME changed students' attention to source information, and how this processing affected their learning. EMME increased participants' attention to the search engine results page, author information and decreased attention of texts from untrustworthy pages. In addition, EMME increased the number of participants who cited at least one document source at post-test. We discuss the potential benefits and limitations of EMME in teaching complex literacy strategies, and the importance of measuring processing data in educational research studies.     

Qualitatively different cognitive processing during online reading primed by different study activities

This article uses eye-tracking technology to examine how study activities such as taking notes or filling in a graphic organizer affect cognitive processing during learning. College students read a computer-presented passage that compared the characteristics of eastern steamboats (top section) and western steamboats (bottom section), either by reading it twice (read-only group), typing notes into a textbox on the right side of the screen (note-taking group), or typing characteristics of the two types of steamboats into a compare-and-contrast graphic organizer on the right side of the screen (graphic organizer group). Compared to the note-taking group, the graphic organizer group displayed more eye movements between the top and bottom of the passage (i.e., integrative saccades, d = 1.03), more eye movements between the text and the type-in window on the right side (i.e., constructive saccades, d = 0.79), fewer constructive saccades during initial reading (d = −0.64), and less time looking to the right side during initial reading (d = −0.81); and scored higher on a comprehension test given afterwards (d = 1.17), although both study groups outscored the read-only group. Results suggest that students in the note-taking group (and read-only group) tended to use a linear learning strategy in which their eyes followed the text in the order presented whereas students in the graphic organizer group tended to use a generative learning strategy in which their eyes searched for connections between specific information across the passage required to make comparisons.     

Using multimedia for e‐learning

This paper reviews 12 research‐based principles for how to design computer‐based multimedia instructional materials to promote academic learning, starting with the multimedia principle (yielding a median effect size of d = 1.67 based on five experimental comparisons), which holds that people learn better from computer‐based instruction containing words and graphics rather than words alone. Principles aimed at reducing extraneous processing (i.e., cognitive processing that is unrelated to the instructional objective) include coherence (d = 0.70), signalling (d = 0.46), redundancy (d = 0.87), spatial contiguity (d = 0.79) and temporal contiguity (d = 1.30). Principles for managing essential processing (i.e., mentally representing the essential material) include segmenting (d = 0.70), pre‐training (d = 0.46) and modality (d = 0.72). Principles for fostering generative processing (i.e., cognitive processing aimed at making sense of the material) include personalization (d = 0.79), voice (d = 0.74) and embodiment (d = 0.36). Some principles have boundary conditions, such as being stronger for low‐ rather than high‐knowledge learners.     

A gender matching effect in learning with pedagogical agents in an immersive virtual reality science simulation

The main objective of this study is to determine whether boys and girls learn better when the characteristics of the pedagogical agent are matched to the gender of the learner while learning in immersive virtual reality (VR). Sixty‐six middle school students (33 females) were randomly assigned to learn about laboratory safety with one of two pedagogical agents: Marie or a drone, who we predicted serve as a role models for females and males, respectively. The results indicated that there were significant interactions for the dependent variables of performance during learning, retention, and transfer, with girls performing better with Marie (d = 0.98, d = 0.67, and d = 1.03; for performance, retention, and transfer, respectively) and boys performing better with the drone (d = ?0.41, d = ?0.45, d = ?0.23, respectively). The results suggest that gender‐specific design of pedagogical agents may play an important role in VR learning environments.     

Benefits of adding anxiety-reducing features to a computer-based multimedia lesson on statistics

The present study examined the effectiveness of techniques intended to reduce anxiety as students learn mathematical content from a computer-based lesson. In a between-subjects experiment, students learned statistical rules through worked examples in a computer-based learning environment that either did (treatment group) or did not (control group) include anxiety reducing features—a coping message delivered through the lesson by an online pedagogical agent concerning how to manage feelings of anxiety, and prompts for expressive writing, in which students summarize their thoughts and feelings. An independent samples t-test showed that the treatment group, which received added anxiety-reducing features, showed higher accuracy than the control group on solving practice problems (d = 0.71) and retention problems (d = 0.63) and reported higher perceived effort on learning the multimedia lesson (d = 0.66). In addition, a standard multiple linear regression found that anxiety, self-efficacy, and cognitive load as a set predicted performance (R² = 0.56), with self-efficacy as the strongest predictor (β = 0.63). Adding anxiety-reducing features to an online lesson may encourage greater effort, which leads to better learning outcomes.     

Navigating the Online World: Gaze,Fixations, and Performance Differences between Younger and Older Users

ABSTRACT

The world’s populations and workforces are aging and older adults are now the fastest growing group of internet users. Unfortunately, age-related declines may prevent older adults from performing web-based tasks with the same ease as their younger counterparts. Recent studies have suggested that peripheral navigation may exacerbate age-related online performance gaps as older adults fail to adequately search peripheral elements. Such studies have however relied on the use of preexisting websites with no control for layout, complexity, or prior familiarity. This study tested whether age-related differences in online performance and search behavior exist across various navigational layouts. Participants (N = 47) completed four online tasks with purpose-built websites of equivalent difficulty, length, and content type, yet with varying navigational layouts (center; left-periphery; top-periphery; inconsistent). Participants’ performance (i.e., accuracy/efficiency), search behavior (i.e., eye gaze/fixation), and satisfaction were examined. Identifiable differences between age-groups and navigational layouts were found, with top-peripheral navigation proving most detrimental to the online performance and satisfaction of both younger and older adults. The results inform design principles that aim to increase older adults’ online functionality, and provide an extensive platform for further research.     

Eye tracking as an MT evaluation technique

Eye tracking has been used successfully as a technique for measuring cognitive load in reading, psycholinguistics, writing, language acquisition etc. for some time now. Its application as a technique for measuring the reading ease of MT output has not yet, to our knowledge, been tested. We report here on a preliminary study testing the use and validity of an eye tracking methodology as a means of semi-automatically evaluating machine translation output. 50 French machine translated sentences, 25 rated as excellent and 25 rated as poor in an earlier human evaluation, were selected. Ten native speakers of French were instructed to read the MT sentences for comprehensibility. Their eye gaze data were recorded non-invasively using a Tobii 1750 eye tracker. The average gaze time and fixation count were found to be higher for the “bad” sentences, while average fixation duration and pupil dilations were not found to be substantially different for output rated as good and output rated as bad. Comparisons between HTER scores and eye gaze data were also found to correlate well with gaze time and fixation count, but not with pupil dilation and fixation duration. We conclude that the eye tracking data, in particular gaze time and fixation count, correlate reasonably well with human evaluation of MT output but fixation duration and pupil dilation may be less reliable indicators of reading difficulty for MT output. We also conclude that eye tracking has promise as a semi-automatic MT evaluation technique, which does not require bi-lingual knowledge, and which can potentially tap into the end users’ experience of machine translation output.     

The instructor's gaze guidance in video lectures improves learning

Instructor behaviour is known to affect learning performance, but it is unclear which specific instructor behaviours can optimize learning. We used eye‐tracking technology and questionnaires to test whether the instructor's gaze guidance affected learners' visual attention, social presence, and learning performance, using four video lectures: declarative knowledge with and without the instructor's gaze guidance and procedural knowledge with and without the instructor's gaze guidance. The results showed that the instructor's gaze guidance not only guided learners to allocate more visual attention to corresponding learning content but also increased learners' sense of social presence and learning. Furthermore, the link between the instructor's gaze guidance and better learning was especially strong for participants with a high sense of social connection with the instructor when they learned procedural knowledge. The findings lead to a strong recommendation for educational practitioners: Instructors should provide gaze guidance in video lectures for better learning performance.     

Can video games change attitudes towards history? Results from a laboratory experiment measuring short- and long-term effects

This study investigates a video game's effects on implicit and explicit attitudes towards depicted historical events in the short- and long-term on a sample of 148 young adults. We used, as an intervention tool, a serious game Czechoslovakia 38–89: Borderlands that deals with the expulsion of the Sudeten Germans from the former Czechoslovakia after the WWII. Results showed more negative pretest-posttest explicit attitude changes towards the expulsion on a general level (d = −0.34) and a specific level (d = −0.53) compared to the control group. Over the long-term, group differences in attitude change remained significant for the specific level (d = −0.44), but not for general one (d = −0.16). Exploratory analysis on the item level indicated that especially attitudes towards the expulsion's (un)fairness were affected by the game. However, no significant changes were found in implicit attitudes in the experimental group. This study is the first of such scale to empirically investigate video games' effects on a society's historical awareness.     

Automatic gaze-based user-independent detection of mind wandering during computerized reading

Mind wandering is a ubiquitous phenomenon where attention involuntarily shifts from task-related thoughts to internal task-unrelated thoughts. Mind wandering can have negative effects on performance; hence, intelligent interfaces that detect mind wandering can improve performance by intervening and restoring attention to the current task. We investigated the use of eye gaze and contextual cues to automatically detect mind wandering during reading with a computer interface. Participants were pseudorandomly probed to report mind wandering while an eye tracker recorded their gaze during the reading task. Supervised machine learning techniques detected positive responses to mind wandering probes from eye gaze and context features in a user-independent fashion. Mind wandering was detected with an accuracy of 72 % (expected accuracy by chance was 60 %) when probed at the end of a page and an accuracy of 67 % (chance was 59 %) when probed in the midst of reading a page. Global gaze features (gaze patterns independent of content, such as fixation durations) were more effective than content-specific local gaze features. An analysis of the features revealed diagnostic patterns of eye gaze behavior during mind wandering: (1) certain types of fixations were longer; (2) reading times were longer than expected; (3) more words were skipped; and (4) there was a larger variability in pupil diameter. Finally, the automatically detected mind wandering rate correlated negatively with measures of learning and transfer even after controlling for prior knowledge, thereby providing evidence of predictive validity. Possible improvements to the detector and applications that utilize the detector are discussed.     

Using eye movements to model the sequence of text–picture processing for multimedia comprehension

This study used eye movement modeling examples (EMME) to support students' integrative processing of verbal and graphical information during the reading of an illustrated text. EMME consists of a replay of eye movements of a model superimposed onto the materials that are processed for accomplishing the task. Specifically, the study investigated the effects of modeling the temporal sequence of text and picture processing as shown in various replays of a model's gazes. Eighty‐four 7th graders were randomly assigned to one of the four experimental conditions: text‐first processing sequence (text‐first EMME), picture‐first processing sequence (picture‐first EMME), picture‐last processing sequence (picture‐last EMME) and no‐EMME (control). Online and offline measures were used. Eye movement indices indicate that only readers in the picture‐first EMME condition spent significantly longer processing the picture and showed stronger integrative processing of verbal and graphical information than students in the no‐EMME condition. Moreover, readers in all EMME conditions outperformed those in the control condition for recall. However, for learning and transfer, only readers in the picture‐first EMME condition were significantly superior to readers of the control condition. Furthermore, both the frequency and duration of integrative processing of verbal and graphical information mediated the effect of condition on learning outcomes.     

Visual Attention for Rendered 3D Shapes

Understanding the attentional behavior of the human visual system when visualizing a rendered 3D shape is of great importance for many computer graphics applications. Eye tracking remains the only solution to explore this complex cognitive mechanism. Unfortunately, despite the large number of studies dedicated to images and videos, only a few eye tracking experiments have been conducted using 3D shapes. Thus, potential factors that may influence the human gaze in the specific setting of 3D rendering, are still to be understood. In this work, we conduct two eye‐tracking experiments involving 3D shapes, with both static and time‐varying camera positions. We propose a method for mapping eye fixations (i.e., where humans gaze) onto the 3D shapes with the aim to produce a benchmark of 3D meshes with fixation density maps, which is publicly available. First, the collected data is used to study the influence of shape, camera position, material and illumination on visual attention. We find that material and lighting have a significant influence on attention, as well as the camera path in the case of dynamic scenes. Then, we compare the performance of four representative state‐of‐the‐art mesh saliency models in predicting ground‐truth fixations using two different metrics. We show that, even combined with a center‐bias model, the performance of 3D saliency algorithms remains poor at predicting human fixations. To explain their weaknesses, we provide a qualitative analysis of the main factors that attract human attention. We finally provide a comparison of human‐eye fixations and Schelling points and show that their correlation is weak.     

Exploring Relationships Between Eye Tracking and Traditional Usability Testing Data

This study explored the relationships between eye tracking and traditional usability testing data in the context of analyzing the usability of Algebra Nation?, an online system for learning mathematics used by hundreds of thousands of students. Thirty-five undergraduate students (20 females) completed seven usability tasks in the Algebra Nation? online learning environment. The participants were asked to log in, select an instructor for the instructional video, post a question on the collaborative wall, search for an explanation of a mathematics concept on the wall, find information relating to Karma Points (an incentive for engagement and learning), and watch two instructional videos of varied content difficulty. Participants’ eye movements (fixations and saccades) were simultaneously recorded by an eye tracker. Usability testing software was used to capture all participants’ interactions with the system, task completion time, and task difficulty ratings. Upon finishing the usability tasks, participants completed the System Usability Scale. Important relationships were identified between the eye movement metrics and traditional usability testing metrics such as task difficulty rating and completion time. Eye tracking data were investigated quantitatively using aggregated fixation maps, and qualitative examination was performed on video replay of participants’ fixation behavior. Augmenting the traditional usability testing methods, eye movement analysis provided additional insights regarding revisions to the interface elements associated with these usability tasks.     

Do moods affect programmers’ debug performance?

There is much research that shows people’s mood can affect their activities. This paper argues that this also applies to programmers, especially their debugging. Literature-based framework is presented linking programming with various cognitive activities as well as linking cognitive activities with moods. Further, the effect of mood on debugging was tested in two experiments. In the first experiment, programmers (n = 72) saw short movie clips selected for their ability to provoke specific moods. Afterward, they completed a debugging test. Results showed the video clips had a significant effect on programmers’ debugging performance; especially, there was a significant difference after watching low- and high-arousal-evoking video clips. In the second experiment, programmers’ mood was manipulated by asking participants (n = 19) to dry run algorithms for at least 16 min. They performed some physical exercises before continuing dry running algorithms again. The results showed a significant increase in arousal and valence that coincided with an improvement in programmers’ task performance after the physical exercises. Together, this suggests that programmers’ moods influence some programming tasks such as debugging.     

Investigating text-picture integration in videos with the multimedia contradiction paradigm

In two experiments, the multimedia contradiction paradigm was used to investigate whether learners map information conveyed through the audio and the picture track of a video. In Experiment 1 (N = 85), the information conveyed through the audio track and the picture track was always consistent (control group) or was made inconsistent by changing the audio track at one point in time (text-wrong group). Experiment 2 (N = 143) added a second inconsistent condition by changing the picture track at one point in time (picture-wrong group). In both experiments, the learners' gaze behaviour differed from that of the control group when inconsistent information was presented in the experimental groups. This indicates that mapping processes—which are an indispensable part of integration—occur when learners process videos. Regarding learning outcomes, no differences between groups were observed. In addition, only a few learners remembered the conflict after learning. Further, recall shifted towards the pictorial information when learners encountered conflict.     

Effectiveness of virtual reality-based instruction on students' learning outcomes in K-12 and higher education: A meta-analysis

The purpose of this meta-analysis is to examine overall effect as well as the impact of selected instructional design principles in the context of virtual reality technology-based instruction (i.e. games, simulation, virtual worlds) in K-12 or higher education settings. A total of 13 studies (N = 3081) in the category of games, 29 studies (N = 2553) in the category of games, and 27 studies (N = 2798) in the category of virtual worlds were meta-analyzed. The key inclusion criteria were that the study came from K-12 or higher education settings, used experimental or quasi-experimental research designs, and used a learning outcome measure to evaluate the effects of the virtual reality-based instruction.Results suggest games (FEM = 0.77; REM = 0.51), simulations (FEM = 0.38; REM = 0.41), and virtual worlds (FEM = 0.36; REM = 0.41) were effective in improving learning outcome gains. The homogeneity analysis of the effect sizes was statistically significant, indicating that the studies were different from each other. Therefore, we conducted moderator analysis using 13 variables used to code the studies. Key findings included that: games show higher learning gains than simulations and virtual worlds. For simulation studies, elaborate explanation type feedback is more suitable for declarative tasks whereas knowledge of correct response is more appropriate for procedural tasks. Students performance is enhanced when they conduct the game play individually than in a group. In addition, we found an inverse relationship between number of treatment sessions learning gains for games.With regards to the virtual world, we found that if students were repeatedly measured it deteriorates their learning outcome gains. We discuss results to highlight the importance of considering instructional design principles when designing virtual reality-based instruction.     

Impact of mainstream classroom setting on attention of children with autism spectrum disorder: an eye-tracking study

It has long been reported that children with autism spectrum disorder (ASD) exhibit attention difficulties while learning. They tend to focus on irrelevant information and can easily be distracted. As a result, they are often confined to a one-to-one teaching environment, with fewer distractions and social interactions than would be present in a mainstream educational setting. In recent years, inclusive mainstream schools have been growing in popularity due to government policies on equality rights. Therefore, it is crucial to investigate attentional patterns of children with ASD in mainstream schools. This study aims to explore the attentional behaviors of children with ASD in a virtual reality simulated classroom. We analyzed four eye-gaze behaviors and performance scores of 45 children: children with ASD (ASD n = 20) and typically developing children (TD n = 25) when performing attention tasks. The gaze behaviors included time to first fixate (TTFF), first fixation duration (FFD), average fixation duration (AFD) and the sum of fixation count (SFC) on fourteen areas of interest (AOIs) in the classroom. Our results showed that children with ASD exhibit similar gaze behaviors to TD children, but with significantly lower performance scores and SFC on the target AOI. These findings showed that classroom settings can influence attentional patterns and the academic performance of children with ASD. Further studies are needed on different modalities for supporting the attention of children with ASD in a mainstream setting.

     

Task and user effects on reading patterns in information search

We report on an investigation into people’s behaviors on information search tasks, specifically the relation between eye movement patterns and task characteristics. We conducted two independent user studies (n = 32 and n = 40), one with journalism tasks and the other with genomics tasks. The tasks were constructed to represent information needs of these two different users groups and to vary in several dimensions according to a task classification scheme. For each participant we classified eye gaze data to construct models of their reading patterns. The reading models were analyzed with respect to the effect of task types and Web page types on reading eye movement patterns. We report on relationships between tasks and individual reading behaviors at the task and page level. Specifically we show that transitions between scanning and reading behavior in eye movement patterns and the amount of text processed may be an implicit indicator of the current task type facets. This may be useful in building user and task models that can be useful in personalization of information systems and so address design demands driven by increasingly complex user actions with information systems. One of the contributions of this research is a new methodology to model information search behavior and investigate information acquisition and cognitive processing in interactive information tasks.     

The impact of an automated learning component against a traditional lecturing environment

This experimental study investigates the effect on the examination performance of a cohort of first‐year undergraduate learners undertaking a Unified Modelling Language (UML) course using an adaptive learning system against a control group of learners undertaking the same UML course through a traditional lecturing environment. The adaptive learning system uses two components for the creation of suitable content for individual learners: a content analyser that automatically generates metadata describing cognitive resources within instructional content and a selection model that utilizes a genetic algorithm to select and construct a course suited to the cognitive ability and pedagogic preference of an individual learner, defined by a digital profile. Using the Kruskal–Wallis H test, it was determined that there was a statistically significant difference between the control group of learners and the learners that participated in the UML course using the adaptive learning system following an examination once the UML course concluded, with p = 0.005, scoring on average 15.71% higher using the adaptive system. However, this observed statistically significant difference observed a small effect size of 20%.