Rethinking the evaluation of algorithm animations as learning aids: an observational study

One important aspect of creating computer programs is having a sound understanding of the underlying algorithms used by programs. Learning about algorithms, just like learning to program, is difficult, however. A number of prior studies have found that using animation to help teach algorithms had less beneficial effects on learning than hoped. Those results surprise many computer science instructors whose intuition leads them to believe that algorithm animations should assist instruction. This article reports on a study in which animation is utilized in more of a “homework” learning scenario rather than a “final exam” scenario. Our focus is on understanding how learners will utilize animation and other instructional materials in trying to understand a new algorithm, and on gaining insight into how animations can fit into successful learning strategies. The study indicates that students use sophisticated combinations of instructional materials in learning scenarios. In particular, the presence of algorithm animations seems to make a complicated algorithm more accessible and less intimidating, thus leading to enhanced student interaction with the materials and facilitating learning.     

Top-down and bottom-up influences on learning from animations

To evaluate how top-down and bottom-up processes contribute to learning from animated displays, we conducted four experiments that varied either in the design of animations or the prior knowledge of the learners. Experiments 1–3 examined whether adding interactivity and signaling to an animation benefits learners in developing a mental model of a mechanical system. Although learners utilized interactive controls and signaling devices, their comprehension of the system was no better than that of learners who saw animations without these design features. Furthermore, the majority of participants developed a mental model of the system that was incorrect and inconsistent with information displayed in the animation. Experiment 4 tested effects of domain knowledge and found, surprisingly, that even some learners with high domain knowledge initially constructed the incorrect mental model. After multiple exposures to the materials, the high knowledge learners revised their mental models to the correct one, while the low-knowledge learners maintained their erroneous models. These results suggest that learning from animations involves a complex interplay between top-down and bottom-up processes and that more emphasis should be placed on how prior knowledge is applied to interpreting animations.     

Evaluating animations as student aids in learning computer algorithms

We conducted two experiments designed to examine whether animations of algorithms would help students learn the algorithms more effectively. Across the two studies we used two different algorithms — depth-first search and binomial heaps — and used two different subject populations — students with little or no computer science background and students who were computer science majors — and examined whether animations helped students acquire procedural and conceptual knowledge about the algorithms. The results suggest that one way animations may aid learning of procedural knowledge is by encouraging learners to predict the algorithm's behavior. However, such a learning improvement was also found when learners made predictions of an algorithm's behavior from static diagrams. This suggests that prediction, rather than animation per se, may have been the key factor in aiding learning in the present studies. These initial experiments served to highlight a number of methodological issues that need to be systematically addressed in future experiments in order to fully test the relationship between animation and prediction as well as to examine other possible benefits of animations on learning.     

Learning from instructional animations: How does prior knowledge mediate the effect of visual cues?

The purpose of this study was to investigate the effects of cueing and prior knowledge on learning and mental effort of students studying an animation with narration. This study employed a 2 (no cueing vs. visual cueing) × 2 (low vs. high prior knowledge) between‐subjects factorial design. The results revealed a significant interaction effect between prior knowledge and cueing on learning. Low prior knowledge learners had higher scores after studying an instructional animation with visual cues, compared to those who studied the same instructional animation without visual cues. Conversely, when cues were not provided, high prior knowledge learners outperformed those high prior knowledge learners who studied with the cued version of an instructional animation. These results indicated that the effects of cueing in an instructional animation change depending on the learners' level of prior knowledge. Specifically, low prior knowledge learners benefited more when visual cues were provided, whereas cues did not facilitate learning for high prior knowledge learners.     

Learning by generating vs. receiving instructional explanations: Two approaches to enhance attention cueing in animations

This study investigated whether learners construct more accurate mental representations from animations when instructional explanations are provided via narration than when learners attempt to infer functional relations from the animation through self-explaining. Also effects of attention guidance by means of cueing are investigated. Psychology students were given retention, inference, and transfer tests after studying a cued or an uncued animation of the cardiovascular system with learner-generated self-explanations or with externally provided instructional explanations. Results indicated that cued animations were more effective than uncued animations. Furthermore, results on retention and transfer indicated no differences between self-explaining and providing instructional explanations, but instructional explanations accompanying animations led to higher inference scores. It is concluded that whether explanations are generated or presented may be less important than the provision of cues that enable focused processing of presented or produced explanations.     

Effects of spatial ability and richness of motion cue on learning in mechanically complex domain

This study seeks to examine the impact of individual differences in the spatial ability of learners to integrate verbal information and three modes of visual representations. Several hypotheses were tested, including that (1) individual difference in spatial ability should influence the learning of theoretical knowledge when the instructional materials present a static visual representation at the lowest motion cue richness, and (2) both animations and the static visual representation containing motion cues should be more effective than static visual representation, especially for learners with low spatial ability. In the experiment, 60 learners were classified as having either low or high spatial ability on the basis of their performance on the Kit of Factor Referenced Cognitive Tests. The learners got knowledge from written explanations describing a four-stroke engine mechanism in a computer-based format. Also, written explanations were reinforced by corresponding visual representations with three levels of motion cue richness (static images, static images with motion cues or animations). Understanding was measured by a problem-solving transfer test. The results indicate that (1) presenting written explanations with corresponding animations did not improve performance of the learners with high spatial ability, (2) for the learners with low spatial ability, learning was enhanced by the use of animations, (3) merely adding motion cues to the static visual representation did not improve learning of the learners with low spatial ability, and (4) use of animations did not help learners with low spatial ability more than those with high spatial ability.     

Special Issue in Honor of Ben Shneiderman's 60th Birthday: Reflections on Human-Computer Interaction

Abstract

Unlike most Web portals in the world, Chinese Web portals are characterized by a huge amount of information, excessive visual stimuli, and very long Web pages. The objective of this study is to investigate the effect of such rich Web portal designs and floating animations on visual search, emphasizing a comparison between Chinese users and German users. Two experiments were conducted to test 2 proposed hypotheses. Experiment 1 studied the effect of Web portal design (rich and simple) on visual search performance (performance time, errors, and satisfaction) with both Chinese and German participants. Experiment 2 studied the effects of static animations (leaderboards, couplets, and large squares) and floating animations (moving down, moving up/down, and random movement) on visual search performance on Web portals. The dependent variables were the performance time, error, satisfaction, and animation recognition. The results indicated that participants using simple Web portals searched faster, made fewer errors, and were more satisfied than participants using rich Web portals. No significant differences were found between the performance time of Chinese participants and German participants. However, satisfaction of Chinese participants was found to be less influenced by the differences between simple and rich Web portal designs, compared with German participants. No significant differences were found in performance time and animation recognition between static animations and floating animations, which indicated that users are able to detect the pattern of animation movements and were able to avoid floating animations as well as static animations. People searching pages with randomly floating animations were found to use significantly more time compared with those searching pages with no animations. Furthermore, users' satisfaction for pages with randomly floating animations, moving down animations, and moving up/down animations was significantly lower than for pages with no animations. Implications for designers and for future research are discussed.     

Principles for using animation in computer-based instruction: theoretical heuristics for effective design

As tools for multimedia and computer-based instruction (CBI) increase in sophistication, it becomes easier for instructional designers to incorporate a range of animations in instructional software. Designers, however, should ask whether animation has the potential to contribute to student learning before investing the resources in development. This paper addresses the viability of using animations in multimedia and CBI. The functions of animations are explored as well as issues related to surface structure and fidelity. The relationship between content structures and the use of animation in CBI is also discussed. Based on these characteristics and purposes, heuristics are provided to guide the use of animation in CBI. Implications of these heuristics are explored and suggestions are provided for future research.     

The role of spatial ability in learning from instructional animations - Evidence for an ability-as-compensator hypothesis

In two experiments, the role of spatial ability in learning from an instructional animation versus a series of static pictures was studied. In both experiments, a statistical interaction of spatial ability and type of visualization was obtained: Low-spatial ability students showed poor learning outcome when learning from pictures while high-spatial students did not; when learning from animation, however, learning outcome was independent from spatial ability. The results are in line with an ability-as-compensator hypothesis which states that constructing mental animations from non-dynamic materials needs spatial ability; with animated learning materials, however, spatial ability is not required. No overall differences between static pictures and animation were found.     

Rotational perspective and learning procedural tasks from dynamic media

There have been conflicting accounts regarding the effectiveness of animations for learning. Procedural motor learning represents one of the few areas in which animations have consistently shown to be facilitative. Some have suggested that this benefit is related to activation of the mirror neuron system (MNS), with higher activation leading to better performance. This study examines this explanation, and observed the effects of instructional media (animation vs. static), as a function of viewing perspective (face-to-face vs. over-the-shoulder) on understanding a procedural motor task (knot tying). Results indicate that performance was significantly improved with animations over static images, however this appeared to be most pronounced in situations which matched the learners' own perspective (i.e., over-the-shoulder). These findings have implications for the design of instructional media for procedural motor tasks and provide confirmation of the assertion that appropriate activation of the perceptual system can be leveraged to increase performance.     

Should hand actions be observed when learning hand motor skills from instructional animations?

This study investigated whether the effectiveness of learning a hand-motor task through an instructional animation required observation of the hands or not. Cognitive load theory was used to predict that both animated conditions (with and without hands) would be equally effective, and that both animations would be superior to an equivalent static graphics presentation. 36 adults were randomly assigned to three groups (With-hands animation, No-hands animation, Statics graphics) and were required to learn how to tie two knots. Test results confirmed that both animations led to superior learning compared to the static presentation. However, the With-hands animation strategy had a further advantage in that it had higher instructional efficiency than the No-hands animation.     

Explaining the segmentation effect in learning from animations: The role of pausing and temporal cueing

Segmentation of animations, that is presenting them in pieces rather than as a continuous stream of information, has been shown to have a beneficial effect on cognitive load and learning for novices. Two different explanations of this segmentation effect have been proposed. Firstly, pauses are usually inserted between the segments, which may give learners extra time to perform necessary cognitive processes. Secondly, because segmentation divides animations into meaningful pieces, it provides a form of temporal cueing which may support learners in perceiving the underlying structure of the process or procedure depicted in the animation. This study investigates which of these explanations is the most plausible. Secondary education students (N = 161) studied animations on probability calculation, after having been randomly assigned to one of four conditions: non-segmented animations, animations segmented by pauses only, animations segmented by temporarily darkening the screen only, and animations segmented by both pauses and temporarily darkening the screen. The results suggest that both pauses and cues play a role in the segmentation effect, but in a different way.     

An exploratory study using inexpensive electroencephalography (EEG) to understand flow experience in computer-based instruction

Flow is an optimal experience that results in intense engagement in an activity. In computer-based instructional environment, flow can be used to examine learning performance. We used questionnaire survey and electroencephalography (EEG) analysis to examine the influence of challenge-skill balance on the flow experience and influence of flow experience on learning performance in a computer-based instructional environment. The results showed that the flow experience of learners depends on challenge-skill balance of learning materials. The research explored the possibility of using an inexpensive non-medical EEG device to research the association between flow experience and challenge-skill balance in educational information systems.     

Exploring Exploring a Word Processor

Studies of people learning to use contemporary word-processing equipment suggest that effective learning is often "active," proceeding by self-initiated problem solving. The instructional manuals that accompany current word-processing systems often penalize and impede active learning. A set of instructional materials was constructed for a commercial word processor, specifically designed to support and encourage an active learning orientation. These "guided exploration (GE) materials are modular, task oriented, procedurally incomplete, and address error recognition and recovery. Learners using the GE materials spent substantially less time yet still performed better on a transfer of learning posttest than learners using commercially developed self-study materials. Qualitative analysis of aspects of the learning protocols of participants suggested that active learning mechanisms may underlie this advantage.     

The effect of multimedia design types on learners’ recall performances with varying short term memory spans

In this study, the effect of multimedia learning environment designed with two different attention types (focused — split) was investigated on recall performances of learners with different short term memory spans (high — medium — low). The participants were 60 undergraduate students who were presented with either focused attention or split attention multimedia learning materials. First, participants’ short term memory spans were determined by Visual — Aural Digit Span Test-Revised (VADS-B) test. Second, they were separated to three groups as high, medium and low. In 3?×?2 nested ANOVA design, one of the groups studied the multimedia designed in split attention type whereas the other had focused attention type design. As they finished the study task, they were given a recall task, which produced their recall performances. Data were analyzed by Nested ANOVA, t-Test and ANCOVA tests. The findings indicated that multimedia instructional designs were effective on recall performances. Learners showed higher recall performances in the multimedia learning environment in focused attention design. However, no significant difference was observed in learners’ recall performances when their STM spans were taken into account. Significant differences were observed between time spent in studying multimedia.     

Trialing cartoons: Teachers’ attitudes towards animation as an ELT instructional tool

This paper explores the attitudes of teachers, as adult learners, towards learning to do animation. A part of popular culture which second-language students enjoy, until recently, animation has been technically too demanding for non-specialists to learn. Adult learners can experience e-learning as transformative, but also as a barrier. Thus, teacher reception is crucial in exploring the feasibility of animation as an instructional tool in language teaching. In all, 44 Hong Kong and mainland Chinese teachers were taught animation over ten weeks. Subsequently, three surveys elicited both quantitative and qualitative data. Appraisal analysis indicated teachers positively realised animation as valuable, worthwhile and satisfactory, but also difficult and time-consuming, and entailed high levels of communication. Quantitative data indicated their view that animation would be well-received by both colleagues and secondary language learners, as an instructional tool.     

Effects of interactivity and instructional scaffolding on learning: Self-regulation in online video-based environments

Online learning often requires learners to be self-directed and engaged. The present study examined students' self-regulatory behaviors in online video-based learning environments. Using an experimental design, this study investigated the effects of a newly designed enhanced video learning environment, which was designed to support or scaffold students' self-regulated or self-directed learning on students' learning behaviors and outcomes. In addition, correspondence between students' self-regulation strategies in traditional learning environments and observed self-regulated learning behaviors in the enhanced video environment were examined. A cross-sectional experimental research design with systematic random assignment of participants to either the control condition (common video) or the experimental condition (enhanced video) was utilized. Undergraduate and graduate students participated in the study (N = 80). Study results indicate that the newly designed enhanced video learning environment was a superior instructional tool than the common video learning environment in terms students' learning performance. In addition, there was correspondence between graduate students' self-reported self-regulation and observed self-regulation, with those high on seeking/learning information and managing their environment/behavior more likely to engage more in interactive note-taking.     

The attention-guiding effect and cognitive load in the comprehension of animations

To be effective, instructional animations should avoid causing high extraneous cognitive load imposed by the high attentional requirements of selecting and processing relevant elements. In accordance with the attention-guiding principle (Bétrancourt, 2005), a study was carried out concerning the impact of cueing on cognitive load and comprehension of animations which depicted a dynamic process in a neurobiology domain. Cueing consisted of zooming in important information at each step of the process. Thirty-six undergraduate psychology students were exposed to an animation three times. Half of the participants received an animation without cueing while the other half received the same animation with cueing. Measures of cognitive load and comprehension performance (questions on isolated elements and on high-element interactivity material) were administered twice, after one and three exposures to the animation. The analyses revealed two main results. First, extraneous cognitive load was reduced by cueing after three exposures. Second, retention of the isolated elements was improved in both animation groups, whereas comprehension of high-element interactive material (i.e., the causal relations between elements) increased only in the cueing condition. Furthermore, a problem solving task showed that cueing supported the development of a more elaborate mental model.