The impact of general and specific performance and self-efficacy on learning with computer-based concept mapping

When students learn a new and challenging task, for which they have very limited corresponding personal or vicarious experiences to refer, it is important to understand how their self-efficacy beliefs evolve during the course of sequential lessons; how they differ in the way their general learning performance and self-efficacy influence their ratings of task-specific self-efficacy and performance; and, how such differences may result in different learning outcomes and motivations for learning. By examining a group of 66 students engaged in learning to construct good concept maps with computer software, this study revealed several important findings on these questions. Students generally tended to initially overestimate their ability to successfully carry out the required task. They also varied in the way their general self-efficacy and performance were associated with their task-specific self-efficacy and performance. And, these differences did appear to be associated with different learning outcomes and motivations for learning. These results indicate that investigating individual differences in students’ patterns of association between general and specific performance and their self-efficacy may lead to a better understanding of how students differ in their levels of motivation and outcomes when learning a new and challenging task.     

Effects of computer-based educational achievement test on test performance and test takers’ motivation

There has been an increasing interest in recent years in developing and using computer-based tests in educational assessment. To replace paper-based tests with computer-based ones, the standards for developing computerized-assessment (International Test Commission., 2004) requires equivalent test scores to be established for the new computer-based test and the conventional paper-based test. However, in most test mode comparability studies, the actual test items used have been identical, and yet significant differences have been found in test scores in paper-based and computer-based modes. This has been reported for several subjects, including science, languages and mathematics. The validity of using computer-based tests in educational assessment must therefore be questioned. This study involves a biology test and a biology motivation questionnaire using a Solomon four-group experimental design to examine the validity of the computer-based test and its effects on test performance and the motivation of test-takers. The findings provide supportive evidence for the validity of computer-based test in educational assessment.     

Examining the designs of computer-based assessment and its impact on student engagement,satisfaction, and pass rates

Many researchers who study the impact of computer-based assessment (CBA) focus on the affordances or complexities of CBA approaches in comparison to traditional assessment methods. This study examines how CBA approaches were configured within and between modules, and the impact of assessment design on students' engagement, satisfaction, and pass rates. The analysis was conducted using a combination of longitudinal visualisations, correlational analysis, and fixed-effect models on 74 undergraduate modules and their 72,377 students. Our findings indicate that educators designed very different assessment strategies, which significantly influenced student engagement as measured by time spent in the virtual learning environment (VLE). Weekly analyses indicated that assessment activities were balanced with other learning activities, which suggests that educators tended to aim for a consistent workload when designing assessment strategies. Since most of the assessments were computer-based, students spent more time on the VLE during assessment weeks. By controlling for heterogeneity within and between modules, learning design could explain up to 69% of the variability in students' time spent on the VLE. Furthermore, assessment activities were significantly related to pass rates, but no clear relation with satisfaction was found. Our findings highlight the importance of CBA and learning design to how students learn online.     

The impact of computer-based procedures on team performance,communication, and situation awareness

In nuclear power plants, operating procedures are adopted to aid the operators in performing their tasks. With the evolution of computer hardware and software, the analog human-machine systems of the main control room in the nuclear power plant have been replaced with digital systems. Moreover, automated operations now outnumber manual operations. The purpose of this study was to evaluate the impact of computer-based procedures on the team performance, communication, and situation awareness of operators in the main control room. To achieve the purpose, a within-subjects experiment was designed and then a survey was conducted. The results showed that the teams had better objective performance and higher situation awareness when using computer-based procedures. In addition, the teams also had lower human error and lower communication rates when using computer-based procedures. This study proposes that computer-based procedures are advantageous to the operation of the systems of the main control rooms in nuclear power plants.Relevance to industryComputerized procedures system is one of the specific features for advanced nuclear power plant. This study explores and analyzes the team performance, communication, and situation awareness difference between paper-based, electronic, and computer-based procedures in detail. It may provide practical information for how to apply computer-based procedures to perform the tasks in the main control room of the advanced nuclear power plant.     

Attention paid to feedback provided by a computer-based assessment for learning on information literacy

Three studies are presented on attention paid to feedback provided by a computer-based assessment for learning on information literacy. Results show that the attention paid to feedback varies greatly. In general the attention focuses on feedback of incorrectly answered questions. In each study approximately fifty percent of the respondents paid attention to feedback of incorrect answers only. Approximately another twenty-five percent did not pay attention to feedback at all. Results suggest that differences in attention paid to feedback are influenced by task difficulty and test length. Supervision, however, does not seem to influence the average attention paid to feedback. On the other hand, results show that indirect and direct supervision lead to a greater impact of feedback provided by a computer-based assessment for learning as the number increases of students taking the test and, as a consequence, paying attention to feedback.     

Effects of computer-based testing on test performance and testing motivation

Computer-based testing (CBT) is a green computing strategies used to reduce paper consumption. However, some scholars have questioned the effectiveness of CBT and suggested for conducting systematic studies on CBT to carefully check its reliability and validity before opting for it. Recently, some studies have evaluated the effectiveness of CBT by comparing it with paper–pencil testing (PPT), and the findings were inconsistent. Besides this, most of these studies have been conducted using quasi or basic experimental designs without identifying testing effects on test takers. The limitation of these design is testing effects might occur when a participant in the control or treatment group is tested at least twice on a same test, and it influences the outcomes of taking a posttest, therefore, the findings might be misinterpreted. This study employed a Solomon four-group experimental design (2 × 3 factorial design) on a group of student teachers (n = 140) to compare CBT and PPT on test performance (test scores), testing time and testing motivation. Results indicated that the CBT mode is more reliable in terms of internal and external validity. The CBT significantly reduced testing time and developed stronger self-efficacy, intrinsic and social testing motivation in the participants.     

The comparative effect of individually-constructed vs. collaboratively-constructed computer-based concept maps

The researchers investigated the comparative effects of individually-constructed and collaboratively-constructed computer-based concept mapping on middle school science concept learning. One hundred and sixty one students completed the entire study. Using prior science performance scores to assure equivalence of student achievement across groups, students were assigned to three groups: a self-selected study strategy group, an individual-concept mapping group, and a collaborative pairs – concept mapping group. Collaboratively and individually-constructing computer-based concept maps had equally positive effects on seventh grade middle school science concept learning as measured on a comprehension test. However, the students who collaboratively constructed concept maps created significantly higher quality concept maps than those who individually constructed concept maps indicating deeper conceptual understanding.     

Benefits of adding anxiety-reducing features to a computer-based multimedia lesson on statistics

The present study examined the effectiveness of techniques intended to reduce anxiety as students learn mathematical content from a computer-based lesson. In a between-subjects experiment, students learned statistical rules through worked examples in a computer-based learning environment that either did (treatment group) or did not (control group) include anxiety reducing features—a coping message delivered through the lesson by an online pedagogical agent concerning how to manage feelings of anxiety, and prompts for expressive writing, in which students summarize their thoughts and feelings. An independent samples t-test showed that the treatment group, which received added anxiety-reducing features, showed higher accuracy than the control group on solving practice problems (d = 0.71) and retention problems (d = 0.63) and reported higher perceived effort on learning the multimedia lesson (d = 0.66). In addition, a standard multiple linear regression found that anxiety, self-efficacy, and cognitive load as a set predicted performance (R² = 0.56), with self-efficacy as the strongest predictor (β = 0.63). Adding anxiety-reducing features to an online lesson may encourage greater effort, which leads to better learning outcomes.     

The effects of social messaging on students' learning and intrinsic motivation in peer assessment

This study aims to gain a better understanding of how the newly arisen social messaging may impact the practice of peer assessment. Seventy-nine ESL (English as second language) students reviewed each other's English essays in three peer assessment groups: a three-member group using wiki (wiki group), a three-member group using social messaging (small messaging group), and a six-member group using social messaging (big messaging group). Data analysis suggested that peer assessment facilitated by social messaging can be at least of the same effectiveness as wiki-facilitated peer assessment on ESL students' writing skills and intrinsic motivation. In addition, the findings indicated that students in the small messaging group outperformed students in the big messaging group on essay writing and reported a significantly higher rating on perceived competence, a positive indicator of the behavioural measures of intrinsic motivation, than students in the big messaging group.     

The goal specificity effect on strategy use and instructional efficiency during computer-based scientific discovery learning

Using a computer-based scientific discovery learning environment on buoyancy in fluids we investigated the effects of goal specificity (nonspecific goals vs. specific goals) for two goal types (problem solving goals vs. learning goals) on strategy use and instructional efficiency. Our empirical findings close an important research gap, because in earlier studies the goal specificity effect either was restricted to one goal type or goal type was confounded with goal specificity. In addition, there is hardly a study with empirical evidence for the goal specificity effect on strategy use, which counts even more for a cognitive cost-benefit ratio as a dependent variable. Instead, in earlier studies the goal specificity effect has been attributed to differences in strategy use and cognitive cost-benefit ratio in a rather theoretical way. In the present study for strategy use an interaction was found between goal specificity and goal type, indicating that the goal specificity effect occurs only in case of problem solving goals, but not in case of learning goals. Compared to students provided with specific problem solving goals, students who worked on nonspecific problem solving goals, used a control of variables-strategy more frequently. Additionally, we found a main effect of goal specificity on instructional efficiency for both of the goal types, pointing at a more favorable relationship between performance gain and cognitive load caused by nonspecific goals.     

The impact of goal specificity and goal type on learning outcome and cognitive load

Two hundred and thirty three 15-year old students conducted experiments within a computer-based learning environment. They were provided with different goals according to an experimental 2 × 2 design with goal specificity (nonspecific goals versus specific goals) and goal type (problem solving goals versus learning goals) as factors. We replicated the findings of other researchers that nonspecific problem solving goals lead to lower cognitive load and better learning than specific problem solving goals. For learning goals, however, we observed this goal specificity effect only on cognitive load but not on learning outcome. Results indicate that the goal specificity affects the element interactivity of a task and cognitive load with both, problem solving goals or learning goals. But differences in overall cognitive load are not sufficient for explaining differences in learning outcome. Additionally, differences in strategy use come into play. Specific problem solving goals seem to restrict students to use a problem solving strategy whereas nonspecific problem solving goals or learning goals allow students to use a learning strategy. We conclude that in order to foster learning, students must be provided with goals that allow them to use a learning strategy. Additionally, providing them with nonspecific goals decreases cognitive load and, thus, enables students to learn with less effort.     

A comparative analysis of the consistency and difference among teacher-assessment, student self-assessment and peer-assessment in a Web-based portfolio assessment environment for high school students

This study explored the consistency and difference of teacher-, student self- and peer-assessment in the context of Web-based portfolio assessment. Participants were 72 senior high school students enrolled in a computer application course. Through the assessment system, the students performed portfolio creation, inspection, self- and peer-assessment; three teachers reviewed portfolios and evaluated learning performances. There were significant differences in the results of the three assessment methods, among which teacher-raters adopted the most rigorous scoring standards, while peer-raters tended to use the most lax standards. The results of self- and teacher-assessment were discovered to be consistent; however, consistency was not discovered between self- and peer-assessment as well as peer- and teacher-assessment. In analyzing their consistency with the end-of-course examination, teacher- and self-assessment demonstrated high consistency, whereas peer-assessment showed a low level of consistency.     

Instructional efficiency of integrated and separated text with animated presentations in computer-based science instruction

This study deals with the instructional efficiency of integrating text and animation into computer-based science instruction. The participants were 84 seventh-grade students in a private primary school in Istanbul. The efficiency of instruction was measured by mental effort and performance level of the learners. The results of the study showed that processing integrated text and animation format in computer-based science instruction requires less mental effort than the separated format, and that the performance of the students in the group with integrated presentation format group is higher than that of students in the group with separated presentation format. Instructional efficiency of the integrated presentation group was found to be higher than that of the separated presentation group. Finally, there was no difference between the two groups for instructional time.     

The effects of augmented reality on students' academic achievement and motivation in a biology course

This study aimed to investigate the effects of augmented reality (AR) activities on students' academic achievement and motivation in a biology course. For this purpose, a mixed study was conducted, and a pretest and posttest control group model was used. In addition, the opinions of the experimental group students and the teacher about the AR activities were taken, and classroom observations were made during the study. The study group consisted of 40 (22 female and 18 male) ninth‐grade biology course students. The control group followed the biology course programme, whereas the experimental group students conducted AR activities in addition to the course programme using tablets. Consequently, it was found that the motivation of the students in the experimental group increased more than that of the students in the control group. However, no significant difference was found between the academic achievement scores of the groups. The teacher and the students stated that AR activities might be effective in increasing course success and motivation.     

Understanding students' performance in a computer-based assessment of complex problem solving: An analysis of behavioral data from computer-generated log files

Computer-based assessments of complex problem solving (CPS) that have been used in international large-scale surveys require students to engage in an in-depth interaction with the problem environment. In this, they evoke manifest sequences of overt behavior that are stored in computer-generated log files. In the present study, we explored the relation between several overt behaviors, which N = 1476 Finnish ninth-grade students (mean age = 15.23, SD = .47 years) exhibited when exploring a CPS environment, and their CPS performance. We used the MicroDYN approach to measure CPS and inspected students' behaviors through log-file analyses. Results indicated that students who occasionally observed the problem environment in a noninterfering way in addition to actively exploring it (noninterfering observation) showed better CPS performance, whereas students who showed a high frequency of (potentially unplanned) interventions (intervention frequency) exhibited worse CPS performance. Additionally, both too much and too little time spent on a CPS task (time on task) was associated with poor CPS performance. The observed effects held after controlling for students' use of an exploration strategy that required a sequence of multiple interventions (VOTAT strategy) indicating that these behaviors exhibited incremental effects on CPS performance beyond the use of VOTAT.     

The effects of modern mathematics computer games on mathematics achievement and class motivation

This study examined the effects of a computer game on students' mathematics achievement and motivation, and the role of prior mathematics knowledge, computer skill, and English language skill on their achievement and motivation as they played the game. A total of 193 students and 10 teachers participated in this study. The teachers were randomly assigned to experimental and control groups. A mixed method of quantitative and interviews were used with Multivariate Analysis of Co-Variance to analyze the data.     

Toward a mathematical model of motivation,volition, and performance

The goal of this study was to (1) empirically examine factors that affect human performance in a simulation-based learning environment, employing the framework of the integrative theory of Motivation, Volition, and Performance (MVP) (Keller, 2008a) and (2) develop and statistically evaluate a mathematical MVP model that can be applied to other digital learning environments. The development of a mathematical MVP model can provide empirical support for the elements included in the MVP theory and serve as a tool for designing effective digital learning environments. A regression analysis of motivational, volitional, and performance data of 62 graduate students that interacted with an online simulation revealed a significant model that explained approximately 70% of the variation in student satisfaction through motivational and volitional processing variables suggested by the MVP theory. Students' interest and curiosity toward the learning environment had the highest positive predicting power on students' satisfaction, while the volition processing variable had the lowest predicting power. Implications for the digital learning environments design and directions for future research are discussed.     

The effects of individually personalized computer-based instructional program on solving mathematics problems

This study investigated the effects of an individually personalized computer-based instructional program on the achievement and attitudes regarding mathematics computational problems and word problems of 104 middle school American students. Students were blocked by math entering knowledge based on pre-test scores, then randomly assigned to a personalized or non-personalized version of the computer-based instructional program. A significant two-way interaction (treatment by math entering knowledge) reflected that personalized higher-level math entering knowledge students and non-personalized higher-level math entering knowledge students had similar post-test scores but personalized lower-level math entering knowledge students scored significantly higher on the post-test than non-personalized lower-level math entering knowledge students. Another significant two-way interaction (math entering knowledge by problem type) reflected that students with higher-level math entering knowledge scored considerably higher on the computational problems than on the word problems while students with lower-level math entering knowledge scored significantly higher on the computational problems than on the word problems. Student attitudes were significantly more favorable toward the personalized computer-based instructional program.     

Structuring asynchronous discussion groups: the impact of role assignment and self-assessment on students' levels of knowledge construction through social negotiation

This article examines the impact of the introduction of roles and the added value of self-assessment on students' level of knowledge construction in online asynchronous discussions in a first-year university course in instructional sciences. Students' postings in 20 discussion groups were used as the research data for this study. All messages, submitted during the 12-week discussion period and comprising four discussion themes of 3 weeks each, were analysed. Repeated-measures multilevel modeling was adopted to analyse the data from the content analysis. The results point at a significant positive impact of assigning roles to students. However, this positive impact depends on the moment of the introduction of the roles. Higher levels of social knowledge construction were found in discussion groups where roles were introduced right at the start of the discussions and faded out towards the end. The results further indicate that self-assessment has no significant added value.