The effectiveness of the meaningful learning-based evaluation for different achieving students in a ubiquitous learning context

In recent years, fueled by the rise of mobile technology, ubiquitous learning has shown great potential in various disciplines, especially in context-aware environments. However, improper applications of this learning approach may also lead to inefficient learning and even reduced learning effectiveness. It is thus necessary to assess whether a ubiquitous learning project is able to achieve the aim of meaningful learning, so as to empower learners in scaffolding their knowledge students with different levels of achievement, especially low-achieving ones. This study aims to investigate the impact of a meaningful learning-based evaluation method on ubiquitous learning, in order to enhance the target system. A quasi-experiment is designed for this purpose, in which the post-evaluation and refined ubiquitous learning activities are adopted for the experimental group, in contrast to the control group, which works without the proposed evaluation method. The findings show that the evaluating ubiquitous learning using the meaningful learning paradigm can significantly enhance learning effectiveness, especially for low-achieving students, although the gains were not so significant for the high-achieving students, as seen in the posttest. Moreover, no significant differences in learning effectiveness were found between the experimental and control groups. The results of the posttest showed that the low-achieving students in the experimental group significantly outperformed those in the control group. This study suggests that the functions of a learning system may be need to be further enhanced to better suit the needs of low-achieving students, and that the use of a meaningful learning-based evaluation method is an effective way to find out what needs to be improved.     

An investigation into the effects that digital media can have on the learning outcomes of individuals who have dyslexia

The effects that media can have on task performance have been greatly debated over the years. Whilst agreement has begun to emerge on the effects media have on cognitive performance, little is understood about the relationship between such media effects and individual differences such as individuals who have dyslexia. This paper presents findings from a study that investigated the effects computer-based media can have on the learning outcomes of individuals who have dyslexia. The purpose of the study was to obtain data that informed the development and design of e-learning and distance learning materials for universal use. The research process was based on Dual Coding Theory and refined by current theories on dyslexia. Findings from the research are intended to help academics and providers of e-learning materials to improve the design and delivery of their learning contents.     

Effects of interactivity and instructional scaffolding on learning: Self-regulation in online video-based environments

Online learning often requires learners to be self-directed and engaged. The present study examined students' self-regulatory behaviors in online video-based learning environments. Using an experimental design, this study investigated the effects of a newly designed enhanced video learning environment, which was designed to support or scaffold students' self-regulated or self-directed learning on students' learning behaviors and outcomes. In addition, correspondence between students' self-regulation strategies in traditional learning environments and observed self-regulated learning behaviors in the enhanced video environment were examined. A cross-sectional experimental research design with systematic random assignment of participants to either the control condition (common video) or the experimental condition (enhanced video) was utilized. Undergraduate and graduate students participated in the study (N = 80). Study results indicate that the newly designed enhanced video learning environment was a superior instructional tool than the common video learning environment in terms students' learning performance. In addition, there was correspondence between graduate students' self-reported self-regulation and observed self-regulation, with those high on seeking/learning information and managing their environment/behavior more likely to engage more in interactive note-taking.     

Digital game-based learning: Impact of instructions and feedback on motivation and learning effectiveness

Although many studies have investigated the effects of digital game-based learning (DGBL) on learning and motivation, its benefits have never been systematically demonstrated. In our first experiment, we sought to identify the conditions under which DGBL is most effective, by analyzing the effects of two different types of instructions (learning instruction vs. entertainment instruction). Results showed that the learning instruction elicited deeper learning than the entertainment one, without impacting negatively on motivation. In our second experiment, we showed that if learners are given regular feedback about their performance, the entertainment instruction results in deep learning. These two experiments demonstrate that a serious game environment can promote learning and motivation, providing it includes features that prompt learners to actively process the educational content.     

Enjoy and learn with educational games: Examining factors affecting learning performance

Educational games have enhanced the value of instruction procedures in institutions and business organizations. Factors that increase students' adoption of learning games have been widely studied in past; however, the effect of these factors on learners' performance is yet to be explored. In this study, factors of Enjoyment, Happiness, and Intention to Use were chosen as important attitudes in learning educational games and increasing learning performance. A two-step between group experiment was conducted: the first study compared game-based learning and traditional instruction in order to verify the value of the game. 41 Gymnasium (middle school) students were involved, and the control and experimental groups were formed based on a pretest method. The second study, involving 46 Gymnasium students, empirically evaluates whether and how certain attitudinal factors affect learners' performance. The results of the two-part experiment showed that a) the game demonstrated good performance (as compared to traditional instruction) concerning the gain of knowledge, b) learners' enjoyment of the game has a significant relation with their performance, and c) learners' intention to use and happiness with the game do not have any relation with their performance. Our results suggest that there are attitudinal factors affecting knowledge acquisition gained by a game.     

Effects of feedback in a computer-based assessment for learning

The effects of written feedback in a computer-based assessment for learning on students’ learning outcomes were investigated in an experiment at a Higher Education institute in the Netherlands. Students were randomly assigned to three groups, and were subjected to an assessment for learning with different kinds of feedback. These are immediate knowledge of correct response (KCR) + elaborated feedback (EF), delayed KCR + EF, and delayed knowledge of results (KR). A summative assessment was used as a post-test. No significant effect was found for the feedback condition on student achievement on the post-test. Results suggest that students paid more attention to immediate than to delayed feedback. Furthermore, the time spent reading feedback was positively influenced by students’ attitude and motivation. Students perceived immediate KCR + EF feedback to be more useful for learning than KR. Students also had a more positive attitude towards feedback in a CBA when they received KCR + EF rather than KR only.     

Designing and developing a language environment for second language writers

This paper presents a field study carried out with learners who used a grammar checker in real writing tasks in an advanced course at a Swedish university. The objective of the study was to investigate how students made use of the grammar checker in their writing while learning Swedish as a second language. Sixteen students with different linguistic and cultural backgrounds participated in the study. A judgment procedure was conducted by the learners on the alarms from the grammar checker. The students’ texts were also collected in two versions; a version written before the session with the grammar checker, and a version after the session. This procedure made it possible to study to what extent the students followed the advice from the grammar checker, and how this was related to their judgments of its behavior.The results obtained demonstrated that although most of the alarms from the grammar checker were accurate, some alarms were very hard for the students to judge correctly. The results also showed that providing the student with feedback on different aspects of their target language use; not only on their errors, and facilitating the processes of language exploration and reflection are important processes to be supported in second-language learning environments.Based on these results, design principles were identified and integrated in the development of Grim, an interactive language-learning program for Swedish. We present the design of Grim, which is grounded in visualization of grammatical categories and examples of language use, providing tools for both focus on linguistic code features and language comprehension.     

Effect of metacognitive strategies and verbal-imagery cognitive style on biology-based video search and learning performance

Videos have diverse content that can assist students in learning. However, because videos are linear media, video users may take a longer time than readers of text to evaluate the context. Therefore, the process of video search may vary from one user to another depending on the users' individual characteristics, and the effectiveness of video learning may also vary across individuals. This study evaluated 100 Taiwanese fifth graders searching for videos related to “understanding animals” on YouTube and examined the effects of the students' metacognitive strategies (planning, monitoring, and evaluating) and verbal-imagery cognitive style on their video searches. The observable indicators were quantitative (search behaviors, search performance, and learning performance) and qualitative (search process observations and interviews). The study concludes that metacognitive strategy is the primary influencer of video search. Students with better metacognitive skills used fewer keywords, browsed fewer videos, and spent less time evaluating videos, but they achieved higher learning performance. They reviewed the video metadata information on the user interface and did not attempt to watch videos on the video recommendation lists, particularly videos that were irrelevant to the task requirements. During the course of the searches, keyword usage had a significant influence on the students' search performance and learning performance. The fewer keywords the students used, the better search and learning performance they were able to achieve. Our results are different from those of previous studies on text, image, and map searches. Accordingly, users must adopt different search strategies when using various types of search engines.     

Children's engagement during digital game-based learning of reading: The effects of time,rewards, and challenge

This study investigated the effects of two game features (the level of challenge and the reward system) on first and second graders' engagement during digital game-based learning of reading. We were particularly interested in determining how well these features managed to maintain children's engagement over the 8-week training period. The children (N = 138) used GraphoGame, a web-based game training letter–sound connections, at home under the supervision of parents. Data regarding the children's gaming and engagement were stored on the GraphoGame online server. A 2 × 2 factorial design was used to investigate the effects of the level of challenge (high challenge vs. high success) and the presence of the reward system (present vs. absent). Children's engagement was measured by session frequency and duration and through an in-game self-report survey that was presented at the end of the each session. According to the results, the children enjoyed GraphoGame but used it less frequently than expected. The reward system seemed to encourage the children to play longer sessions at the beginning of the training period, but this effect vanished after a few sessions. The level of challenge had no significant effect on children's engagement. The results suggest a need to investigate further the effectiveness of various game features in maintaining learner's engagement until the goals set for learning are achieved.     

Effects of different video lecture types on sustained attention,emotion, cognitive load,and learning performance

Although online courseware often includes multimedia materials, exactly how different video lecture types impact student performance has seldom been studied. Therefore, this study explores how three commonly used video lectures styles affect the sustained attention, emotion, cognitive load, and learning performance of verbalizers and visualizers in an autonomous online learning scenario by using a two-factor experimental design, brainwave detection, emotion-sensing equipment, cognitive load scale, and learning performance test sheet. Analysis results indicate that, while the three video lecture types enhance learning performance, learning performance with lecture capture and picture-in-picture types is superior to that associated with the voice-over type. Verbalizers and visualizers achieve the same learning performance with the three video types. Additionally, sustained attention induced by the voice-over type is markedly higher than that with the picture-in-picture type. Sustained attention of verbalizers is also significantly higher than that of visualizers when learning with the three video lectures. Moreover, the positive and negative emotions induced by the three video lectures do not appear to significantly differ from each other. Also, cognitive load related to the voice-over type is significantly higher than that with by the lecture capture and picture-in-picture types. Furthermore, the cognitive load for visualizers markedly exceeds that of verbalizers who are presented with the voice-over type. Results of this study significantly contribute to efforts to design of video lectures and also provide a valuable reference when selecting video lecture types for online learning.     

Student engagement with assessment and feedback: Some lessons from short-answer free-text e-assessment questions

Students were observed directly, in a usability laboratory, and indirectly, by means of an extensive evaluation of responses, as they attempted interactive computer-marked assessment questions that required free-text responses of up to 20 words and as they amended their responses after receiving feedback. This provided more general insight into the way in which students actually engage with assessment and feedback, which is not necessarily the same as their self-reported behaviour. Response length and type varied with whether the question was in summative, purely formative, or diagnostic use, with the question itself, and most significantly with students’ interpretation of what the question author was looking for. Feedback was most effective when it was understood by the student, tailored to the mistakes that they had made and when it prompted students rather than giving the answer. On some occasions, students appeared to respond to the computer as if it was a human marker, supporting the ‘computers as social actors’ hypothesis, whilst on other occasions students seemed very aware that they were being marked by a machine.     

What is important in electronic textbooks for students of different achievement levels?

This paper reports the findings of a study that explored which characteristics of electronic textbooks correlated with knowledge acquisition by learners of different achievement levels. The study was carried out on 35 units of electronic textbooks that were studied by 19 high-achieving and 19 low-achieving students in four Estonian schools. The low-achieving students profited from clear instructions, familiar icons, examples, and answering from the keyboard. The high-achieving students benefited from key-combinations, menus with different levels, the Internet, analogies and lower density of terms in the content of the material. In electronic textbooks, not only the content, but also the design of the software, should be different for learners with a different achievement level.     

Effects of student characteristics and question design on Internet search results usage in a Taiwanese classroom

A large percentage of students who use Internet search engines accept the results they are given without challenge, which conflicts with many learning theories that emphasize exploration. Longer and more in-depth searches require critical analyses of results as part of a process that is thought to produce better learning outcomes. With the development of social media, social motivation is increasingly considered an important factor in learning. To boost motivation for knowledge exploration, many instructors are adding elements of high interest to students to their teaching materials and activities. Here we investigate the effects of social motivation-associated priming on learner behaviors involving search engine results, using differences in learner characteristics (peer relationships and math skills) and search question type (add-on and integrated) as the main distinguishing factors. Our sample consisted of 81 students between the ages of 14 and 15 in a junior high school located in northern Taiwan. All of the participants were experienced in using computers, the Web, and search engines. Our results indicate (a) no significant effects of peer relationships or prior math achievement on the use of search results in that subject area; (b) the use of add-on questions (i.e., presenting math questions after presenting interpersonal relationship questions) encourages the appropriate use of summary results; and (c) the use of integrated questions (i.e., learners read stories involving a mix of interpersonal relationship and math information before addressing search questions) encourages students to read more information from search results rather than focusing on finding single “correct” answers. We interpret the third finding as evidence of greater enjoyment of exploration.     

The comparisons of the influences of prior knowledge on two game-based learning systems

Game-based learning provides many benefits, such as enhancing the interaction with students and stimulating their learning motivation. Thus, it is popular to learners who have diverse characteristics. To this end, individual differences play an essential role. Among various individual differences, previous studies demonstrated that prior knowledge has great effects on game-based learning. However, such studies mainly considered a single game-based learning system. To address this issue, this study examined how prior knowledge affects students' reactions to two different types of game based learning systems, i.e., the Machinarium and the CSI: Web Adventures. The former delivers procedural knowledge while the latter provides declarative knowledge. The results from this study indicate that prior knowledge has positive impacts in the CSI group while it has negative impacts in the Machinarium group. These findings imply that prior knowledge is useful for the context of declarative knowledge while it may not be helpful for the context of procedural knowledge.     

The effects of computer-assisted material on students’ cognitive levels,misconceptions and attitudes towards science

The purpose of this study was to investigate the effects of a Computer-assisted Instruction Material (CAIM) related to “photosynthesis” topic on student cognitive development, misconceptions and attitudes. The study conducted in 2002–2003 academic year and was carried out in two different classes taught by the same teacher, in which there were fifty two 11th grade high school students, in central city of Trabzon in Turkey. An experimental research design including the photosynthesis achievement test (PAT), the photosynthesis concept test (PCT) and science attitude scale (SAS) was applied at the beginning and at the end of the research as pre-test and post-test. After the treatment, general achievement in PAT increased by 10% in favour of experiment group (EG) at (p < 0.05) significant level. Although the increase in cognitive development at knowledge level was 14.8% in the EG and 18.2% in the control group (CG), the development at comprehension and application levels were 19.8–18.5 in the EG and 1.75–0.86 in the CG, respectively. This result showed that using CAIM in teaching photosynthesis topic was very effective for students to reach comprehension and application levels of cognitive domain. However, CAIM did not change major misconceptions related to photosynthesis topic in EG as expected. Meanwhile, same misconceptions in EG about source of energy for plants and their nutrition were decreased more than CG. It was also found out that there was little change about students’ attitudes towards science education in both groups.     

The effect of a computer-based cartooning tool on children’s cartoons and written stories

This paper reports a study assessing a new computer tool for cartoon storytelling, created by the authors for a target audience in the upper half of the English and Welsh Key Stage 2 (years 5 and 6, covering ages 9–11 years). The tool attempts to provide users with more opportunities for expressive visualisation than previous educational software; its design was motivated by earlier work connecting “moving image literacy” with print literacy, and it was used here in storywriting preparation work: users first visualised a known story, then wrote their versions of it based on the cartoons they had made. The stories produced are compared with stories written using two other preparation activities, one a pencil-and-paper cartooning exercise and the other a teacher’s normal planning session, which also resulted in a retelling of a known story.     

‘Pretty Lights’ and Maths! Increasing student engagement and enhancing learning through the use of electronic voting systems

University classes in Mathematics are traditionally perceived to be uninspiring and devoid of active student–lecturer communication. Large undergraduate classes further compound the difficulty of engaging students and enabling viable student–lecturer feedback. At the Mathematics Education Centre, Loughborough University, some staff members have been using electronic voting systems (EVS) to enliven the classroom and enable large numbers of students to respond to questions in real time during class. In this paper, we present an evaluation case study, based on student perceptions, of the impact of EVS use on student learning and engagement. The results show that majority of students are hugely positive about the usefulness and overall advantageousness of EVS use in classes. Results also show that EVS use does increase the likelihood of students participating and engaging in class, as even students who do not view EVS as being particularly useful stated that they are more likely to participate in classes where EVS are used than otherwise. However, there seems to be no correlation between EVS use and improvement (or otherwise) in student grades.     

Making sense of multitasking: The role of Facebook

Media multitasking and Facebook use are commonplace among college and university-aged students. While the two are often linked and each has been independently associated with reductions in academic performance, their relationship to each other is not particularly well understood.This relationship was examined by analysing comprehensive time-based logs of students' computer-based tasks, including Facebook, during unsupervised, self-directed learning sessions. A total of 3372 sessions contributed by 1249 students were analysed. Multitasking was extremely common – around 99% of sessions involved some multitasking (at least three instances of a particular task within a 20 min period). Facebook was the second most common task overall (University was first), accounting for 9.2% of all task instances and being present in 44% of sessions. Sessions containing Facebook typically contained more, shorter duration tasks and were significantly more likely to include multitasking behaviour. The introduction of Facebook within a session was associated with an increase in multitasking and a reduction in focused (no more than two tasks in a 20 min period) behaviour. Facebook users (students who contributed at least five sessions and used Facebook in at least one of these sessions) were also more likely to multitask and less likely to engage in focused behaviour. These results confirm that Facebook use is a key contributor to students' task switching and multitasking behaviours.     

Developing multi-dimensional evaluation criteria for English learning websites with university students and professors

Many English learning websites have been developed worldwide, but little research has been conducted concerning the development of comprehensive evaluation criteria. The main purpose of this study is thus to construct a multi-dimensional set of criteria to help learners and teachers evaluate the quality of English learning websites. These evaluation guidelines are based on web usability, learning materials, functionality of assisting language learning, technology integration, and learner preferences. In order to achieve this goal, the researchers used a rigorous four-phase procedure and utilized both qualitative and quantitative research methods with university students and professors in Design-based Research to construct and refine the evaluation criteria in 2008–2009. Fifty-eight evaluation criteria were refined and finalized. In addition, to cope with the difficulties in applying the derived criteria and explaining what the results indicate, this study presents a practical application by evaluating an English learning website.     

Examining the effectiveness of technology use in classrooms: A tertiary meta-analysis

Identifying effective literacy instruction programs has been a focal point for governments, educators and parents over the last few decades (Ontario Ministry of Education, 2004, 2006; Council of Ontario Directors of Education, 2011). Given the increasing use of computer technologies in the classroom and in the home, a variety of information communication technology (ICT) interventions for learning have been introduced. Meta-analyses comparing the impact of these programs on learning, however, have yielded inconsistent findings (Andrews et al., 2007, Torgerson and Zhu, 2003, Slavin et al., 2008, Slavin et al., 2009). The present tertiary meta-analytic review re-assesses outcomes presented in three previous meta-analyses. Four moderator variables assessed the impact of the systematic review from which they were retrieved, training and support, implementation fidelity and who delivered the intervention (teacher versus researcher). Significant results were found when training and support was entered as a moderator variable with the small overall effectiveness of the ICTs (ES = 0.18), similar to those found in previous research, increasing significantly (ES = 0.57). These findings indicate the importance of including implementation factors such as training and support, when considering the relative effectiveness of ICT interventions.