Mining LMS data to develop an “early warning system” for educators: A proof of concept

Earlier studies have suggested that higher education institutions could harness the predictive power of Learning Management System (LMS) data to develop reporting tools that identify at-risk students and allow for more timely pedagogical interventions. This paper confirms and extends this proposition by providing data from an international research project investigating which student online activities accurately predict academic achievement. Analysis of LMS tracking data from a Blackboard Vista-supported course identified 15 variables demonstrating a significant simple correlation with student final grade. Regression modelling generated a best-fit predictive model for this course which incorporates key variables such as total number of discussion messages posted, total number of mail messages sent, and total number of assessments completed and which explains more than 30% of the variation in student final grade. Logistic modelling demonstrated the predictive power of this model, which correctly identified 81% of students who achieved a failing grade. Moreover, network analysis of course discussion forums afforded insight into the development of the student learning community by identifying disconnected students, patterns of student-to-student communication, and instructor positioning within the network. This study affirms that pedagogically meaningful information can be extracted from LMS-generated student tracking data, and discusses how these findings are informing the development of a customizable dashboard-like reporting tool for educators that will extract and visualize real-time data on student engagement and likelihood of success.     

Analyzing academic achievement of junior high school students by an improved rough set model

Rough set theory (RST) is an emerging technique used to deal with problems in data mining and knowledge acquisition. However, the RST approach has not been widely explored in the field of academic achievement. This investigation developed an improved RST (IMRST) model, which employs linear discriminant analysis to determine a reduct of RST, and analyzed the academic achievements of junior high school students in Taiwan. An interactive interface was created so that students could answer questions to predict their academic achievement and they could learn essential skills for improving their academic achievement. Empirical results showed that the IMRST model selects crucial information from data without predetermining factors and can provide accurate rates for inference rules. Hence, the developed IMRST model is a promising alternative for analyzing academic achievement data.     

A creative thinking approach to enhancing the web-based problem solving performance of university students

Along with the advancement of information and communication technology, researchers have pointed out the necessity and challenges of developing effective instructional strategies to enhance students' web-based problem-solving performance, which refers to the ability of investigating a series of related problems via searching for, abstracting and summarizing information on the web. In this study, a creative thinking strategy is proposed to cope with this problem. Moreover, an experiment was conducted on 80 freshmen from two classes of a university to evaluate the effectiveness of the proposed approach. The experimental results show that the proposed approach improved the students' web-based problem solving performance in comparison with the conventional approach in terms of “problem finding” and “idea finding.” Moreover, it was found that the proposed approach could improve the “fact finding” performance of the students with intuitive-type cognitive style. Accordingly, some implications and suggestions are given for educators who attempt to conduct web-based problem-solving activity.     

Student perception of academic writing skills activities in a traditional programming course

Employers of computing graduates have high expectations of graduates in terms of soft skills, the most desirable of these being communication skills. Not only must the graduates exhibit writing skills, but they are expected to be highly proficient therein. The consequence of this expectation is not only performance pressure exerted on the graduate, but also the laying down of a challenge at the door of computing instructors to encourage and instil successfully the art of writing in the graduate.Varied and numerous initiatives have been launched in the computing discipline to address this particular challenge over the past few decades. Despite documented success on the many experiences of incorporating writing into the computing discipline, the fact is that the challenge is as relevant today as it was in the 1980s. There is sufficient evidence to suggest that agreement is yet to be reached on exactly how to integrate successfully writing skills into traditional computing curricula, thereby familiarising computing students with an essential activity of their chosen profession. Furthermore, the reporting of successful experiences tends to focus on the application of a variety of teaching strategies, is predominantly based on the experiences of instructors and is dominated by a lack of quantifiable results.This apparent lack of sufficient empirical evidence prompted a rigorous investigation into the perceived benefits of the integration of writing skills activities into a traditional intermediary level programming course. The main aim of the study is to measure the perceived benefit of each of a number of academic writing interventions, facilitated to skill students appropriately in the art of writing for the computing discipline. Each intervention is designed to support particular principles of academic writing. The ultimate outcome of the study is a survey which allows student participants to measure the perceived benefit-impact of each intervention.The qualitative and quantitative findings of this study suggest that students perceive most of the commonly used academic writing activities as beneficial in the construction of a report. What is apparent, however, is that those activities that are most frequently used by instructors are not necessarily perceived by students as being the most useful activities.     

An assessment of the influence of perceived enjoyment and attitude on the intention to use technology among pre-service teachers: A structural equation modeling approach

This study examined the influence of perceived enjoyment on pre-service teachers’ intention to use technology, in addition to re-appraising the role of attitude toward use in the Technology Acceptance Model (TAM; Davis, Bagozzi, &; Warshaw, 1989). One hundred and fifty-three participants in Singapore completed a survey questionnaire measuring their responses to five constructs from a research model that was developed specifically for the study. Structural Equation Modeling (SEM) showed that perceived enjoyment was a significant predictor of perceived usefulness, perceived ease of use, and intention to use technology. The findings of this study support the view of Davis et al. (1989) from over 30 years ago that attitude toward use contributes only modestly to the TAM.     

Using eye tracking to understand learners' reading process through the concept-mapping learning strategy

The author used an eye-tracking methodology to examine the influence of the concept-mapping learning strategy on learners performing an English reading task. Eighty-six freshmen enrolled in English courses participated in this control-group pretest-posttest experiment, and received either traditional or concept-mapping instruction for learning English reading skills. A concept-mapping strategy was introduced to the learners in the experimental group to improve their reading ability. The results of independent t tests, in which eye-tracking data on fixation time and fixation path were used, indicated that the participants who received concept-mapping instruction exhibited shorter fixation times on the core concept and other content in the text, compared with the group that received traditional instruction. Moreover, the experimental group demonstrated longer and irregular rereading paths than the control group (CG) did. These results indicated that concept mapping serves as a reference to assist average readers to improve and to identify primary ideas that clarify the meaning of an article.     

A hybrid approach to promoting students’ web-based problem-solving competence and learning attitude

Fostering problem-solving abilities has long been recognized as an important issue in education; however, past studies have shown that it is difficult and challenging to find effective learning strategies or tools for improving students’ problem-solving abilities. To cope with this problem, in this study, a hybrid approach that integrates the cognitive apprenticeship model with the collaborative learning strategy is proposed for conducting web-based problem-solving activities. Students’ problem-solving performance is examined in such a hybrid learning context. Furthermore, past studies indicate that cognitive load could affect learners’ performance; thus, the influence of cognitive load on students’ problem-solving effectiveness with this new approach is investigated in depth. The experimental results show that middle- and low-achievement students in the experimental group gained significant benefits from the hybrid approach in comparison with those who learned with the traditional approach. Accordingly, a discussion of how to accommodate the needs of different learning ability groups is provided.     

Effects of feedback in a computer-based assessment for learning

The effects of written feedback in a computer-based assessment for learning on students’ learning outcomes were investigated in an experiment at a Higher Education institute in the Netherlands. Students were randomly assigned to three groups, and were subjected to an assessment for learning with different kinds of feedback. These are immediate knowledge of correct response (KCR) + elaborated feedback (EF), delayed KCR + EF, and delayed knowledge of results (KR). A summative assessment was used as a post-test. No significant effect was found for the feedback condition on student achievement on the post-test. Results suggest that students paid more attention to immediate than to delayed feedback. Furthermore, the time spent reading feedback was positively influenced by students’ attitude and motivation. Students perceived immediate KCR + EF feedback to be more useful for learning than KR. Students also had a more positive attitude towards feedback in a CBA when they received KCR + EF rather than KR only.     

Effects of computer-assisted instruction on students’ achievement in Taiwan: A meta-analysis

A meta-analysis was performed to synthesize existing research comparing the effects of computer-assisted instruction (CAI) versus traditional instruction (TI) on students’ achievement in Taiwan. Fifty-two studies were located from four sources, and their quantitative data was transformed into effect size (ES). The overall grand mean of the study-weighted ES for all 52 studies was 0.55. The results suggest that CAI is more effective than TI in Taiwan. In addition, two of the seventeen variables selected for this study (i.e., statistical power, and comparison group) had a statistically significant impact on the mean ES.     

An auto-scoring mechanism for evaluating problem-solving ability in a web-based learning environment

The rapid development of computer and network technologies has attracted researchers to investigate strategies for and the effects of applying information technologies in learning activities; simultaneously, learning environments have been developed to record the learning portfolios of students seeking web information for problem-solving. Although previous research has demonstrated the benefits of applying information technologies to learning activities, the difficulties in doing so have also been revealed. One of the major difficulties is the lack of a mechanism to assist teachers in evaluating the problem-solving ability of the students, such that constructive suggestions can be given to the students, and tutoring strategies can be improved accordingly.     

Exploring academic teachers’ continuance toward the web-based learning system: The role of causal attributions

The Expectation Confirmation Model (ECM) is a popular model used to explain the continuance of information system usage. However, past studies have found that the ECM, based on extrinsic motivations (e.g. perceived usefulness, user satisfaction), has limitations insofar as people often have both intrinsic and extrinsic motivations simultaneously. This study used the belief of causal attributions to extend the ECM and verify it in a web-based learning system (WLS) context. Nine hypotheses were derived from the modified ECM. Empirical data were collected by a mail survey from faculty with experience of WLS in Taiwanese universities. The final sample used for further analysis comprised of 144 respondents. The results revealed that the modified ECM has greater power to explain the continued intention of WLS usage even if the effects of user satisfaction and perceived usefulness on continued intention are reduced.     

The ECDL programme in Italian Universities

The European Computer Driving Licence (ECDL) programme aims at testing practical skills and competences in using ICT tools. This paper presents the results of a monitoring exercise aimed at analyzing the impact of the ECDL programme in the Italian Universities. The ECDL programme, adopted in most Italian Universities since the year 2000, has involved several tens of thousands of students. Our investigation focused on the experiences of the Universities in the year 2003 and, in particular, on the organizational and teaching issues addressed for the implementation of the programme. The analysis has shown that the implementation relied on a large variety of choices about what concerns the teaching organization, the credits awarded to the students and the type of ECDL certification required. These choices varied as a function of the size of the Universities, whereas their geographical location did not seem to have any influence.     

Comparing the social knowledge construction behavioral patterns of problem-based online asynchronous discussion in e/m-learning environments

In recent years, researchers have conducted various studies on applying wireless networking technology and mobile devices in education settings. However, research on behavioral patterns in learners' online asynchronous discussions with mobile devices is limited. The purposes of this study are to develop a mobile learning system, mobile interactive teaching feedback system (MITFS), linked to both mobile devices and the internet, to support learners with online asynchronous discussion, and combine content analysis and sequential analysis to compare and contrast the social knowledge construction behavioral patterns of problem-based asynchronous discussion in e-learning and m-learning environments. This study investigated four weeks of online discussions in an “Introduction to Computer Science” course involving forty first year university students. The control group (online asynchronous discussion without mobile devices) and the experimental group (online asynchronous discussion with mobile devices) in the group discussions were explored. By using content analysis and sequential analysis for the problem-based online asynchronous discussion of the behavioral patterns and differences between students in control and experimental group, the results showed that using mobile devices in online asynchronous discussion influenced students' learning performance. Some interesting results were found. Firstly, when the students used mobile devices in discussion situations, they could more engage in reflecting thinking, sharing more information, and further facilitating social knowledge construction among group members. Secondly, the experimental group performed better than the control group in terms of participation and diversity in knowledge construction behavioral patterns. Finally, based upon the findings, some implications are proposed for further research.     

Using synchronous peer tutoring system to promote elementary students’ learning in mathematics

The face-to-face computer-supported collaborative learning has emerged as an important strategy to elementary students' learning. Few studies have explicitly incorporated the structured peer tutoring to the synchronous environment in mathematics. This study was aimed to explore the effects of the synchronous peer tutoring system on children's mathematics learning. In the project, there were 88 10-11-year-old students who peer tutored each other in mathematics in the face-to-face online environment for a year. Compared to the control group, students in the experimental group had significant gains in mathematics learning, especially in the arithmetic and application types of questions. This study demonstrated the positive effects of peer support via the online synchronous learning on students' self-concept and attitudes toward mathematics learning. The results indicated that the longer the vulnerable pupils engaged in peer tutoring online, the more they benefited from the process. The finding suggests that students demonstrate different mathematics reasoning skills when they are paired with peers at different levels of ability. These findings demonstrate that the synchronous peer tutoring system is an effective tool to enhance elementary students' learning in mathematics, as well as promote positive self-concepts.     

Effects of teaching and learning styles on students’ reflection levels for ubiquitous learning

Ubiquitous learning (u-learning), in conjunction with supports from the digital world, is recognized as an effective approach for situating students in real-world learning environments. Earlier studies concerning u-learning have mainly focused on investigating the learning attitudes and learning achievements of students, while the causations such as learning style and teaching style were usually ignored. This study aims to investigate the effects of teaching styles and learning styles on reflection levels of students within the context of u-learning. In particular, we investigated the teaching styles at the dimensions of brainstorming and instruction and recall and the learning styles at the dimensions of active and reflective learning. The experiment was conducted with 39 fifth grader students at an elementary school in southern Taiwan. A u-learning environment was established at a butterfly ecology garden to conduct experiments for natural science courses. The experimental results of one-way ANCOVA show that those students who received a matching teaching–learning style presented a significant improvement in their reflection level. That is, matching the learning styles of students with the appropriate teaching styles can significantly improve students’ reflection levels in a u-learning environment.     

Fuzzy cognitive map modelling educational software adoption

Educational software adoption across UK secondary schools is seen as unsatisfactory. Based on stakeholders’ perceptions, this paper uses fuzzy cognitive maps (FCMs) to model this adoption context. It discusses the development of the FCM model, using a mixed-methods approach and drawing on participants from three UK secondary schools. The study presents three phases involved in the development of the model, where individual FCMs were developed in phase one and then the individual FCMs were aggregated in phase two. In phase three, further interrelationships identified from the empirical data were assigned weightings and added, resulting in the final FCM model. Following a static analysis of the model, the resulting FCM offers a visual medium of factors key to the adoption of educational software as perceived by relevant stakeholders. As a holistic model it provides insight into the context of educational software adoption in schools, which can be used to guide both educational decision-makers in where to focus their efforts and software developers in terms of more focused and appropriate software development efforts.     

Students' perceptions of instructors' roles in blended and online learning environments: A comparative study

This study develops an instrument—the Online Instructor Role and Behavior Scale (OIRBS)—and uses it to examine students' perceptions of instructors' roles in blended and online learning environments. A total sample of 750 university students participated in this study. Through a confirmatory factor analysis, the OIRBS was validated in five constructs: course designer and organizer (CDO), discussion facilitator (DF), social supporter (SS), technology facilitator (TF), and assessment designer (AD). The results show that the five factor structures remained invariant across the blended learning and online learning. Both students in blended learning environments and students in online learning environments exhibited the greatest weight in the CDO dimension, followed by the TF and DF dimensions. In addition, students in the online learning environments scored higher in the DF dimension than did those in the blended learning environments.     

Applying the Practical Inquiry Model to investigate the quality of students’ online discourse in an information ethics course based on Bloom’s teaching goal and Bird’s 3C model

The goal of this study is to better understand how the study participants’ cognitive discourse is displayed in their learning transaction in an asynchronous, text-based conferencing environment based on Garrison’s Practical Inquiry Model (2001). The authors designed an online information ethics course based on Bloom’s taxonomy of educational objectives and Bird’s 3C model (Content–Construction–Consolidation). The content analyzed included the participants’ message posts, the quality of the dialogue and the scaffolding strategies for mentoring used by group leaders and teachers.     

ABRACADABRA in the hands of teachers: The effectiveness of a web-based literacy intervention in grade 1 language arts programs

This study explored how teacher variations in use of a web-technology of known effectiveness, affected growth in literacy. Teachers used ABRACADACABRA (ABRA) (http://grover.concordia.ca/abra/version1/abracadabra.html), in Grade 1 Language Arts classes. Children (N = 60) across three classrooms were exposed to ABRA activities and a control class followed the regular provincial literacy program. Interventions were run by briefly-trained teachers for a total of 16 h. Each teacher chose to use the program in qualitatively distinct ways that corresponded to the first three stages of Sandholtz, Ringstaff, and Dwyer’s (1997) technology integration model, namely: Entry, Adoption and Adaptation. Growth in literacy between pre- and post-test revealed significant differences associated with technology integration style across all measures of literacy and related language skills. The largest and most-widespread effects were evident for the Adaptation group that linked technology content to wider learning themes in the classroom. In terms of overall growth in standardised literacy scores across all six such measures used, Adaptation proved to be 60% more effective than Adoption. Results suggest that explorations of exactly how teachers use technology have important implications for practice as well as for the interpretation of field-based studies of the effectiveness of educational technology.