Evaluation of the criteria and effectiveness of distance e-learning with consistent fuzzy preference relations

The electronic learning (e-learning) has gradually become more and more important in today’s school in Taiwan. Many colleges and universities offer distance e-learning courses or programs for students. An effective teaching or learning through a distance web e-learning system depends on many factors (or criteria). The analytic hierarchy process (AHP) model is suitable for dealing with the multi-criteria problems. This paper utilizes the consistent fuzzy preference relations (CFPR) in AHP model to evaluate these factors. The CFPR is computational simplicity and guarantees the consistence of decision matrices. Rating the criteria is important. An empirical example using CFPR in AHP model to find the weights is presented. The weight can point out which factor is important, especially when the time, manpower, and financial support are limited. The rating results can be directly used to evaluate the distance e-learning effectiveness and can provide teachers and decision-makers in schools important information for improving e-learning practice in the future.     

An approach to usability evaluation of e-learning applications

Despite recent advances of electronic technologies in e-learning, a consolidated evaluation methodology for e-learning applications is not available. The evaluation of educational software must consider its usability and more in general its accessibility, as well as its didactic effectiveness. This work is a first step towards the definition of a methodology for evaluating e-learning applications. Specific usability attributes capturing the peculiar features of these applications are identified. A preliminary user study involving a group of e-students, observed during their interaction with an e-learning application in a real situation, is reported. Then, the proposal is put forward to adapt to the e-learning domain a methodology for systematic usability evaluation, called SUE. Specifically, evaluation patterns are proposed that are able to drive the evaluators in the analysis of an e-learning application.     

Multi-dimensional students’ evaluation of e-learning systems in the higher education context: An empirical investigation

There has been little research on assessment of learning management systems (LMS) within educational organizations as both a web-based learning system for e-learning and as a supportive tool for blended learning environments. This study proposes a conceptual e-learning assessment model, hexagonal e-learning assessment model (HELAM) suggesting a multi-dimensional approach for LMS evaluation via six dimensions: (1) system quality, (2) service quality, (3) content quality, (4) learner perspective, (5) instructor attitudes, and (6) supportive issues. A survey instrument based on HELAM has been developed and applied to 84 learners. This sample consists of students at both undergraduate and graduate levels who are users of a web-based learning management system, U-Link, at Brunel University, UK. The survey instrument has been tested for content validity, reliability, and criterion-based predictive validity. The analytical results strongly support the appropriateness of the proposed model in evaluating LMSs through learners’ satisfaction. The explanatory factor analysis showed that each of the six dimensions of the proposed model had a significant effect on the learners’ perceived satisfaction. Findings of this research will be valuable for both academics and practitioners of e-learning systems.     

Investigating students’ perceived satisfaction,behavioral intention,and effectiveness of e-learning: A case study of the Blackboard system

Although the benefits of e-learning have been discussed in various previous studies; it is a critical issue of better understanding the reasons why some learners are dissatisfied with the e-learning experience. Therefore, this research investigates learners’ satisfaction, behavioral intentions, and the effectiveness of the Blackboard e-learning system. A total of 424 university students were surveyed using a standard questionnaire. The results showed that perceived self-efficacy is a critical factor that influences learners’ satisfaction with the Blackboard e-learning system. Perceived usefulness and perceived satisfaction both contribute to the learners’ behavioral intention to use the e-learning system. Furthermore, e-learning effectiveness can be influenced by multimedia instruction, interactive learning activities, and e-learning system quality. This research proposes a conceptual model for understanding learners’ satisfaction, behavioral intention, and effectiveness of using the e-learning system.     

Explaining and predicting users’ continuance intention toward e-learning: An extension of the expectation–confirmation model

Although e-learning has been prompted to various education levels, the intention to continue using such systems is still very low, and the acceptance-discontinuance anomaly phenomenon (i.e., users discontinue using e-learning after initially accepting it) is a common occurrence. This paper synthesizes the expectation–confirmation model (ECM), the technology acceptance model (TAM), the theory of planned behavior (TPB), and the flow theory to hypothesize a theoretical model to explain and predict the users’ intentions to continue using e-learning. The hypothesized model is validated empirically using a sample collected from 363 learners of a Web-based learning program designed for continuing education. The results demonstrate that satisfaction has the most significant effect on users’ continuance intention, followed by perceived usefulness, attitude, concentration, subjective norm, and perceived behavior control as significant but weaker predictors. The implications of these findings for e-learning practitioners are discussed at the end of this work.     

A development of learning widget on m-learning and e-learning environments

This article describes the development of learning widget on m-learning and e-learning environments. A widget is a small, simple and useful application supporting user-oriented contents. The user may select and install widgets that are convenient as well as an auto-updating application including weather or calendar. These widgets are especially more useful, because they are able to be installed on a mobile device, a website or a desktop computer. If we take advantage of widgets for education, we may use this learning tool for delivering and pulling learning contents, essences of lessons or word learning. To that end, we developed an effective learning widget and then verified its usability, usefulness and effectiveness for m-learning and e-learning. That is, we evaluated the learning widget with a heuristic evaluation method. We identified 72 interface problems by using a set of 10 usability criteria or heuristics. In addition, we considered how to design the learning widget with consideration given to devices on m-learning and e-learning. Moreover, we experimented by conducting a pilot test with 34 students, a field test with 60 teachers and technology acceptance model (TAM) analysis with 15 teachers. We verified the effectiveness and usefulness of learning with a questionnaire, a quiz and TAM, where the subjects, after using the learning widget in real learning activities, rated the widget's efficacy. The result shows that the learning widget is useful for m-learning and e-learning environments.     

基于混合VIKOR方法的供应商选择决策模型

通过文献统计方法构建供应商选择评价准则体系. 针对主客观评价准则下多种数据类型混合的评价信息, 提出根据采集到的原始数据客观确定决策者和决策准则权重, 避免主观赋权带来的二次不确定性. 采用基于混合信息的VIKOR方法集结评价值, 以克服数据类型的不可公度性和数据转换造成的信息损失. 从理论、数值实验和敏感性分析3 个维度与TOPSIS 方法进行对比, 表明了所提出方法在供应商选择决策过程中的特点和优势.

     

PeRSIVA: An empirical evaluation method of a student model of an intelligent e-learning environment for computer programming

In this paper an evaluation method for assessing the effectiveness, accuracy and validity of a student model was presented. Our method is called PeRSIVA and is a combination of the well-known evaluation method of Kirkpatrick and the layered evaluation framework. These well-known and commonly used evaluation techniques have been selected in order to design an accurate and correct evaluation methodology, since there are no clear guidelines in the literature for the evaluation of the student model of an adaptive tutoring system. Furthermore, PeRSIVA method was used to evaluate the hybrid student model, which combines an overlay model with stereotypes and fuzzy logic techniques, of an e-learning system. Particularly, PeRSIVA assesses the results of student modeling in students' satisfaction, performance, progress, behavior and state, as well as the validity of the conclusions drawn by the student model and the validity of the adaptation decision making. The e-learning system was used by the students of a postgraduate program in the field of informatics in the University of Piraeus and the evaluation results demonstrated learning improvements in students and adaptation success to students' needs.     

Multi-dimensional students’ evaluation of e-learning systems in the higher education context: An empirical investigation

There has been little research on assessment of learning management systems (LMS) within educational organizations as both a web-based learning system for e-learning and as a supportive tool for blended learning environments. This study proposes a conceptual e-learning assessment model, hexagonal e-learning assessment model (HELAM) suggesting a multi-dimensional approach for LMS evaluation via six dimensions: (1) system quality, (2) service quality, (3) content quality, (4) learner perspective, (5) instructor attitudes, and (6) supportive issues. A survey instrument based on HELAM has been developed and applied to 84 learners. This sample consists of students at both undergraduate and graduate levels who are users of a web-based learning management system, U-Link, at Brunel University, UK. The survey instrument has been tested for content validity, reliability, and criterion-based predictive validity. The analytical results strongly support the appropriateness of the proposed model in evaluating LMSs through learners’ satisfaction. The explanatory factor analysis showed that each of the six dimensions of the proposed model had a significant effect on the learners’ perceived satisfaction. Findings of this research will be valuable for both academics and practitioners of e-learning systems.     

Do patterns help novice evaluators? A comparative study

Evaluating e-learning systems is a complex activity which requires considerations of several criteria addressing quality in use as well as educational quality. Heuristic evaluation is a widespread method for usability evaluation, yet its output is often prone to subjective variability, primarily due to the generality of many heuristics. This paper presents the pattern-based (PB) inspection, which aims at reducing this drawback by exploiting a set of evaluation patterns to systematically drive inspectors in their evaluation activities. The application of PB inspection to the evaluation of e-learning systems is reported in this paper together with a study that compares this method to heuristic evaluation and user testing. The study involved 73 novice evaluators and 25 end users, who evaluated an e-learning application using one of the three techniques. The comparison metric was defined along six major dimensions, covering concepts of classical test theory and pragmatic aspects of usability evaluation. The study showed that evaluation patterns, capitalizing on the reuse of expert evaluators know-how, provide a systematic framework which reduces reliance on individual skills, increases inter-rater reliability and output standardization, permits the discovery of a larger set of different problems and decreases evaluation cost. Results also indicated that evaluation in general is strongly dependent on the methodological apparatus as well as on judgement bias and individual preferences of evaluators, providing support to the conceptualisation of interactive quality as a subjective judgement, recently brought forward by the UX research agenda.