Multi-dimensional students’ evaluation of e-learning systems in the higher education context: An empirical investigation

There has been little research on assessment of learning management systems (LMS) within educational organizations as both a web-based learning system for e-learning and as a supportive tool for blended learning environments. This study proposes a conceptual e-learning assessment model, hexagonal e-learning assessment model (HELAM) suggesting a multi-dimensional approach for LMS evaluation via six dimensions: (1) system quality, (2) service quality, (3) content quality, (4) learner perspective, (5) instructor attitudes, and (6) supportive issues. A survey instrument based on HELAM has been developed and applied to 84 learners. This sample consists of students at both undergraduate and graduate levels who are users of a web-based learning management system, U-Link, at Brunel University, UK. The survey instrument has been tested for content validity, reliability, and criterion-based predictive validity. The analytical results strongly support the appropriateness of the proposed model in evaluating LMSs through learners’ satisfaction. The explanatory factor analysis showed that each of the six dimensions of the proposed model had a significant effect on the learners’ perceived satisfaction. Findings of this research will be valuable for both academics and practitioners of e-learning systems.     

Using SERVQUAL to assess the quality of e-learning experience

Distance learning programs have been dramatically expanding in accordance with demand. Assessment of the quality of e-learning has become a strategic issue, one that is critical to program survival. In this study we propose a modified SERVQUAL instrument for assessing e-learning quality. The instrument consists of five dimensions: Assurance, Empathy, Responsiveness, Reliability, and Website Content. Data analysis from 203 e-learning students shows that four out of these five dimensions (except Reliability) play a significant role in perceived e-learning quality, which in turn affects learners’ satisfaction and future intentions to enroll in online courses. Managerial implications of the major findings are provided.     

Investigating students’ perceived satisfaction,behavioral intention,and effectiveness of e-learning: A case study of the Blackboard system

Although the benefits of e-learning have been discussed in various previous studies; it is a critical issue of better understanding the reasons why some learners are dissatisfied with the e-learning experience. Therefore, this research investigates learners’ satisfaction, behavioral intentions, and the effectiveness of the Blackboard e-learning system. A total of 424 university students were surveyed using a standard questionnaire. The results showed that perceived self-efficacy is a critical factor that influences learners’ satisfaction with the Blackboard e-learning system. Perceived usefulness and perceived satisfaction both contribute to the learners’ behavioral intention to use the e-learning system. Furthermore, e-learning effectiveness can be influenced by multimedia instruction, interactive learning activities, and e-learning system quality. This research proposes a conceptual model for understanding learners’ satisfaction, behavioral intention, and effectiveness of using the e-learning system.     

Intelligent automatic community grouping system by multiagents

We have developed an intelligent ubiquitous web-based e-learning system based on multiagents. The proposed system, intelligent ubiquitous web-based e-learning multiagent system, uses the new distributed multiagent framework and neural networks for e-learning grouping. The proposed system implements the user’s individual satisfaction network by analyzing the degree of satisfaction among learners in groups in a web environment. The satisfaction network is personalized by providing weights to the learners’ degree of satisfaction in the e-learning grouping. It constructs the learners’ satisfaction network model about the e-learning grouping. Based on this network model, the proposed system can decide if the group remains, or is reorganized, or breaks down for the next time, and the system learns about these states. This work was presented in part at the 12th International Symposium on Artificial Life and Robotics, Oita, Japan, January 25–27, 2007     

A study of student satisfaction in a blended e-learning system environment

This study proposes a research model that examines the determinants of student learning satisfaction in a blended e-learning system (BELS) environment, based on social cognitive theory. The research model is tested using a questionnaire survey of 212 participants. Confirmatory factor analysis (CFA) was performed to test the reliability and validity of the measurements. The partial least squares (PLS) method was used to validate the measurement and hypotheses. The empirical findings indicate that computer self-efficacy, performance expectations, system functionality, content feature, interaction, and learning climate are the primary determinants of student learning satisfaction with BELS. The results also show that learning climate and performance expectations significantly affect learning satisfaction. Computer self-efficacy, system functionality, content feature and interaction significantly affect performance expectations. Interaction has a significant effect on learning climate. The findings provide insight into those factors that are likely significant antecedents for planning and implementing a blended e-learning system to enhance student learning satisfaction.     

What drives a successful e-Learning? An empirical investigation of the critical factors influencing learner satisfaction

E-learning is emerging as the new paradigm of modern education. Worldwide, the e-learning market has a growth rate of 35.6%, but failures exist. Little is known about why many users stop their online learning after their initial experience. Previous research done under different task environments has suggested a variety of factors affecting user satisfaction with e-Learning. This study developed an integrated model with six dimensions: learners, instructors, courses, technology, design, and environment. A survey was conducted to investigate the critical factors affecting learners’ satisfaction in e-Learning. The results revealed that learner computer anxiety, instructor attitude toward e-Learning, e-Learning course flexibility, e-Learning course quality, perceived usefulness, perceived ease of use, and diversity in assessments are the critical factors affecting learners’ perceived satisfaction. The results show institutions how to improve learner satisfaction and further strengthen their e-Learning implementation.     

An empirical analysis of the antecedents of web-based learning continuance

Like any other product, service and Web-based application, the success of Web-based learning depends largely on learners’ satisfaction and other factors that will eventually increase learners’ intention to continue using it. This paper integrates the concept of subjective task value and fairness theory to construct a model for investigating the motivations behind learners’ intention to continue using Web-based learning. The model theorizes that four components of subjective task value (i.e., attainment, utility, intrinsic, and cost) and three dimensions of fairness (i.e., distributive, procedural, and interactional) affect learners’ satisfaction. We also argue that satisfaction and four distinct components of subjective task value influence learners’ intention to continue using Web-based learning. The hypothesized model is validated empirically using data collected from 202 learners of a Web-based learning program designed for continuing education. The results showed that attainment value, utility value, intrinsic value, distributive fairness, and interactional fairness exhibited significant positive effects on satisfaction. Utility value and satisfaction play significant roles in shaping learners’ intention to continue using Web-based learning.     

Surveying instructor and learner attitudes toward e-learning

The trend of using e-learning as a learning and/or teaching tool is now rapidly expanding into education. Although e-learning environments are popular, there is minimal research on instructors’ and learners’ attitudes toward these kinds of learning environments. The purpose of this study is to explore instructors’ and learners’ attitudes toward e-learning usage. Accordingly, 30 instructors and 168 college students are asked to answer two different questionnaires for investigating their perceptions. After statistical analysis, the results demonstrate that instructors have very positive perceptions toward using e-learning as a teaching assisted tool. Furthermore, behavioral intention to use e-learning is influenced by perceived usefulness and self-efficacy. Regarding to learners’ attitudes, self-paced, teacher-led, and multimedia instruction are major factors to affect learners’ attitudes toward e-learning as an effective learning tool. Based on the findings, this research proposes guidelines for developing e-learning environments.     

e-Learning process management and the e-learning performance: Results of a European empirical study

The aim of this paper was to design and assess a comprehensive model for managing the e-learning process and to define the relationship between systematic implementation of the model, outcomes of certain e-learning aspects and subject of e-learning. The validation of the model was performed by using two questionnaires sent via e-mail to teachers and field experts from the chosen sample of 14 European schools participating in an EU-funded project. Research results imply the existence of a clear link between planning and controlling of the e-learning process and its learning outcomes. On the other hand, no empirical relationship between the e-learning outcomes and the subject of learning has been established. It is believed that the model and its practical implications can be used by institutions engaged in e-learning, or as a process model for introducing e-learning related activities.     

Design a personalized e-learning system based on item response theory and artificial neural network approach

In web-based educational systems the structure of learning domain and content are usually presented in the static way, without taking into account the learners’ goals, their experiences, their existing knowledge, their ability (known as insufficient flexibility), and without interactivity (means there is less opportunity for receiving instant responses or feedbacks from the instructor when learners need support). Therefore, considering personalization and interactivity will increase the quality of learning. In the other side, among numerous components of e-learning, assessment is an important part. Generally, the process of instruction completes with the assessment and it is used to evaluate learners’ learning efficiency, skill and knowledge. But in web-based educational systems there is less attention on adaptive and personalized assessment. Having considered the importance of tests, this paper proposes a personalized multi-agent e-learning system based on item response theory (IRT) and artificial neural network (ANN) which presents adaptive tests (based on IRT) and personalized recommendations (based on ANN). These agents add adaptivity and interactivity to the learning environment and act as a human instructor which guides the learners in a friendly and personalized teaching environment.     

Transactional distance revisited: Bridging face and empirical validity

Implementation of e-learning, whether in academic institutions or in the corporate world, is fast growing. While there has been a plethora of research in the field of e-learning, most empirical results remain inconsistent. One problem with such inconsistencies is the lack of clear takeaways that can guide practitioners on the best practices of e-learning. In this paper, we propose an overarching theoretical framework based on Moore’s transactional distance theory to examine e-learning. While this theory has existed for some time and has face validity, it has not received empirical support. We re-examine the core tenets of the theory, and test them in a manner that is ontologically consistent with the focus of the theory on learners’ perceptions, thereby bridging the gap between the theory’s face and empirical validity. We find strong support for the influence of transactional distance factors on our outcome of interest, i.e. individuals’ intentions to return for another e-learning experience. Our results help us arrive at contributions to research and practice, which include suggestions to enhance the success of e-learning initiatives.     

An empirical investigation linking learners’ adoption of blended learning to their intention of full e-learning

Learning management system (LMS) is playing a major role in higher academic institutions worldwide. Even though full e-learning is becoming a feasible strategy for a number of institutions in the world, some institutions, especially those in developing countries, are resisting a full e-learning environment. Consequently, these academic institutions initially adopt LMS for blended learning to assess their readiness for full e-learning transformation. There are a number of studies that investigate the determinants of full e-learning, but very limited studies investigate the link between learners’ perception of blended learning and full e-learning. The objective of this study was to link learners’ adoption (perceived ease of use, perceived usefulness (PU) and satisfaction) of LMS in blended learning and their personal characteristics (self-efficacy, technology experience and personal innovativeness) to their intention to use full e-learning. Data were collected through a questionnaire from 512 learners in Oman. The study found that personal innovativeness, PU and satisfaction of LMS in blended learning are significant to learners’ intention to engage in full e-learning. Thus, learners’ adoption of LMS in blended learning boosts their intention to full e-learning. The results provide useful insights for practitioners and researchers on full e-learning planning and strategy.     

Framework to improve e-learner satisfaction and further strengthen e-learning implementation

In this study, a framework has been designed to guide institutions to better improve learner satisfaction and further strengthen their e-learning implementation. Undergraduate participants (n = 600) completed an online survey of 132 items. This article will first report on the development and validation of an instrument that attempts to reveal factors that affect user satisfaction, and then a multiple regression analysis and a path analysis help further investigate which factors can significantly predict learner satisfaction. The factor analysis identified 14 different factors. These factors were further categorized by the researchers into 6 dimensions i.e. learner dimension, instructor’s dimension, course dimension, technology dimension, design dimension, and the environment dimension. The multiple regression analysis showed that e-learners satisfaction can mostly be predicted by learner interaction with others. Findings of this research will help institutions by providing them with psychometric properties that add pedagogical value to e-courses.     

Assessing the usability of a visual tool for the definition of e-learning processes

In this paper, we present a usability study aiming at assessing a visual language-based tool for developing adaptive e-learning processes. The tool implements the adaptive self-consistent learning object SET (ASCLO-S) visual language, a special case of flow diagrams, to be used by instructional designers to define classes of learners through stereotypes and to specify the more suited adaptive learning process for each class of learners. The usability study is based on the combined use of two techniques: a questionnaire-based survey and an empirical analysis. The survey has been used to achieve feedbacks from the subjects’ point of view. In particular, it has been useful to capture the perceived usability of the subjects. The outcomes show that both the proposed visual notation and the system prototype are suitable for instructional designers with or without experience on the computer usage and on tools for defining e-learning processes. This result is further confirmed by the empirical analysis we carried out by analysing the correlation between the effort to develop adaptive e-learning processes and some measures suitable defined for those processes. Indeed, the empirical analysis revealed that the effort required to model e-learning processes is not influenced by the experience of the instructional designer with the use of e-learning tools, but it only depends on the size of the developed process.     

Performance measurement of e-learning using student satisfaction analysis

The purpose of this paper is to analyze e-learning system quality through the analysis of student satisfaction and the usage of learning materials. This analysis takes into consideration both online and traditional students who are using the same e-learning system. The goal of the analysis is to identify factors which influence student satisfaction and to address heterogeneous styles and needs of both groups of students, so that future pedagogical and motivational methods in teaching and learning can be appropriately selected, developed and implemented. It was of particular interest to explore student satisfaction with quality of an e-learning system and online study approach. Reasons that may influence opinions of online and traditional students are examined and presented. The results are used to give recommendations for e-learning improvements and to propose the model with 4 groups of dimensions for performance measurement each of which best represents satisfaction of both groups of students.     

Validation of an evaluation model for learning management systems

Abstract   This study aims to validate a model for evaluating learning management systems (LMS) used in e-learning fields. A survey of 163 e-learning experts, regarding 81 validation items developed through literature review, was used to ascertain the importance of the criteria. A concise list of explanatory constructs, including two principle factors, was extracted from the responses of 133 respondents by exploratory factor analysis. Factor I was labeled as 'instruction management, screen design, and technology' and factor II, 'interaction and evaluation'. Factors I and II consisted of four and three sub-factors, respectively. Confirmatory factor analysis, in turn, was carried out to assess the degree to which the plausibility of factor models was empirically confirmed. Overall fit measures (RMSEA, CFI, NNFI indexes) showed the model to be quite suitable for LMS evaluation. Reliability estimates using Cronbach's alpha indicated that the evaluation tool had good internal consistency. The overall results of the analyses suggest that this evaluation tool is highly instrumental for LMS.     

Experience with usability evaluation of e-learning systems

Progress in the field of e-learning has been slow, with related problems mainly associated with the poor design of e-learning systems. Moreover, because of a depreciated importance of usability, usability studies are not very frequent. This paper reports the experience with the usability assessment of intelligent learning and teaching systems which are based on TEx-Sys model and are intended to enhance the process of knowledge acquisition in daily classroom settings. The applied scenario-based usability evaluation, as a combination of behaviour and opinion based measurements, enabled to quantify usability in terms of users’ (teachers’ and students’) performance and satisfaction. According to the achieved results, the main directions for interface redesign are offered. The acquired experience indicates that useful usability assessments with a significant identification of interface limitations can be performed quite easily and quickly. On the other hand, it raised a series of questions which, in order to be clarified, require further comprehensive research, the more so if the employment of universal design within e-learning context is considered.     

Designing a semantic bliki system to support different types of knowledge and adaptive learning

Though blogs and wikis have been used to support knowledge management and e-learning, existing blogs and wikis cannot support different types of knowledge and adaptive learning. A case in point, types of knowledge vary greatly in category and viewpoints. Additionally, adaptive learning is crucial to improving one’s learning performance. This study aims to design a semantic bliki system to tackle such issues. To support various types of knowledge, this study has developed a new social software called “bliki” that combines the advantages of blogs and wikis. This bliki system also applies Semantic Web technology to organize an ontology and a variety of knowledge types. To aid adaptive learning, a function called “Book” is provided to enable learners to arrange personalized learning goals and paths. The learning contents and their sequences and difficulty levels can be specified according to learners’ metacognitive knowledge and collaborative activities. An experiment is conducted to evaluate this system and the experimental results show that this system is able to comprehend various types of knowledge and to improve learners’ learning performance.     

Investigating university student preferences and beliefs about learning in the web-based context

Psychological studies have shown that personal beliefs about learning and environmental preferences affect learning behaviors. However, these learner characteristics have not been widely discussed in the web-based context. By developing questionnaires, this study attempted to detect learners’ web-based learning environmental preferences (WLEP) and beliefs about web-based learning (BWL). The scope of WLEP focused on the pedagogical dimension of the web-based learning environment, while BWL concerned the attributes and control factors of the web-based learning. There were about five hundreds of Taiwan university students participating in the study. Through factor analysis, the scales discussed in the study revealed a satisfactory validity and reliability in assessing students’ preferences and beliefs. Further analyses showed that university students preferred more of individual and structured instructional configurations while expected the outward mode of interaction. In general, students held a rather contextual belief about web-based learning, which was found to be correlated with their environmental preferences.