Multi-dimensional students’ evaluation of e-learning systems in the higher education context: An empirical investigation

There has been little research on assessment of learning management systems (LMS) within educational organizations as both a web-based learning system for e-learning and as a supportive tool for blended learning environments. This study proposes a conceptual e-learning assessment model, hexagonal e-learning assessment model (HELAM) suggesting a multi-dimensional approach for LMS evaluation via six dimensions: (1) system quality, (2) service quality, (3) content quality, (4) learner perspective, (5) instructor attitudes, and (6) supportive issues. A survey instrument based on HELAM has been developed and applied to 84 learners. This sample consists of students at both undergraduate and graduate levels who are users of a web-based learning management system, U-Link, at Brunel University, UK. The survey instrument has been tested for content validity, reliability, and criterion-based predictive validity. The analytical results strongly support the appropriateness of the proposed model in evaluating LMSs through learners’ satisfaction. The explanatory factor analysis showed that each of the six dimensions of the proposed model had a significant effect on the learners’ perceived satisfaction. Findings of this research will be valuable for both academics and practitioners of e-learning systems.     

Multi-dimensional students’ evaluation of e-learning systems in the higher education context: An empirical investigation

There has been little research on assessment of learning management systems (LMS) within educational organizations as both a web-based learning system for e-learning and as a supportive tool for blended learning environments. This study proposes a conceptual e-learning assessment model, hexagonal e-learning assessment model (HELAM) suggesting a multi-dimensional approach for LMS evaluation via six dimensions: (1) system quality, (2) service quality, (3) content quality, (4) learner perspective, (5) instructor attitudes, and (6) supportive issues. A survey instrument based on HELAM has been developed and applied to 84 learners. This sample consists of students at both undergraduate and graduate levels who are users of a web-based learning management system, U-Link, at Brunel University, UK. The survey instrument has been tested for content validity, reliability, and criterion-based predictive validity. The analytical results strongly support the appropriateness of the proposed model in evaluating LMSs through learners’ satisfaction. The explanatory factor analysis showed that each of the six dimensions of the proposed model had a significant effect on the learners’ perceived satisfaction. Findings of this research will be valuable for both academics and practitioners of e-learning systems.     

Using a cognition-motivation-control view to assess the adoption intention for Web-based learning

People expect Web technology to facilitate learning, particularly in higher education. A key issue involves the factors motivating the adoption of the Web for learning. Drawing upon social cognitive theory (SCT) and the theory of planned behavior (TPB), this study adopts a cognition-motivation-control view to assess learner adoption intentions for Web-based learning. The proposed model is validated by surveying 319 undergraduate students who had enrolled in Web-based courses and attended a 12-hour training program on using a Web-based system for academic learning. The empirical findings identified that efficacy control and efficacy expectations can be used to guide learner adaptation learning behaviors on the Web. The limitations of this study are discussed and future research directions suggested.     

基于SCORM标准的远程教育通信模型研究

学习管理系统（learning management system,LMS）的互操作性（interoperability）是当今远程教育研究的重要领域,实现LMS系统互操作性的关键是LMS系统和CBT（computer based training）课件之间的通信机制。在分析和研究基于SCORM标准的LMS／CBT远程教育通信模型的基础上,提出了基于SCORM标准实现学习资源标准化的解决方案。这一方案已在中国工商银行的远程教育实践中得到成功应用,对研究学习资源的标准化建设工作有一定的参考意义。     

Sharing instructors experience of learning management system: A technology perspective of user satisfaction in distance learning course

The use of educational technology increased rapidly in higher education. Learning Management System (LMS) is the most popular educational technology system used in distance learning. There are only a few studies have been carried out to measure instructors satisfaction in distance learning courses, although instructors satisfaction is considered as very important for the course involvement and increasing the students interactions with the course content. Hence, this study proposed a detailed framework to measure instructors’ satisfaction of using LMS. The findings prove that perceived usefulness and service quality are taking the highest share on affecting the instructor satisfactions. This study limited to higher education’s instructors and used a questionnaire survey to collect the data. Hence, the LMS should be designed based on the needs of the instructors as well as the students, by adopting the latest technologies. In the contrary, building LMS without taking the instructors’ satisfaction into account will affect negatively the distance learning course outcomes.     

Open‐source learning management systems: a predictive model for higher education

The present study investigated the role of pedagogical, technical, and institutional profile factors in an institution of higher education's decision to select an open‐source learning management system (LMS). Drawing on the results of previous research that measured patterns of deployment of open‐source software (OSS) in US higher education and the relative importance of specific selection criteria when considering an OSS LMS, a Web survey was conducted among 285 Chief Information Officers and Chief Academic Officers from a variety of US institutions to determine patterns of OSS LMS deployment, as well as the influence of specific selection criteria. Binary logistic regression analysis revealed Carnegie Classification, institutional governance, previous experience with OSS, focus on student learning, and the commitment to organizational self‐reliance to be key influencers in the OSS LMS selection decision. The implications of these findings for future research and for institutional decision‐making are discussed.     

Interoperability of Learning Objects Copyright in the LUISA Semantic Learning Management System

Abstract

Semantic Web technology is able to provide the required computational semantics for interoperability of learning resources across different Learning Management Systems (LMS) and Learning Object Repositories (LOR). The EU research project LUISA (Learning Content Management System Using Innovative Semantic Web Services Architecture) addresses the development of a reference semantic architecture for the major challenges in the search, interchange, and delivery of learning objects in a service-oriented context. One of the key issues, highlighted in this paper, is Digital Rights Management (DRM) interoperability. A Semantic Web approach to copyright management has been followed, which places a Copyright Ontology as the key component for interoperability among existing DRM systems and other licensing schemes like Creative Commons. Moreover, Semantic Web tools like reasoners, rule engines and semantic queries facilitate the implementation of an interoperable copyright management component in the LUISA architecture.     

The impact of organisational support,technical support,and self-efficacy on faculty perceived benefits of using learning management system

This paper examines how organisational support influences learning management system (LMS) self-efficacy, technical support, and faculty-perceived benefits. An empirical study based upon responses from 379 instructors at several universities is conducted. Structural equation modelling is applied to develop and assess the measurement model, and analyse the relationships among the factors in the structural model. The results support the hypotheses that organisational support plays a primary role in enhancing faculty’s LMS self-efficacy and technical support. In turn, improved LMS self-efficacy and technical support lead to faculty-perceived benefits from using LMS. Implications are that universities can increase the use of LMS and achieve more effective outcomes from faculty for web-based distance learning and web-assisted course curricula by structuring their organisations to better support faculty in both technical and self-efficacy areas.     

Negotiation based adaptive learning sequences: Combining adaptivity and adaptability

This study proposes a negotiation-based approach to combine the notion of adaptivity (system-controlled adaptation) and adaptability (user-controlled adaptation) for an adaptive learning system. The system suggests adaptations and the student also submits his/her adaptation preference. When the student preference opposes the system suggestion, the student then negotiates with the system to reach an agreement of adaptation. A negotiation-based adaptive learning system (NALS) is implemented to support the generation of personalized adaptive learning sequences by system negotiations with students regarding assessments of learning performance (i.e. negotiated open student model) of the current content and choices of the next learning content (i.e. negotiation of adaptation). Students require two metacognitions in deciding adaptive learning sequences: self-assessment for evaluating their understanding of the current content and regulation for choosing appropriate learning content. Negotiated open student model are used for assist student self-assessment and negotiation of adaptation are used for assist student regulation of content choices. An experiment was conducted to compare a system-controlled adaptive learning system (SALS, adaptivity), a user-controlled adaptive learning system (UALS, adaptability), and a NALS. The results revealed that NALS promoted better metacognitions in student calibration (i.e. self-assessment) accuracy and learning content choices (i.e. regulation). Preliminary evidences also showed that NALS promoted better student performance in a delay test. The results further suggested that students with poor calibration accuracy and inappropriate content choices were not suitable to use UALS and were suitable to use SALS. The NALS can also be used for training students to make appropriate adaptation for learning.     

Development and evaluation of a Web 2.0 annotation system as a learning tool in an e-learning environment

The emergence of Web 2.0 technology provides more opportunities to foster online communication and sharing in an e-learning environment. The purpose of this study was to develop a Web 2.0 annotation system, MyNote, based on the Web 2.0 core concepts which emphasize ease of access and active sharing and then to gain an understanding about people’s perceptions of MyNote from a usability perspective. MyNote was employed on multimedia learning objects in a Learning Management System (LMS), and out of the LMS as well in this study. The evaluation results showed that, with factor analysis, interactivity, usefulness, helpfulness, and willingness for future use were categorized to represent the perceptions of MyNote. It was also found that the factors of interactivity and helpfulness were statistically significant to predict the future use of MyNote. Lastly, the habit of taking notes also affected learners’ perceptions of using MyNote.     

Original source tracing enabled by e-learning contents system based on crowdsourcing

A crowdsourcing environment, where there is a very large volume of diverse content resulting from the participation of a mass of unspecified individuals, has resulted in significant changes in education. This paper presents an e-learning content system to manage the inclusion of crowdsourced material on the Web within lecture materials. The e-learning content system comprises a scrape system, learning content editor, and tracing system. As Web content may change with the progress of time, teachers (and students) must check whether the Web-based materials previously used in their classes have been updated. Accordingly, we designed scrape metadata specifications for tracing the original source. These metadata include information on copyrights and tracing, rather than basic data regarding the original source, to allow users to determine whether the original source has been updated. An editor was also configured so that the scraped Web content could be immediately incorporated into the teaching materials for enhanced convenience. The change point tracing accuracy test and utility evaluation performed using this system show that the accuracy of the change point tracing was 97.1% and that this system effectively saves time as compared with checking for changes by entering each URL directly.     

Developing early warning systems to predict students’ online learning performance

An early warning system can help to identify at-risk students, or predict student learning performance by analyzing learning portfolios recorded in a learning management system (LMS). Although previous studies have shown the applicability of determining learner behaviors from an LMS, most investigated datasets are not assembled from online learning courses or from whole learning activities undertaken on courses that can be analyzed to evaluate students’ academic achievement. Previous studies generally focus on the construction of predictors for learner performance evaluation after a course has ended, and neglect the practical value of an “early warning” system to predict at-risk students while a course is in progress. We collected the complete learning activities of an online undergraduate course and applied data-mining techniques to develop an early warning system. Our results showed that, time-dependent variables extracted from LMS are critical factors for online learning. After students have used an LMS for a period of time, our early warning system effectively characterizes their current learning performance. Data-mining techniques are useful in the construction of early warning systems; based on our experimental results, classification and regression tree (CART), supplemented by AdaBoost is the best classifier for the evaluation of learning performance investigated by this study.     

Informal learning through expertise mining in the social web

The advent of Web 2.0, also called the Social Web, has changed the way people interact with the Web. Assisted by the technologies associated with this new trend, users now play a much more active role as content providers. This Web paradigm shift has also changed how companies operate and interact with their employees, partners and customers. The challenge for companies and research institutions is now to develop (semi-) automated tools for gathering usable and explicit knowledge from such content. With the aim of facilitating the achievement of such a challenge, in this work a platform architecture for informal learning, which is based on semantic technologies, is proposed. Such platform permits to perform expertise mining from Social Web-generated content. Given a topic of interest, the system carries out semantically enhanced operations on blog and microblog posts to identify experts in that specific topic area. The comprehensive evaluation of the tool has demonstrated very promising results and is also presented in this article.     

Enhancing Learning Objects with an Ontology-Based Memory

The reusability in learning objects has always been a hot issue. However, we believe that current approaches to e-Learning failed to find a satisfying answer to this concern. This paper presents an approach that enables capitalization of existing learning resources by first creating "content metadatardquo through text mining and natural language processing and second by creating dynamically learning knowledge objects, i.e., active, adaptable, reusable, and independent learning objects. The proposed model also suggests integrating explicitly instructional theories in an on-the-fly composition process of learning objects. Semantic Web technologies are used to satisfy such an objective by creating an ontology-based organizational memory able to act as a knowledge base for multiple training environments.     

Enhancing formal e‐learning with edutainment on social networks

This paper reports on the investigation of the possibilities of enhancing the formal e‐learning process by harnessing the potential of informal game‐based learning on social networks. The goal of the research is to improve the outcomes of the formal learning process through the design and implementation of an educational game on a social network and its integration with a learning management system (LMS) of an educational institution. As a proof of concept, a Facebook educational game that enables students to learn and test their knowledge was developed. Furthermore, the game was integrated with the Moodle LMS and the evaluation was performed within the e‐learning system at the Faculty of Organizational Sciences, University of Belgrade. The results show that the application of social network edutainment in an e‐learning ecosystem has a positive impact on both the students' results and their satisfaction with the learning process as a whole.     

Intelligent Web-based education system for adaptive learning

This paper depicts a set of integrated tools to build an intelligent Web-based education system. Our purpose is to create a Web learning environment that can be tailored to the Learners’ needs. The Web learning environment is composed of Authoring Tool, Evaluation System, Interactive Voice System and a Virtual Laboratory for programming in Java. All tools use Web Services and have the characteristics of powerful adaptability for the management, authoring, delivery and monitoring of learning content. Part of the decision-making inside the intelligent Web-based education system was made with a multi-agent system.     

Web content adaptation to improve server overload behavior

This paper presents a study of Web content adaptation to improve server overload performance, as well as an implementation of a Web content adaptation software prototype. When the request rate on a Web server increases beyond server capacity, the server becomes overloaded and unresponsive. The TCP listen queue of the server's socket overflows exhibiting a drop-tail behavior. As a result, clients experience service outages. Since clients typically issue multiple requests over the duration of a session with the server, and since requests are dropped indiscriminately, all clients connecting to the server at overload are likely to experience connection failures, even though there may be enough capacity on the server to deliver all responses properly for a subset of clients. In this paper, we propose to resolve the overload problem by adapting delivered content to load conditions to alleviate overload. The premise is that successful delivery of a less resource intensive content under overload is more desirable to clients than connection rejection or failures.The paper suggests the feasibility of content adaptation from three different viewpoints; (a) potential for automating content adaptation with minimal involvement of the content provider, (b) ability to achieve sufficient savings in resource requirements by adapting present-day Web content while preserving adequate information, and (c) feasibility to apply content adaptation technology on the Web with no modification to existing Web servers, browsers or the HTTP protocol.     

Web service adaptation: A decade’s overview

With the exponential growth of communication and information technologies, adaptation has gained a significant attention as it becomes a key feature of service-based systems, allowing them to operate and evolve in highly dynamic and uncertain environments. Although several Web service standards and frameworks have been proposed and extended, existing solutions do not provide a suitable architecture, in which all aspects of monitoring and adaptation (e.g., proactive, cross-layer, and autonomic adaptation) can be expressed. In addition, the emergence of new computing environments to host and execute various types of services (Web/cloud services, big data-intensive services, mobile services, microservices, etc.) raises the need for more efficient monitoring and adaptation systems. This survey aims to bring a synthesis and a road-map to the adaptation of service-based systems. We also discuss adaptation solutions in emerging service models, such as cloud services and big services. Based on an adaptation taxonomy which we extracted from the surveyed approaches, and by identifying the main requirements and goals of service adaptation in Web, cloud and big data environments, detailed analysis and discussions, as well as the open issues, are provided.     

FuzzyQoI model: A fuzzy logic-based modelling of users' quality of interaction with a learning management system under blended learning

Learning management systems (LMSs) in higher education institutions (HEIs) provide the potential for rich learning environments built on social constructivist theories under the concept of blended (b-)learning. An essential factor, however, in determining the efficacy of online teaching–learning is the users' quality of interaction (QoI) with LMSs; yet, in many cases, QoI has not been properly acquired, mainly, due to its inherent qualitative character. Stemming from the latter, this study introduces a new model, namely FuzzyQoI, that, by employing fuzzy logic constructs, it quantitatively estimates the users' (professors' and students') QoI with the LMS Moodle within a b-learning environment. In the FuzzyQoI model, a set of 110 LMS Moodle metrics is considered to form 12 codified inputs to a five-level fuzzy inference system equipped with 600 expert's fuzzy rules. The potential and effectiveness of the realisation of the FuzzyQoI in practice are evaluated from its trialling on LMS data drawn from a large users' database (75 professors and 1037 students), referring to a five-course b-learning process of 51 weeks at a HEI. Experimental results have shown that the proposed FuzzyQoI model efficiently identified (dis)similarities in LMS interaction trends, correlations, distributions and dependencies with the time-period of the LMS use, both for the user-dependent and user-independent (group-like) cases. Consequently, the proposed FuzzyQoI functions as a means for better understanding and explaining the nature of underlying aspects, which influence the users' interaction behaviour under the LMS-based b-learning approach.     

基于Web注释的移动客户端自适应机制的研究

由于各种移动设备在计算能力上有所区别,目前没有一种较好的机制能够使Web内容根据移动设备的性能转换成不同页面大小。该文给出一种通过注释Web页面的方法来实现自适应移动设备,在不改动原有网站的基础上根据移动设备的计算能力生成合适的Web内容。与其它方法相比,该方法在设备自适应性、用户订制和转换速度等方面有较大的提高。