Mobile formative assessment tool based on data mining techniques for supporting web-based learning

Current trends clearly indicate that online learning has become an important learning mode. However, no effective assessment mechanism for learning performance yet exists for e-learning systems. Learning performance assessment aims to evaluate what learners learned during the learning process. Traditional summative evaluation only considers final learning outcomes, without concerning the learning processes of learners. With the evolution of learning technology, the use of learning portfolios in a web-based learning environment can be beneficially adopted to record the procedure of the learning, which evaluates the learning performances of learners and produces feedback information to learners in ways that enhance their learning. Accordingly, this study presents a mobile formative assessment tool using data mining, which involves six computational intelligence theories, i.e. statistic correlation analysis, fuzzy clustering analysis, grey relational analysis, K-means clustering, fuzzy association rule mining and fuzzy inference, in order to identify the key formative assessment rules according to the web-based learning portfolios of an individual learner for the performance promotion of web-based learning. Restated, the proposed method can help teachers to precisely assess the learning performance of individual learner utilizing only the learning portfolios in a web-based learning environment. Hence, teachers can devote themselves to teaching and designing courseware, since they save a lot of time in measuring learning performance. More importantly, teachers can understand the main factors influencing learning performance in a web-based learning environment based on the interpretable learning performance assessment rules obtained. Experimental results indicate that the evaluation results of the proposed scheme are very close to those of summative assessment results and the factor analysis provides simple and clear learning performance assessment rules. Furthermore, the proposed learning feedback with formative assessment can clearly promote the learning performances and interests of learners.     

Task results processing for the needs of task-oriented design environments

This paper presents learners’ task results gathered by means of an example task-oriented environment for knowledge testing and processed by EXCEL. The processing is domain- and task-independent and includes automatic calculation of several important task and session’s parameters, drawing specific graphics, generating tables, and analyzing the correlation coefficient. The intention is to design and implement a specialized tool called postprocessor for support of a common task base, learner’s model and decision making of the author and teacher by means of an environment for individually planned teaching courseware.     

TEx-Sys model for building intelligent tutoring systems

Special classes of asynchronous e-learning systems are the intelligent tutoring systems which represent an advanced learning and teaching environment adaptable to individual student’s characteristics. Authoring shells have an environment that enables development of the intelligent tutoring systems. In this paper we present, in entirety, for the first time, our approach to research, development and implementation related to intelligent tutoring systems and ITS authoring shells. Our research relies on the traditional intelligent tutoring system, the consideration that teaching is control of learning and principles of good human tutoring in order to develop the Tutor–Expert System model for building intelligent tutoring systems in freely chosen domain knowledge. In this way we can wrap up an ongoing process that has lasted for the previous fifteen years. Prototype tests with the implemented systems have been carried out with students from a primary education to an academic level. Results of those tests are advantageous, according to surveys, and the implemented and deployed software satisfies functionalities and actors’ demands.     

Complementary machine intelligence and human intelligence in virtual teaching assistant for tutoring program tracing

This study proposes a virtual teaching assistant (VTA) to share teacher tutoring tasks in helping students practice program tracing and proposes two mechanisms of complementing machine intelligence and human intelligence to develop the VTA. The first mechanism applies machine intelligence to extend human intelligence (teacher answers) to evaluate the correctness of student program tracing answers, to locate student errors, and to generate hints to indicate errors. The second mechanism applies machine intelligence to reuse human intelligence (previous hints that the teacher gave to other students in a similar error situation) to provide program-specific hints. Two evaluations were conducted with 85 and 64 participants, respectively. The evaluation results showed that the system helped above 89% of students correct their errors. The error-indicating hints generated by the first mechanism help students correct more than half of errors. Each teacher-generated hint was reused averagely three times by the second mechanism. The results also revealed that some error situations occurred frequently and occupied a major occurred percentage of student error situations. In sum, the VTA and these two mechanisms reduce teacher tutoring load and reduce the complexity of developing machine intelligence.     

Providing adaptation and guidance for design learning by problem solving: The design planning approach in DomoSim-TPC environment

Experimental learning environments based on simulation usually require monitoring and adaptation to the actions the users carry out. Some systems provide this functionality, but they do so in a way which is static or cannot be applied to problem solving tasks. In response to this problem, we propose a method based on the use of intermediate languages to provide adaptation in design learning scenarios. Although we use some approaches which are familiar from other domains (e.g., programming tutors) they are novel as regards their application to a very different domain and as a result we have incorporated new strategies. The purpose of our proposal is to provide monitoring, guidance and adaptive features for PlanEdit, a tool for the learning of integral automation methods in buildings and housing by design. This tool is part of a collaborative environment, called DomoSim-TPC, which supports distance learning of domotical design. We have carried out an experiment to obtain some data which confirm that our position can be effective for group learning of domotical design, studying the relationship between the quantity of model work carried out and the errors made.     

A conversational intelligent tutoring system to automatically predict learning styles

This paper proposes a generic methodology and architecture for developing a novel conversational intelligent tutoring system (CITS) called Oscar that leads a tutoring conversation and dynamically predicts and adapts to a student’s learning style. Oscar aims to mimic a human tutor by implicitly modelling the learning style during tutoring, and personalising the tutorial to boost confidence and improve the effectiveness of the learning experience. Learners can intuitively explore and discuss topics in natural language, helping to establish a deeper understanding of the topic. The Oscar CITS methodology and architecture are independent of the learning styles model and tutoring subject domain. Oscar CITS was implemented using the Index of Learning Styles (ILS) model (Felder & Silverman, 1988) to deliver an SQL tutorial. Empirical studies involving real students have validated the prediction of learning styles in a real-world teaching/learning environment. The results showed that all learning styles in the ILS model were successfully predicted from a natural language tutoring conversation, with an accuracy of 61–100%. Participants also found Oscar’s tutoring helpful and achieved an average learning gain of 13%.     

Controlled experiment replication in evaluation of e-learning system’s educational influence

We believe that every effectiveness evaluation should be replicated at least in order to verify the original results and to indicate evaluated e-learning system’s advantages or disadvantages. This paper presents the methodology for conducting controlled experiment replication, as well as, results of a controlled experiment and an internal replication that investigated the effectiveness of intelligent authoring shell eXtended Tutor–Expert System (xTEx-Sys). The initial and the replicated experiment were based on our approach that combines classical two-group experimental design and with factoral design. A trait that distinguishes this approach from others is the existence of arbitrary number of checkpoint-tests to determine the effectiveness in intermediate states. We call it a pre-and-post test control group experimental design with checkpoint-tests. The gained results revealed small or even negative effect sizes, which could be explained by the fact that the xTEx-Sys’s domain knowledge presentation is rather novel for students and therefore difficult to grasp and apply in earlier phases of the experiment. In order to develop and improve the xTEx-Sys, further experiments must be conducted.     

Adaptive educational hypermedia accommodating learning styles: A content analysis of publications from 2000 to 2011

Implementing instructional interventions to accommodate learner differences has received considerable attention. Among these individual difference variables, the empirical evidence regarding the pedagogical value of learning styles has been questioned, but the research on the issue continues. Recent developments in Web-based implementations have led scholars to reconsider the learning style research in adaptive systems. The current study involved a content analysis of recent studies on adaptive educational hypermedia (AEH) which addressed learning styles. After an extensive search on electronic databases, seventy studies were selected and exposed to a document analysis. Study features were classified under several themes such as the research purposes, methodology, features of adaptive interventions and student modeling, and findings. The analysis revealed that the majority of studies proposed a framework or model for adaptivity whereas few studies addressed the effectiveness of learning style-based AEH. Scales were used for learning style identification more than automatic student modeling. One third of the studies provided a framework without empirical evaluation with students. Findings on concrete learning outcomes were not strong enough; however, several studies revealed that suggested models influenced student satisfaction and success. Current trends, potential research gaps and implications were discussed.     

Personalized multi-student improvement based on Bayesian cybernetics

This work presents innovative cybernetics (feedback) techniques based on Bayesian statistics for drawing questions from an Item Bank towards personalized multi-student improvement. A novel software tool, namely Module for Adaptive Assessment of Students (or, MAAS for short), implements the proposed (feedback) techniques. In conclusion, a pilot application to two Computer Science courses during a period of 4 years demonstrates the effectiveness of the proposed techniques. Statistical evidence strongly suggests that the proposed techniques can improve student performance. The benefits of automating a quicker delivery of University quality education to a large body of students can be substantial as discussed here.     

A web-based formative assessment tool for Masters students: A pilot study

The use of web-based learning and assessment tools is growing in tertiary institutions around the world. To date, very few papers have reported the development and evaluation of a web-based formative assessment tool for postgraduate students. The aim of the present paper was to report on the development and evaluation of an online formative assessment tool for this student group. The web-based formative assessment tool was evaluated by a sample of undergraduate students, postgraduate students and academic staff within a psychology department in order to determine the suitability and sensitivity of the tool. The results of this pilot test suggest that the development of such a tool is both appropriate and feasible for Masters students studying psychology.     

Collaborative robotic instruction: A graph teaching experience

Graphing is a key skill in the study of Physics. Drawing and interpreting graphs play a key role in the understanding of science, while the lack of these has proved to be a handicap and a limiting factor in the learning of scientific concepts. It has been observed that despite the amount of previous graph-working experience, students of all ages experience a series of difficulties when trying to comprehend graphs or when trying to relate them with physical concepts such as position, velocity and acceleration. Several computational tools have risen to improve the students’ understanding of kinematical graphs; however, these approaches fail to develop graph construction skills. On the other hand, Robots have opened new opportunities in learning. Nevertheless, most of their educational applications focus on Robotics related subjects, such as robot programming, robot construction, and artificial intelligence. This paper describes a robotic activity based on face-to-face computer supported collaborative learning. By means of a set of handhelds and a robot wirelessly interconnected, the aim of the activity is to develop graph construction and graph interpretation skills while also reinforcing kinematics concepts. Results show that students using the robotic activity achieve a significant increase in their graph interpreting skills. Moreover, when compared with a similar computer-simulated activity, it proved to be almost twice as effective. Finally, the robotic application proved to be a highly motivating activity for the students, fostering collaboration among them.     

Ability-training-oriented automated assessment in introductory programming course

Learning to program is a difficult process for novice programmers. AutoLEP, an automated learning and assessment system, was developed by us, to aid novice programmers to obtain programming skills. AutoLEP is ability-training-oriented. It adopts a novel assessment mechanism, which combines static analysis with dynamic testing to analyze student programs. It not only helps students to sufficiently test the programs, but also evaluates whether the programs meet the specification or not. AutoLEP encourages students to find and work through bugs, by providing automatic syntactic and structural checking, and immediate detailed feedback. This can improve students’ learning experience in programming and reduce the workload of the teaching staff. AutoLEP has been used in the C programming course at Harbin Institute of Technology and many other universities since 2004. The feedback on AutoLEP and its incorporation into the introductory programming course has been positive, both by students and teaching staff.     

Practical guidelines for designing and evaluating educationally oriented recommendations

There is a need for designing educationally oriented recommendations that deal with educational goals as well as learners' preferences and context in a personalised way. They have to be both based on educators' experience and perceived as adequate by learners. This paper compiles practical guidelines to produce personalised recommendations that are meant to foster active learning in online courses. These guidelines integrate three different methodologies: i) user centred design as defined by ISO 9241-210, ii) the e-learning life cycle of personalised educational systems, and iii) the layered evaluation of adaptation features. To illustrate guidelines actual utility, generality and flexibility, the paper describes their applicability to design educational recommendations in two different contexts, which in total involved 125 educators and 595 learners. These applications show benefits for learners and educators. Following this approach, we are targeting to cope with one of the main challenges in current massive open online courses, which are expected to provide personalised education to an increasing number of students without the continuous involvement of educators in supporting learners during their course interactions.