Bayesian knowledge tracing,logistic models,and beyond: an overview of learner modeling techniques

Learner modeling is a basis of personalized, adaptive learning. The research literature provides a wide range of modeling approaches, but it does not provide guidance for choosing a model suitable for a particular situation. We provide a systematic and up-to-date overview of current approaches to tracing learners’ knowledge and skill across interaction with multiple items, focusing in particular on the widely used Bayesian knowledge tracing and logistic models. We discuss factors that influence the choice of a model and highlight the importance of the learner modeling context: models are used for different purposes and deal with different types of learning processes. We also consider methodological issues in the evaluation of learner models and their relation to the modeling context. Overall, the overview provides basic guidelines for both researchers and practitioners and identifies areas that require further clarification in future research.     

Activity sequence modelling and dynamic clustering for personalized e-learning

Monitoring and interpreting sequential learner activities has the potential to improve adaptivity and personalization within educational environments. We present an approach based on the modeling of learners?? problem solving activity sequences, and on the use of the models in targeted, and ultimately automated clustering, resulting in the discovery of new, semantically meaningful information about the learners. The approach is applicable at different levels: to detect pre-defined, well-established problem solving styles, to identify problem solving styles by analyzing learner behaviour along known learning dimensions, and to semi-automatically discover learning dimensions and concrete problem solving patterns. This article describes the approach itself, demonstrates the feasibility of applying it on real-world data, and discusses aspects of the approach that can be adjusted for different learning contexts. Finally, we address the incorporation of the proposed approach in the adaptation cycle, from data acquisition to adaptive system interventions in the interaction process.     

Learner modeling for adaptive scaffolding in a Computational Thinking-based science learning environment

Learner modeling has been used in computer-based learning environments to model learners’ domain knowledge, cognitive skills, and interests, and customize their experiences in the environment based on this information. In this paper, we develop a learner modeling and adaptive scaffolding framework for Computational Thinking using Simulation and Modeling (CTSiM)—an open ended learning environment that supports synergistic learning of science and Computational Thinking (CT) for middle school students. In CTSiM, students have the freedom to choose and coordinate use of the different tools provided in the environment, as they build and test their models. However, the open-ended nature of the environment makes it hard to interpret the intent of students’ actions, and to provide useful feedback and hints that improves student understanding and helps them achieve their learning goals. To address this challenge, we define an extended learner modeling scheme that uses (1) a hierarchical task model for the CTSiM environment, (2) a set of strategies that support effective learning and model building, and (3) effectiveness and coherence measures that help us evaluate student’s proficiency in the different tasks and strategies. We use this scheme to dynamically scaffold learners when they are deficient in performing their tasks, or they demonstrate suboptimal use of strategies. We demonstrate the effectiveness of our approach in a classroom study where one group of 6th grade students received scaffolding and the other did not. We found that students who received scaffolding built more accurate models, used modeling strategies effectively, adopted more useful modeling behaviors, showed a better understanding of important science and CT concepts, and transferred their modeling skills better to new scenarios.     

Automatic detection of learning styles for an e-learning system

A desirable characteristic for an e-learning system is to provide the learner the most appropriate information based on his requirements and preferences. This can be achieved by capturing and utilizing the learner model. Learner models can be extracted based on personality factors like learning styles, behavioral factors like user’s browsing history and knowledge factors like user’s prior knowledge. In this paper, we address the problem of extracting the learner model based on Felder–Silverman learning style model. The target learners in this problem are the ones studying basic science. Using NBTree classification algorithm in conjunction with Binary Relevance classifier, the learners are classified based on their interests. Then, learners’ learning styles are detected using these classification results. Experimental results are also conducted to evaluate the performance of the proposed automated learner modeling approach. The results show that the match ratio between the obtained learner’s learning style using the proposed learner model and those obtained by the questionnaires traditionally used for learning style assessment is consistent for most of the dimensions of Felder–Silverman learning style.     

Item difficulty estimation: An auspicious collaboration between data and judgment

The evolution from static to dynamic electronic learning environments has stimulated the research on adaptive item sequencing. A prerequisite for adaptive item sequencing, in which the difficulty of the item is constantly matched to the ability level of the learner, is to have items with a known difficulty level. The difficulty level can be estimated by means of the item response theory (IRT). However, the requirement of a large sample size for calibrating items based on IRT models is not easily met in many practical learning situations. The aim of this paper is to search for relatively simple and fast alternative estimation methods and to review the accuracy of these methods as compared to IRT-based calibration in one single setting, and this for various sample sizes. Using real data, six alternative estimation methods are compared next to IRT-based calibration: proportion correct, learner feedback, expert rating, one-to-many comparison (learner), one-to-many comparison (expert) and the Elo rating system. Results indicate that proportion correct has the strongest relation with IRT-based difficulty estimates, followed by learner feedback, the Elo rating system, expert rating and finally one-to-many comparison. Learner feedback and one-to-many comparison (learner) provide stable estimates even with a small sample size. IRT, proportion correct and the Elo rating system provide reliable estimates, especially with a sample size of 200-250 learners. The alternative estimation methods can be utilized for adaptive item sequencing when IRT-based calibration does not yet provide reliable estimates or can be used as a prior in a Bayesian estimation method.     

Group online adaptive learning

Sharing information among multiple learning agents can accelerate learning. It could be particularly useful if learners operate in continuously changing environments, because a learner could benefit from previous experience of another learner to adapt to their new environment. Such group-adaptive learning has numerous applications, from predicting financial time-series, through content recommendation systems, to visual understanding for adaptive autonomous agents. Here we address the problem in the context of online adaptive learning. We formally define the learning settings of Group Online Adaptive Learning and derive an algorithm named Shared Online Adaptive Learning (SOAL) to address it. SOAL avoids explicitly modeling changes or their dynamics, and instead shares information continuously. The key idea is that learners share a common small pool of experts, which they can use in a weighted adaptive way. We define group adaptive regret and prove that SOAL maintains known bounds on the adaptive regret obtained for single adaptive learners. Furthermore, it quickly adapts when learning tasks are related to each other. We demonstrate the benefits of the approach for two domains: vision and text. First, in the visual domain, we study a visual navigation task where a robot learns to navigate based on outdoor video scenes. We show how navigation can improve when knowledge from other robots in related scenes is available. Second, in the text domain, we create a new dataset for the task of assigning submitted papers to relevant editors. This is, inherently, an adaptive learning task due to the dynamic nature of research fields evolving in time. We show how learning to assign editors improves when knowledge from other editors is available. Together, these results demonstrate the benefits for sharing information across learners in concurrently changing environments.

     

Personalized e-learning system using Item Response Theory

Personalized service is important on the Internet, especially in Web-based learning. Generally, most personalized systems consider learner preferences, interests, and browsing behaviors in providing personalized services. However, learner ability usually is neglected as an important factor in implementing personalization mechanisms. Besides, too many hyperlink structures in Web-based learning systems place a large information burden on learners. Consequently, in Web-based learning, disorientation (losing in hyperspace), cognitive overload, lack of an adaptive mechanism, and information overload are the main research issues. This study proposes a personalized e-learning system based on Item Response Theory (PEL-IRT) which considers both course material difficulty and learner ability to provide individual learning paths for learners. The item characteristic function proposed by Rasch with a single difficulty parameter is used to model the course materials. To obtain more precise estimation of learner ability, the maximum likelihood estimation (MLE) is applied to estimate learner ability based on explicit learner feedback. Moreover, to determine an appropriate level of difficulty parameter for the course materials, this study also proposes a collaborative voting approach for adjusting course material difficulty. Experiment results show that applying Item Response Theory (IRT) to Web-based learning can achieve personalized learning and help learners to learn more effectively and efficiently.     

ADAPTIVE DECISION SUPPORT SYSTEM FOR AUTOMATIC PHYSICAL EFFORT PLAN GENERATION—DATA-DRIVEN APPROACH

Mathematical models delivered using both expert knowledge and experimental data improve understanding of dynamic properties of the system under consideration. This is useful for different purposes, such as prediction, diagnosis, decision making, and system control. A data-driven approach has been found to be particularly useful in designing adaptive decision support systems. We demonstrate the usefulness of data-driven models in a custom application designed for sport training management. We have developed a system that makes use of expert knowledge together with measurement data (heart rate, electromyography, and acceleration) as well as environmental (Global Positioning System) in order to generate an optimal training plan. The system performs such tasks as modeling of the athlete's cardiovascular system, estimation of the athlete's parameters, and adaptation of the model to the athlete.     

Personalizing the Interaction in a Web-based Educational Hypermedia System: the case of INSPIRE

In this paper we present an Adaptive Educational Hypermedia prototype, named INSPIRE. The approach employed in INSPIRE emphasizes the fact that learners perceive and process information in very different ways, and integrates ideas from theories of instructional design and learning styles. Our aim is to make a shift towards a more 'learning-focused' paradigm of instruction by providing a sequence of authentic and meaningful tasks that matches learner' preferred way of studying. INSPIRE, throughout its interaction with the learner, dynamically generates learner-tailored lessons that gradually lead to the accomplishment of learner's learning goals. It supports several levels of adaptation: from full system-control to full learner-control, and offers learners the option to decide on the level of adaptation of the system by intervening in different stages of the lesson generation process and formulating the lesson contents and presentation. Both the adaptive and adaptable behavior of INSPIRE are guided by the learner model which provides information about the learner, such as knowledge level on the domain concepts and learning style. The learner model is exploited in multiple ways: curriculum sequencing, adaptive navigation support, adaptive presentation, and supports system's adaptable behavior. An empirical study has been performed to evaluate the adaptation framework and assess learners' attitudes towards the proposed instructional design.     

TAGUS — A user and learner modeling workbench

In this paper we will describe, outline and exemplify the functionalities and architecture of a User and Learner Modeling System called TAGUS (within the project Theory and Applications for General User/Learner-modeling Systems).TAGUS was developed with two main goals: (1) to develop a framework to represent models of users and learners where the meta-cognitive activities of learners were taken into account; and (2) to try to capture in a system some general mechanisms and techniques for user and learner modeling in the form of services.The basic idea of TAGUS is to achieve a kind of workbench where some techniques and approaches for user and learner modeling are implemented and applied. TAGUS provides a set of services, to be used by people testing methods or by applications using user models. These services, provided to external agents, embed some mechanisms for maintaining models of the users and learners. Thus, TAGUS plays a role of a user and learner modeling server.To achieve this goal, we first identified some basic mechanisms in user and learner modeling, and based on them we established a general modeling cycle. This cycle involves two main stages: the acquisition and the maintenance of the model. The different strategies and techniques are specified in separate modules or knowledge sources in TAGUS, which uses them to execute parts of that cycle. The architecture of TAGUS is composed of: a User or Learner Model (ULM); a set of maintenance functions; an acquisition engine; a reason maintenance system; a meta-reasoner and two interfaces.At the same time, TAGUS provides a language for defining the models of the users and learners, which can be used with different techniques, in order to test the models and simulate them in the system. This language is used not only to represent the models, but also as a way of establishing the communication between TAGUS and its environment.TAGUS was built incrementally around a set of core functions for the manipulation of the User or Learner Model (ULM). Other layers of this set were built where the last layer corresponds to the services TAGUS supplies.     

Deterministic approach to robust adaptive learning of fuzzy models

This study is concerned with the adaptive learning of an interpretable Sugeno-type fuzzy inference system, in a deterministic framework, in the presence of data uncertainties and modeling errors. The authors explore the use of H/sup /spl infin// estimation theory and least squares estimation for online learning of membership functions and consequent parameters without making any assumption and requiring a priori knowledge of upper bounds, statistics, and distribution of data uncertainties and modeling errors. The issues of data uncertainties, modeling errors, and time variations have been considered mathematically in a sensible way. The proposed robust approach to the adaptive learning of fuzzy models has been illustrated through the examples of adaptive system identification, time-series prediction, and estimation of an uncertain process.     

An attribute-based ant colony system for adaptive learning object recommendation

Teachers usually have a personal understanding of what “good teaching” means, and as a result of their experience and educationally related domain knowledge, many of them create learning objects (LO) and put them on the web for study use. In fact, most students cannot find the most suitable LO (e.g. learning materials, learning assets, or learning packages) from webs. Consequently, many researchers have focused on developing e-learning systems with personalized learning mechanisms to assist on-line web-based learning and to adaptively provide learning paths. However, although most personalized learning mechanism systems neglect to consider the relationship between learner attributes (e.g. learning style, domain knowledge) and LO’s attributes. Thus, it is not easy for a learner to find an adaptive learning object that reflects his own attributes in relationship to learning object attributes. Therefore, in this paper, based on an ant colony optimization (ACO) algorithm, we proposed an attributes-based ant colony system (AACS) to help learners find an adaptive learning object more effectively. Our paper makes three critical contributions: (1) It presents an attribute-based search mechanism to find adaptive learning objects effectively; (2) An attributes-ant algorithm was proposed; (3) An adaptive learning rule was developed to identify how learners with different attributes may locate learning objects which have a higher probability of being useful and suitable; (4) A web-based learning portal was created for learners to find the learning objects more effectively.     

Towards new forms of knowledge communication: the adaptive dimension of a web-based learning environment

Adaptive Educational Hypermedia Systems aim to increase the functionality of hypermedia by making it personalised to individual learners. The adaptive dimension of these systems mainly supports knowledge communication between the system and the learner by adapting the content or the appearance of hypermedia to the knowledge level, goals and other characteristics of each learner. The main objectives are to protect learners from cognitive overload and disorientation by supporting them to find the most relevant content and path in the hyperspace. In the approach presented in this paper, learners' knowledge level and individual traits are used as valuable information to represent learners' current state and personalise the educational system accordingly, in order to facilitate learners to achieve their personal learning goals and objectives. Learners' knowledge level is approached through a qualitative model of the level of performance that learners exhibit with respect to the concepts they study and is used to adapt the lesson contents and the navigation support. Learners' individual traits and especially their learning style represent the way learners perceive and process information, and are exploited to adapt the presentation of the educational material of a lesson. The proposed approach has been implemented through various adaptation technologies and incorporated into a prototype hypermedia system. Finally, a pilot study has been conducted to investigate system's educational effectiveness.     

Design and development of an innovative individualized adaptive and intelligent e-learning system for teaching–learning of probability unit: Details of UZWEBMAT

In this study, an innovative adaptive and intelligent web based e-learning system, UZWEBMAT (Turkish abbreviation of Adaptive and INtelligent WEB based MAThematics teaching–learning system) was designed, developed and implemented. This e-learning system was intended for learning and teaching secondary school level permutation-combination-binomial expansion and probability subjects. Content which was prepared according to Turkish curriculum for secondary school mathematics course was transformed into learning objects in three different ways in accordance with VAK (Visual–Auditory–Kinesthetic) learning styles. Primary/secondary/tertiary learning styles of learners registering the system are determined and each learner receives the content appropriate for his/her dominant learning style. Also, they can be directed to contents of other styles according to their performances thanks to an expert system. Learning objects constituting the content were prepared according to constructivist approach. An active role for the learner was the purpose. Tips and intelligent solution supports within the learning objects were presented with expert system support to the learners. With this structure, UZWEBMAT bears the characteristics of intelligent tutoring system as well as an adaptive e-learning environment. All the movements of learners studying with UZWEBMAT are recorded and the necessary information is reported to both learners and teachers in a visualized way.     

Monitoring Online Tests through Data Visualization

We present an approach and a system to let tutors monitor several important aspects related to online tests, such as learner behavior and test quality. The approach includes the logging of important data related to learner interaction with the system during the execution of online tests and exploits data visualization to highlight information useful to let tutors review and improve the whole assessment process. We have focused on the discovery of behavioral patterns of learners and conceptual relationships among test items. Furthermore, we have led several experiments in our faculty in order to assess the whole approach. In particular, by analyzing the data visualization charts, we have detected several previously unknown test strategies used by the learners. Last, we have detected several correlations among questions, which gave us useful feedbacks on the test quality.     

一种基于自适应证据推理规则的集成学习方法

当前集成学习中的结合策略难以兼顾各个基学习器之间的信息和模型的可解释性。使用证据推理（evidential reasoning,ER）规则作为结合策略,将各个基学习器结果作为证据参与融合,可以较好地解决以上问题。但传统ER规则的证据参数是单一的,对不同的基学习器模型使用相同的证据参数显然是不合理的。为此,提出一种基于自适应证据推理（adaptive-evidential reasoning,A-ER）规则的集成学习方法,该方法在每次证据融合前对证据的类别进行判断,针对不同的证据类别自适应分配不同的证据参数。通过不同的分类案例表明,该方法与案例中其他方法相比具有更高的分类精度,证明了该方法使证据参数设置更加合理且具有更好的可解释性和泛化能力。     

Feature-subspace aggregating: ensembles for stable and unstable learners

This paper introduces a new ensemble approach, Feature-Subspace Aggregating (Feating), which builds local models instead of global models. Feating is a generic ensemble approach that can enhance the predictive performance of both stable and unstable learners. In contrast, most existing ensemble approaches can improve the predictive performance of unstable learners only. Our analysis shows that the new approach reduces the execution time to generate a model in an ensemble through an increased level of localisation in Feating. Our empirical evaluation shows that Feating performs significantly better than Boosting, Random Subspace and Bagging in terms of predictive accuracy, when a stable learner SVM is used as the base learner. The speed up achieved by Feating makes feasible SVM ensembles that would otherwise be infeasible for large data sets. When SVM is the preferred base learner, we show that Feating SVM performs better than Boosting decision trees and Random Forests. We further demonstrate that Feating also substantially reduces the error of another stable learner, k-nearest neighbour, and an unstable learner, decision tree.     

Adaptive testing for hierarchical student models

This paper presents an approach to student modeling in which knowledge is represented by means of probability distributions associated to a tree of concepts. A diagnosis procedure which uses adaptive testing is part of this approach. Adaptive tests provide well-founded and accurate diagnosis thanks to the underlying probabilistic theory, i.e., the Item Response Theory. Most adaptive testing proposals are based on dichotomous models, where he student answer can only be considered either correct or incorrect. In the work described here, a polytomous model has been used, i.e., answers can be given partial credits. Thus, models are more informative and diagnosis is more efficient. This paper also presents an algorithm for estimating question characteristic curves, which are necessary in order to apply the Item Response Theory to a given domain and hence must be inferred before testing begins. Most prior estimation procedures need huge sets of data. We have modified preexisting procedures in such a way that data requirements are significantly reduced. Finally, this paper presents the results of some controlled evaluations that have been carried out in order to analyze the feasibility and advantages of this approach. This paper (or a similar version) is not currently under review by a journal or conference, nor will it be submitted to such within the next three months.