“How do you know that I don’t understand?” A look at the future of intelligent tutoring systems

Many software systems would significantly improve performance if they could adapt to the emotional state of the user, for example if Intelligent Tutoring Systems (ITSs), ATM’s, ticketing machines could recognise when users were confused, frustrated or angry they could guide the user back to remedial help systems so improving the service. Many researchers now feel strongly that ITSs would be significantly enhanced if computers could adapt to the emotions of students. This idea has spawned the developing field of affective tutoring systems (ATSs): ATSs are ITSs that are able to adapt to the affective state of students. The term “affective tutoring system” can be traced back as far as Rosalind Picard’s book Affective Computing in 1997.This paper presents research leading to the development of Easy with Eve, an ATS for primary school mathematics. The system utilises a network of computer systems, mainly embedded devices to detect student emotion and other significant bio-signals. It will then adapt to students and displays emotion via a lifelike agent called Eve. Eve’s tutoring adaptations are guided by a case-based method for adapting to student states; this method uses data that was generated by an observational study of human tutors. This paper presents the observational study, the case-based method, the ATS itself and its implementation on a distributed computer systems for real-time performance, and finally the implications of the findings for Human Computer Interaction in general and e-learning in particular. Web-based applications of the technology developed in this research are discussed throughout the paper.     

Using non-verbal cues to (automatically) assess children’s performance difficulties with arithmetic problems

Intelligent tutoring systems often make use of students’ answers to adapt instruction or feedback on a task. In this paper, we explore the alternative possibility of adapting a system based on the perceived affective and cognitive state of a student. A system can potentially better adapt to the needs of each individual student by using non-verbal behavior. We used a new experimental paradigm inspired by ‘brain training’ software to collect primary school children’s answers to easy and difficult arithmetic problems and made audiovisual recordings of their answers. Adult observers rated these films on perceived difficulty level. Results showed that adults were able to correctly interpret children’s perceived level of difficulty, especially if they saw their face (compared to hearing their voice). They paid attention to features such as ‘looking away’, and ‘frowning’. Then we checked whether we could also automatically predict if the posed problem was either easy or difficult based on the first second of their response. This ‘thin-slice analysis’ could correctly predict the difficulty level in 71% of all cases. When trained on sufficiently many recordings, Adaptive Tutoring Systems should be able to detect children’s state and adapt the difficulty level of their learning materials accordingly.     

Implementation and performance evaluation of parameter improvement mechanisms for intelligent e-learning systems

In recent years, designing useful learning diagnosis systems has become a hot research topic in the literature. In order to help teachers easily analyze students’ profiles in intelligent tutoring system, it is essential that students’ portfolios can be transformed into some useful information to reflect the extent of students’ participation in the curriculum activity. It is observed that students’ portfolios seldom reflect students’ actual studying behaviors in the learning diagnosis systems given in the literature; we thus propose three kinds of learning parameter improvement mechanisms in this research to establish effective parameters that are frequently used in the learning platforms. The proposed learning parameter improvement mechanisms can calculate the students’ effective online learning time, extract the portion of a message in discussion section which is strongly related to the learning topics, and detect plagiarism in students’ homework, respectively. The derived numeric parameters are then fed into a Support Vector Machine (SVM) classifier to predict each learner’s performance in order to verify whether they mirror the student’s studying behaviors. The experimental results show that the prediction rate for the SVM classifier can be increased up to 35.7% in average after the inputs to the classifier are “purified” by the learning parameter improvement mechanisms. This splendid achievement reveals that the proposed algorithms indeed produce the effective learning parameters for commonly used e-learning platforms in the literature.     

Prediction of student’s mood during an online test using formula-based and neural network-based method

Building computerized mechanisms that will accurately, immediately and continually recognize a learner’s affective state and activate an appropriate response based on integrated pedagogical models is becoming one of the main aims of artificial intelligence in education. The goal of this paper is to demonstrate how the various kinds of evidence could be combined so as to optimize inferences about affective states during an online self-assessment test. A formula-based method has been developed for the prediction of students’ mood, and it was tested using data emanated from experiments made with 153 high school students from three different regions of a European country. The same set of data is analyzed developing a neural network method. Furthermore, the formula-based method is used as an input parameter selection module for the neural network method. The results vindicate to a great degree the formula-based method’s assumptions about student’s mood and indicate that neural networks and conventional algorithmic methods should not be in competition but complement each other for the development of affect recognition systems. Moreover, it becomes apparent that neural networks can provide an alternative for and improvements over tutoring systems’ affect recognition methods.     

The design of instructional multimedia in e-Learning: A Media Richness Theory-based approach

The rapid development of computer and Internet technologies has made e-Learning become an important learning method. There has been a considerable increase in the needs for multimedia instructional material in e-Learning recently as such content has been shown to attract a learner’s attention and interests. The multimedia content alone, however, does not necessarily result in significant positive learning performance and satisfaction. Moreover, it is expensive to design and develop multimedia instructional material. There is a lack of extant research to address the critical issue of how to develop effective multimedia instructional content that leads to desirable learning performance and satisfaction. The objective of our paper is to propose and empirically test a model that examines the impact of the fitness of instructional content and media on a learner’s performance and satisfaction.     

A BDI approach to infer student’s emotions in an intelligent learning environment

In this article we describe the use of mental states approach, more specifically the belief-desire-intention (BDI) model, to implement the process of affective diagnosis in an educational environment. We use the psychological OCC model, which is based on the cognitive theory of emotions and is possible to be implemented computationally, in order to infer the learner’s emotions from his actions in the system interface. In our work we profit from the reasoning capacity of the BDI model in order to infer the student’s appraisal (a cognitive evaluation of a person that elicits an emotion), which allows us to deduce student’s emotions. The system reasons about an emotion-generating situation and tries to infer the user’s emotion by using the OCC model. Besides, the BDI model is very adequate to infer and also model students affective states since the emotions have a dynamic nature.     

The evaluation of an intelligent tutoring system

Abstract

Numerous intelligent tutoring systems (ITSs) have been developed to date. However, one aspect of their development which is consistently ignored, judging from the literature, is the crucial activity of evaluation. It is also ironical that researchers generally agree that the benefits of this activity could be far-reaching. ITSs are often described to some degree of detail but an appraisal as to their (potential) usefulness is seldom given. The relative novelty of doing this was the main motivation for this paper, which starts by attempting to shed some light on why evaluation has become such a taboo activity, not only in intelligent tutoring, but in artificial intelligence (AI) research in general. It then overviews a tutor, the fractions intelligent tutoring system (FITS), and reports on how it was appraised. The approaches used are neither ideal nor generally accepted, but may well provide an adequate starting point in the belief that an attempt at an honest evaluation of any sort is better than no evaluation at all.     

User modelling and user adapted interaction in an intelligent tutoring system

User modelling and user-adapted interaction are crucial to the provision of true individualised instruction, which intelligent tutoring systems strive to achieve. This paper presents how user (student) modelling and student adapted instruction is achieved in FITS, an intelligent tutoring system for the fractions domain. Some researchers have begun questioning both the need for detailed student models as well as the pragmatic possibility of building them. The key contributions of this paper are in its attempt to rehabilitate student modelling/adaptive tutoring within ITSs and in FITS's practical use of simple techniques to realise them with seemingly encouraging results; some illustrations are given to demonstrate the latter.     

Temporal Data Mining for Educational Applications

Intelligent tutoring systems (ITSs) acquire rich data about students' behavior during learning; data mining techniques can help to describe, interpret and predict student behavior, and to evaluate progress in relation to learning outcomes. This paper surveys a variety of data mining techniques for analyzing how students interact with ITSs, including methods for handling hidden state variables, and for testing hypotheses. To illustrate these methods we draw on data from two ITSs for math instruction. Educational datasets provide new challenges to the data mining community, including inducing action patterns, designing distance metrics, and inferring unobservable states associated with learning.     

Intelligent tutoring systems: an overview

This is a non-expert overview of Intelligent Tutoring Systems (ITSs), a way in which Artificial Intelligence (AI) techniques are being applied to education. It introduces ITSs and the motivation for them. It looks at its history: its evolution from Computer-Assisted Instruction (CAI). After looking at the structure of a typical ITS, the paper further examines and discusses some other architectures. Several classic ITSs are reviewed, mainly due to their historical significance or because they best demonstrate some of the principles of intelligent tutoring. A reasonably representative list of ITSs is also provided in order to provide a better appreciation of this vibrant field as well as reveal the scope of existing tutors. The paper concludes, perhaps more appropriately, with some of the author's viewpoints on a couple of controversial issues in the intelligent tutoring domain.     

TEx-Sys model for building intelligent tutoring systems

Special classes of asynchronous e-learning systems are the intelligent tutoring systems which represent an advanced learning and teaching environment adaptable to individual student’s characteristics. Authoring shells have an environment that enables development of the intelligent tutoring systems. In this paper we present, in entirety, for the first time, our approach to research, development and implementation related to intelligent tutoring systems and ITS authoring shells. Our research relies on the traditional intelligent tutoring system, the consideration that teaching is control of learning and principles of good human tutoring in order to develop the Tutor–Expert System model for building intelligent tutoring systems in freely chosen domain knowledge. In this way we can wrap up an ongoing process that has lasted for the previous fifteen years. Prototype tests with the implemented systems have been carried out with students from a primary education to an academic level. Results of those tests are advantageous, according to surveys, and the implemented and deployed software satisfies functionalities and actors’ demands.     

Experience beyond knowledge: Pragmatic e-learning systems design with learning experience

With the growing demand in e-learning system, traditional e-learning systems have dramatically evolved to provide more adaptive ways of learning, in terms of learning objectives, courses, individual learning processes, and so on. This paper reports on differences in learning experience from the learner’s perspectives when using an adaptive e-learning system, where the learner’s knowledge or skill level is used to configure the learning path. Central to this study is the evaluation of a dynamic content sequencing system (DCSS), with empirical outcomes being interpreted using Csikszentmihalyi’s flow theory (i.e., Flow, Boredom, and Anxiety). A total of 80 participants carried out a one-way between-subject study controlled by the type of e-learning system (i.e., the DCSS vs. the non-DCSS). The results indicated that the lower or medium achievers gained certain benefits from the DCSS, whilst the high achievers in learning performance might suffer from boredom when using the DCSS. These contrasting findings can be suggested as a pragmatic design guideline for developing more engaging computer-based learning systems for unsupervised learning situations.     

Towards an intelligent tutor for a complex mathematical domain

Abstract: The domain of mathematics has played a special part in the evolution of Intelligent Tutoring Systems (ITSs), beginning as far back as the 1950s when conventional Computer Assisted Instruction (CAI) came into being. A brief historical review of this evolutionary process is presented, followed by a selective survey of some of the intelligent tutoring systems in the mathematics domain as well as some of their shortcomings and the criticisms levelled against them. The work achieved so far towards the realisation of an intelligent tutoring system for a complex mathematical domain is presented. A major conclusion is that the 'mal-rule' methodology for developing tutoring systems may lend itself to simple 'primitive' domains, but its credibility is seriously questioned when it is applied to more 'complex' domains. An alternative methodology is proposed to solve this problem, together with an illustration.     

Adaptive item‐based learning environments based on the item response theory: possibilities and challenges

The popularity of intelligent tutoring systems (ITSs) is increasing rapidly. In order to make learning environments more efficient, researchers have been exploring the possibility of an automatic adaptation of the learning environment to the learner or the context. One of the possible adaptation techniques is adaptive item sequencing by matching the difficulty of the items to the learner's knowledge level. This is already accomplished to a certain extent in adaptive testing environments, where the test is tailored to the person's ability level by means of the item response theory (IRT). Even though IRT has been a prevalent computerized adaptive test (CAT) approach for decades and applying IRT in item‐based ITSs could lead to similar advantages as in CAT (e.g. higher motivation and more efficient learning), research on the application of IRT in such learning environments is highly restricted or absent. The purpose of this paper was to explore the feasibility of applying IRT in adaptive item‐based ITSs. Therefore, we discussed the two main challenges associated with IRT application in such learning environments: the challenge of the data set and the challenge of the algorithm. We concluded that applying IRT seems to be a viable solution for adaptive item selection in item‐based ITSs provided that some modifications are implemented. Further research should shed more light on the adequacy of the proposed solutions.     

Cluster-based predictive modeling to improve pedagogic reasoning

This paper discusses a predictive modeling framework actualized in a learning agent that uses logged tutorial interactions to discover predictive characteristics of students. The agent automatically forms cluster models that are described in terms of student–system interaction attributes, i.e., in terms of the student’s knowledge state and behaviour and system’s tutoring actions. The agent utilizes the knowledge of its various clusters together with a weighting scheme to learn predictive models of high-level student information, specifically, the time it will take the student to respond to a problem and whether the response is correct, that can be utilized to support individualized adaptation. We investigated utilizing the Self-Organizing Map and AutoClass as clustering algorithms and the naïve Bayesian classifier and single layer neural network as weighting algorithms. Empirical results show that by utilizing cluster knowledge the agent’s predictions are acceptably strong for response time and accurate at the average for response correctness. Further investigation is needed to validate the scalability of the framework given other datasets and possibly migrate to other approaches that can obtain more meaningful cluster models, detect richer attribute relations, and provide better approximations to further improve prediction of response behaviour for a more informed pedagogical decision-making by the system.     

Implement web learning environment based on data mining

The need for providing learners with web-based learning content that match their accessibility needs and preferences, as well as providing ways to match learning content to user’s devices has been identified as an important issue in accessible educational environment. For a web-based open and dynamic learning environment, personalized support for learners becomes more important. In order to achieve optimal efficiency in a learning process, individual learner’s cognitive learning style should be taken into account. Due to different types of learners using these systems, it is necessary to provide them with an individualized learning support system. However, the design and development of web-based learning environments for people with special abilities has been addressed so far by the development of hypermedia and multimedia based on educational content. In this paper a framework of individual web-based learning system is presented by focusing on learner’s cognitive learning process, learning pattern and activities, as well as the technology support needed. Based on the learner-centered mode and cognitive learning theory, we demonstrate an online course design and development that supports the students with the learning flexibility and the adaptability. The proposed framework utilizes data mining algorithm for representing and extracting a dynamic learning process and learning pattern to support students’ deep learning, efficient tutoring and collaboration in web-based learning environment. And experiments do prove that it is feasible to use the method to develop an individual web-based learning system, which is valuable for further study in more depth.     

A conversational intelligent tutoring system to automatically predict learning styles

This paper proposes a generic methodology and architecture for developing a novel conversational intelligent tutoring system (CITS) called Oscar that leads a tutoring conversation and dynamically predicts and adapts to a student’s learning style. Oscar aims to mimic a human tutor by implicitly modelling the learning style during tutoring, and personalising the tutorial to boost confidence and improve the effectiveness of the learning experience. Learners can intuitively explore and discuss topics in natural language, helping to establish a deeper understanding of the topic. The Oscar CITS methodology and architecture are independent of the learning styles model and tutoring subject domain. Oscar CITS was implemented using the Index of Learning Styles (ILS) model (Felder & Silverman, 1988) to deliver an SQL tutorial. Empirical studies involving real students have validated the prediction of learning styles in a real-world teaching/learning environment. The results showed that all learning styles in the ILS model were successfully predicted from a natural language tutoring conversation, with an accuracy of 61–100%. Participants also found Oscar’s tutoring helpful and achieved an average learning gain of 13%.     

A genetic fuzzy expert system for automatic question classification in a competitive learning environment

Intelligent tutoring systems are efficient tools to automatically adapt the learning process to the student’s progress and needs. One of the possible adaptations is to apply an adaptive question sequencing system, which matches the difficulty of the questions to the student’s knowledge level. In this context, it is important to correctly classify the questions to be presented to students according to their difficulty level. Many systems have been developed for estimating the difficulty of questions. However the variety in the application environments makes difficult to apply the existing solutions directly to other applications. Therefore, a specific solution has been designed in order to determine the difficulty level of open questions in an automatic and objective way. This solution can be applied to activities with special temporal and running features, as the contests developed through QUESTOURnament, which is a tool integrated into the e-learning platform Moodle. The proposed solution is a fuzzy expert system that uses a genetic algorithm in order to characterize each difficulty level. From the output of the algorithm, it defines the fuzzy rules that are used to classify the questions. Data registered from a competitive activity in a Telecommunications Engineering course have been used in order to validate the system against a group of experts. Results show that the system performs successfully. Therefore, it can be concluded that the system is able to do the questions classification labour in a competitive learning environment.     

Mobile educational features in authoring tools for personalised tutoring

One important field where mobile technology can make significant contributions is Education. In the fast pace of modern life, students and instructors would appreciate using constructively some spare time that they may have, in order to work on lessons at any place, even when away from offices, classrooms and labs where computers are usually located. In this paper, we describe a mobile authoring tool that we have developed and is called Mobile Author. Mobile Author can be used by human instructors either from a computer or a mobile phone to create their own Intelligent Tutoring Systems (ITSs) and to distribute them to their students. After the ITSs have been created, students can also use any computer or mobile phone to have access to theory and tests. The tutoring systems can assess the students' performance, inform the data-bases that record the students' progress and provide advice adapted to the needs of individual students. Finally, instructors can monitor their students' progress and communicate with their students during the course. The mobile features of both the authoring tool itself and the resulting ITSs from it have been evaluated by instructors and students, respectively. The results of the evaluation showed that mobile features are indeed considered useful.     

Gaze tutor: A gaze-reactive intelligent tutoring system

We developed an intelligent tutoring system (ITS) that aims to promote engagement and learning by dynamically detecting and responding to students' boredom and disengagement. The tutor uses a commercial eye tracker to monitor a student's gaze patterns and identify when the student is bored, disengaged, or is zoning out. The tutor then attempts to reengage the student with dialog moves that direct the student to reorient his or her attentional patterns towards the animated pedagogical agent embodying the tutor. We evaluated the efficacy of the gaze-reactive tutor in promoting learning, motivation, and engagement in a controlled experiment where 48 students were tutored on four biology topics with both gaze-reactive and non-gaze-reactive (control condition) versions of the tutor. The results indicated that: (a) gaze-sensitive dialogs were successful in dynamically reorienting students’ attentional patterns to the important areas of the interface, (b) gaze-reactivity was effective in promoting learning gains for questions that required deep reasoning, (c) gaze-reactivity had minimal impact on students’ state motivation and on self-reported engagement, and (d) individual differences in scholastic aptitude moderated the impact of gaze-reactivity on overall learning gains. We discuss the implications of our findings, limitations, future work, and consider the possibility of using gaze-reactive ITSs in classrooms.