COMET: A Collaborative Tutoring System for Medical Problem-Based Learning

This paper discussed about the developed collaborative intelligent tutoring system for medical PBL called Comet (collaborative medical tutor). Comet uses Bayesian networks to model the knowledge and activity of individual students as well as small groups. It applies generic tutoring algorithms to these models and generates tutorial hints that guide problem solving. An early laboratory study shows a high degree of agreement between the hints generated by Comet and those of experienced human tutors. Evaluations of Comet's clinical-reasoning model and the group reasoning path provide encouraging support for the general framework.     

Modeling individual and collaborative problem-solving in medical problem-based learning

Today a great many medical schools have turned to a problem-based learning (PBL) approach to teaching as an alternative to traditional didactic medical education to teach clinical-reasoning skills at the early stages of medical education. While PBL has many strengths, effective PBL tutoring is time-intensive and requires the tutor to provide a high degree of personal attention to the students, which is difficult in the current academic environment of increasing demands on faculty time. This paper describes the student modeling approach used in the COMET intelligent tutoring system for collaborative medical PBL. To generate appropriate tutorial actions, COMET uses a model of each student’s clinical reasoning for the problem domain. In addition, since problem solving in group PBL is a collaborative process, COMET uses a group model that enables it to do things like focus the group discussion, promote collaboration, and suggest peer helpers. Bayesian networks are used to model individual student knowledge and activity, as well as that of the group. The validity of the modeling approach has been tested with student models in the areas of head injury, stroke, and heart attack. Receiver operating characteristic (ROC) curve analysis shows that the models are highly accurate in predicting individual student actions. Comparison with human tutors shows that the focus of group activity determined by the model agrees with that suggested by the majority of the human tutors with a high degree of statistical agreement (McNemar test, p = 0.774, Kappa = 0.823).     

Exploring feedback and student characteristics relevant for personalizing feedback strategies

Personalized tutoring feedback is a powerful method that expert human tutors apply when helping students to optimize their learning. Thus, research on tutoring feedback strategies tailoring feedback according to important factors of the learning process has been recognized as a promising issue in the field of computer-based adaptive educational technologies. Our paper seeks to contribute to this area of research by addressing the following aspects: First, to investigate how students' gender, prior knowledge, and motivational characteristics relate to learning outcomes (knowledge gain and changes in motivation). Second, to investigate the impact of these student characteristics on how tutoring feedback strategies varying in content (procedural vs. conceptual) and specificity (concise hints vs. elaborated explanations) of tutoring feedback messages affect students' learning and motivation. Third, to explore the influence of the feedback parameters and student characteristics on students' immediate post-feedback behaviour (skipping vs. trying to accomplish a task, and failing vs. succeeding in providing a correct answer). To address these issues, detailed log-file analyses of an experimental study have been conducted. In this study, 124 sixth and seventh graders have been exposed to various tutoring feedback strategies while working on multi-trial error correction tasks in the domain of fraction arithmetic. The web-based intelligent learning environment ActiveMath was used to present the fraction tasks and trace students' progress and activities. The results reveal that gender is an important factor for feedback efficiency: Male students achieve significantly lower knowledge gains than female students under all tutoring feedback conditions (particularly, under feedback strategies starting with a conceptual hint). Moreover, perceived competence declines from pre- to post-test significantly more for boys than for girls. Yet, the decline in perceived competence is not accompanied by a decline in intrinsic motivation, which, instead, increases significantly from pre- to post-test. With regard to the post-feedback behaviour, the results indicate that students skip further attempts more frequently after conceptual than after procedural feedback messages.     

Searching for the two sigma advantage: Evaluating algebra intelligent tutors

This study evaluated 2 off-the-shelf, computer-based, mathematics intelligent-tutoring systems that provide instruction in algebra during a remedial mathematics summer program. The majority of the enrolled high school students failed to pass algebra in the previous semester. Students were randomly assigned in approximately equal proportions to work with the Carnegie Learning Algebra Cognitive Tutor or the ALEKS Algebra Course. Using the tutoring system exclusively, the students completed a 4-h-a-day, 14-day summer school high school algebra class for credit. The results revealed that both tutoring systems produced statistically and practically meaningful learning gains on measures of arithmetic and algebra knowledge.     

Productive use of learning resources in an online problem-based learning environment

This study examined students’ use of learning resources in a technologically-mediated online learning environment. Undergraduate student groups were engaged in an online problem-based learning (PBL) environment, rich with pre-selected video and knowledge resources. Quantitative and qualitative analyses showed that students accessed resources fairly frequently and benefited from them. Resources helped students construct a rich understanding of the problem and provided ideas for problem solutions. Detailed analyses of resource exploration along with contrasting case analyses between high-achieving and low-achieving student groups suggested that for learning to be effective in resource-rich environments, students first need to develop an understanding of the resources and learn how to access them efficiently. Second, students need to learn to process the contents of resources in meaningful ways so that they can integrate diverse resources to form a coherent understanding and apply them to solve problems. Finally, students need to develop knowledge and skills to use resources collaboratively, such as sharing and relating to each other’s resources. The results indicated that students, especially low-achieving students, need guidance to use resources effectively in resource-rich learning environments.     

Interactive reconstructive student modeling: A machine‐learning approach

Reconstructive bug modeling is a well‐known approach to student modeling in intelligent tutoring systems, suitable for modeling procedural tasks. Domain knowledge is decomposed into the set of primitive operators and the set of conditions of their applicability. Reconstructive modeling is capable of describing errors that come from irregular application of correct operators. The main obstacle to successfulness of this approach is such decomposition of domain knowledge to primitive operators with a very low level of abstraction so that bugs could never occur within them. The other drawback of this modeling scheme is its efficiency because it is usually done offline, due to vast search spaces involved.

This article reports a novel approach to reconstructive modeling based on machine‐learning techniques for inducing procedures from traces. The approach overcomes the problems of reconstructive modeling by its interactive nature. It allows online model generation by using domain knowledge and knowledge about the student to focus the search on the portion of the problem space the student is likely to traverse while solving the problem. Furthermore, the approach is not only incremental, but also truly interactive because it involves the student in explicit dialogs about his or her goals. In such a way, it is possible to determine whether the student knows the operator he or she is trying to apply. Pedagogical actions and the student model are generated interchangeably, thus allowing for dynamic adaptation of instruction, problem generation, and immediate feedback on student's errors. The approach presented is examined in the context of the symbolic integration tutoring system (SINT), an intelligent tutoring system (ITS) for the domain of symbolic integration.     

Learning with intelligent tutors and worked examples: selecting learning activities adaptively leads to better learning outcomes than a fixed curriculum

The main learning activity provided by intelligent tutoring systems is problem solving, although several recent projects investigated the effectiveness of combining problem solving with worked examples. Previous research has shown that learning from examples is an effective learning strategy, especially for novice learners. A worked example provides step-by-step explanations of how a problem is solved. Many studies have compared learning from examples to unsupported problem solving, and suggested presenting worked examples to students in the initial stages of learning, followed by problem solving once students have acquired enough knowledge. This paper presents a study in which we compare a fixed sequence of alternating worked examples and tutored problem solving with a strategy that adapts learning tasks to students’ needs. The adaptive strategy determines the type of the task (a worked example, a faded example or a problem to be solved) based on how much assistance the student received on the previous problem. The results show that students in the adaptive condition learnt significantly more than their peers who were presented with a fixed sequence of worked examples and problem solving. Novices from the adaptive condition learnt faster than novices from the control group, while the advanced students from the adaptive condition learnt more than their peers from the control group.     

Incorporating tutoring principles into interactive knowledge acquisition

This paper argues that interactive knowledge acquisition systems would benefit from a tighter and more thorough incorporation of tutoring and learning principles. Current acquisition systems learn from users in a passive manner, and could instead be designed to incorporate the proactive capabilities that one expects of a good student. We first describe our analysis of the literature on teacher–student interaction and present a compilation of tutoring and learning principles that are relevant to interactive knowledge acquisition systems. We then point out what tutoring and learning principles have been used to date in the acquisition literature, though unintentionally and implicitly, and discuss how a more thorough and explicit representation of these principles would help improve how computers learn from users. We present our design and an initial implementation of an acquisition dialogue system called SLICK that represents acquisition principles and goals explicitly and declaratively, making the system actively reason about various acquisition tasks and generate its interactions dynamically. Finally, we discuss promising directions in designing acquisition systems by structuring interactions with users according to tutoring and learning principles.     

Complementary machine intelligence and human intelligence in virtual teaching assistant for tutoring program tracing

This study proposes a virtual teaching assistant (VTA) to share teacher tutoring tasks in helping students practice program tracing and proposes two mechanisms of complementing machine intelligence and human intelligence to develop the VTA. The first mechanism applies machine intelligence to extend human intelligence (teacher answers) to evaluate the correctness of student program tracing answers, to locate student errors, and to generate hints to indicate errors. The second mechanism applies machine intelligence to reuse human intelligence (previous hints that the teacher gave to other students in a similar error situation) to provide program-specific hints. Two evaluations were conducted with 85 and 64 participants, respectively. The evaluation results showed that the system helped above 89% of students correct their errors. The error-indicating hints generated by the first mechanism help students correct more than half of errors. Each teacher-generated hint was reused averagely three times by the second mechanism. The results also revealed that some error situations occurred frequently and occupied a major occurred percentage of student error situations. In sum, the VTA and these two mechanisms reduce teacher tutoring load and reduce the complexity of developing machine intelligence.     

Learning Java: a programmed instruction approach using Applets

A programmed instruction approach to knowledge acquisition is operationalized by student–teacher interactions that involve monitoring and managing the moment-by-moment progress of a learner throughout the process of achieving a criterion of mastery in a knowledge domain. The teacher is generic and may include another person, a structured text, or a computer. When the steps and increments leading to a task's completion and mastery can be enumerated, the process of interaction-based learning lends itself to implementation with computer-based tutoring systems. The present paper describes a computer-based programmed instruction tutoring system that teaches a learner how to write a simple Java™ computer program. The system is based on a series of Java Applets, which are computer programs that are downloaded from a network server and executed by a client browser such as Netscape®. The use of Java Applets makes the tutoring system available to learners over the World Wide Web. Performance acquisition data are presented to show an individual learner's progression to task completion and mastery using the tutoring system. Survey data are presented to show the subjective responses of a class of students to the use of the tutoring system. The adoption of this computer-based tutoring system is justified as one component within a personalized system of instruction that also includes lectures and collaborative learning experiences.     

基于改进微粒群优化的学习路径优化控制方法

对学生学习的路径控制在智能化教学系统中是一个重要的问题。该文以知识空间理论为基础建立了学习状态空间,通过改进的微粒群算法对该学习状态空间的学习路径进行最优化控制,并利用死亡惩罚函数法把约束最优化学习路径问题转化成了无约束的最优化学习路径控制问题,引入交换子和交换序的概念对微粒群算法进行改进。在结果分析中,通过动态参数法,即动态变化交换子保留概率的方法提高微粒群的收敛效果,达到了最优化学习路径控制的目的。     

Using Bayesian Networks to Manage Uncertainty in Student Modeling

When a tutoring system aims to provide students with interactive help, it needs to know what knowledge the student has and what goals the student is currently trying to achieve. That is, it must do both assessment and plan recognition. These modeling tasks involve a high level of uncertainty when students are allowed to follow various lines of reasoning and are not required to show all their reasoning explicitly. We use Bayesian networks as a comprehensive, sound formalism to handle this uncertainty. Using Bayesian networks, we have devised the probabilistic student models for Andes, a tutoring system for Newtonian physics whose philosophy is to maximize student initiative and freedom during the pedagogical interaction. Andes’ models provide long-term knowledge assessment, plan recognition, and prediction of students’ actions during problem solving, as well as assessment of students’ knowledge and understanding as students read and explain worked out examples. In this paper, we describe the basic mechanisms that allow Andes’ student models to soundly perform assessment and plan recognition, as well as the Bayesian network solutions to issues that arose in scaling up the model to a full-scale, field evaluated application. We also summarize the results of several evaluations of Andes which provide evidence on the accuracy of its student models.This revised version was published online in July 2005 with corrections to the author name VanLehn.     

A paradigm for handwriting-based intelligent tutors

This paper presents the interaction design of, and demonstration of technical feasibility for, intelligent tutoring systems that can accept handwriting input from students. Handwriting and pen input offer several affordances for students that traditional typing-based interactions do not. To illustrate these affordances, we present evidence, from tutoring mathematics, that the ability to enter problem solutions via pen input enables students to record algebraic equations more quickly, more smoothly (fewer errors), and with increased transfer to non-computer-based tasks. Furthermore our evidence shows that students tend to like pen input for these types of problems more than typing. However, a clear downside to introducing handwriting input into intelligent tutors is that the recognition of such input is not reliable. In our work, we have found that handwriting input is more likely to be useful and reliable when context is considered, for example, the context of the problem being solved. We present an intelligent tutoring system for algebra equation solving via pen-based input that is able to use context to decrease recognition errors by 18% and to reduce recognition error recovery interactions to occur on one out of every four problems. We applied user-centered design principles to reduce the negative impact of recognition errors in the following ways: (1) though students handwrite their problem-solving process, they type their final answer to reduce ambiguity for tutoring purposes, and (2) in the small number of cases in which the system must involve the student in recognition error recovery, the interaction focuses on identifying the student’s problem-solving error to keep the emphasis on tutoring. Many potential recognition errors can thus be ignored and distracting interactions are avoided. This work can inform the design of future systems for students using pen and sketch input for math or other topics by motivating the use of context and pragmatics to decrease the impact of recognition errors and put user focus on the task at hand.     

Gaze tutor: A gaze-reactive intelligent tutoring system

We developed an intelligent tutoring system (ITS) that aims to promote engagement and learning by dynamically detecting and responding to students' boredom and disengagement. The tutor uses a commercial eye tracker to monitor a student's gaze patterns and identify when the student is bored, disengaged, or is zoning out. The tutor then attempts to reengage the student with dialog moves that direct the student to reorient his or her attentional patterns towards the animated pedagogical agent embodying the tutor. We evaluated the efficacy of the gaze-reactive tutor in promoting learning, motivation, and engagement in a controlled experiment where 48 students were tutored on four biology topics with both gaze-reactive and non-gaze-reactive (control condition) versions of the tutor. The results indicated that: (a) gaze-sensitive dialogs were successful in dynamically reorienting students’ attentional patterns to the important areas of the interface, (b) gaze-reactivity was effective in promoting learning gains for questions that required deep reasoning, (c) gaze-reactivity had minimal impact on students’ state motivation and on self-reported engagement, and (d) individual differences in scholastic aptitude moderated the impact of gaze-reactivity on overall learning gains. We discuss the implications of our findings, limitations, future work, and consider the possibility of using gaze-reactive ITSs in classrooms.     

Student modeling approaches: A literature review for the last decade

This paper constitutes a literature review on student modeling for the last decade. The review aims at answering three basic questions on student modeling: what to model, how and why. The prevailing student modeling approaches that have been used in the past 10 years are described, the aspects of students’ characteristics that were taken into consideration are presented and how a student model can be used in order to provide adaptivity and personalisation in computer-based educational software is highlighted. This paper aims to provide important information to researchers, educators and software developers of computer-based educational software ranging from e-learning and mobile learning systems to educational games including stand alone educational applications and intelligent tutoring systems. In addition, this paper can be used as a guide for making decisions about the techniques that should be adopted when designing a student model for an adaptive tutoring system. One significant conclusion is that the most preferred technique for representing the student’s mastery of knowledge is the overlay approach. Also, stereotyping seems to be ideal for modeling students’ learning styles and preferences. Furthermore, affective student modeling has had a rapid growth over the past years, while it has been noticed an increase in the adoption of fuzzy techniques and Bayesian networks in order to deal the uncertainty of student modeling.     

Curriculum knowledge representation and manipulation inknowledge-based tutoring systems

A knowledge-based tutoring system (KBTS) is a computer-based instructional system that uses artificial intelligence techniques to help people learn some subjects. We found that the knowledge communication process involving a KBTS and a human student can be decomposed into a series of communication cycles, where each cycle concentrates on one topic and contains four major phases: planning, discussing, evaluating and remedying. The major contributions of this work are the development of a generic architecture for supporting the knowledge communication between a KBTS and a student, and a graphical notation and schema for supporting the curriculum knowledge representation and manipulation during the planning phase of a tutoring process. The curriculum knowledge about a course can help a tutoring system determine the sequences in which the topics will be discussed with the students effectively and diagnose the students' mistakes. The curriculum knowledge base contains the goal structure of the course, prerequisite relations, and multiple ways of organizing topics, among others. As an example, we focus on developing SQL-TUTOR, a KBTS for the domain of SQL programming. This system has features such as an efficient control mechanism, explicit curriculum knowledge representation, and individualized private tutoring. For allowing the students relative freedom to decide how to study the domain knowledge about a subject, the system provides the students with a group of operators to hand-tailor the learning schedules according to their special backgrounds, requests, and interests     

Interactive Correction and Recommendation for Computer Language Learning and Training

Active learning and training is a particularly effective form of education. In various domains, skills are equally important to knowledge. We present an automated learning and skills training system for a database programming environment that promotes procedural knowledge acquisition and skills training. The system provides meaningful knowledge-level feedback such as correction of student solutions and personalized guidance through recommendations. Specifically, we address automated synchronous feedback and recommendations based on personalized performance assessment. At the core of the tutoring system is a pattern-based error classification and correction component that analyzes student input in order to provide immediate feedback and in order to diagnose student weaknesses and suggest further study material. A syntax-driven approach based on grammars and syntax trees provides the solution for a semantic analysis technique. Syntax tree abstractions and comparison techniques based on equivalence rules and pattern matching are specific approaches.     

Supporting skill integration in an intelligent tutoring system for code tracing

Background

Skill integration is vital in students' mastery development and is especially prominent in developing code tracing skills which are foundational to programming, an increasingly important area in the current STEM education. However, instructional design to support skill integration in learning technologies has been limited.

Objectives

The current work presents the development and empirical evaluation of instructional design targeting students' difficulties in code tracing particularly in integrating component skills in the Trace Table Tutor (T3), an intelligent tutoring system.

Methods

Beyond the instructional features of active learning, step-level support, and individualized problem selection of intelligent tutoring systems (ITS), the instructional design of T3 (e.g., hints, problem types, problem selection) was optimized to target skill integration based on a domain model where integrative skills were represented as combinations of component skills. We conducted an experimental study in a university-level introductory Python programming course and obtained three findings.

Results and Conclusions

First, the instructional features of the ITS technology support effective learning of code tracing, as evidenced by significant learning gains (medium-to-large effect sizes). Second, performance data supports the existence of integrative skills beyond component skills. Third, an instructional design focused on integrative skills yields learning benefits beyond a design without such focus, such as improving performance efficiency (medium-to-large effect sizes).

Major Takeaways

Our work demonstrates the value of designing for skill integration in learning technologies and the effectiveness of the ITS technology for computing education, as well as provides general implications for designing learning technologies to foster robust learning.