Computer adaptive practice of Maths ability using a new item response model for on the fly ability and difficulty estimation

In this paper we present a model for computerized adaptive practice and monitoring. This model is used in the Maths Garden, a web-based monitoring system, which includes a challenging web environment for children to practice arithmetic. Using a new item response model based on the Elo (1978) rating system and an explicit scoring rule, estimates of the ability of persons and the difficulty of items are updated with every answered item, allowing for on the fly item calibration. In the scoring rule both accuracy and response time are accounted for. Items are sampled with a mean success probability of .75, making the tasks challenging yet not too difficult. In a period of ten months our sample of 3648 children completed over 3.5 million arithmetic problems. The children completed about 33% of these problems outside school hours. Results show better measurement precision, high validity and reliability, high pupil satisfaction, and many interesting options for monitoring progress, diagnosing errors and analyzing development.     

Personalized e-learning system using Item Response Theory

Personalized service is important on the Internet, especially in Web-based learning. Generally, most personalized systems consider learner preferences, interests, and browsing behaviors in providing personalized services. However, learner ability usually is neglected as an important factor in implementing personalization mechanisms. Besides, too many hyperlink structures in Web-based learning systems place a large information burden on learners. Consequently, in Web-based learning, disorientation (losing in hyperspace), cognitive overload, lack of an adaptive mechanism, and information overload are the main research issues. This study proposes a personalized e-learning system based on Item Response Theory (PEL-IRT) which considers both course material difficulty and learner ability to provide individual learning paths for learners. The item characteristic function proposed by Rasch with a single difficulty parameter is used to model the course materials. To obtain more precise estimation of learner ability, the maximum likelihood estimation (MLE) is applied to estimate learner ability based on explicit learner feedback. Moreover, to determine an appropriate level of difficulty parameter for the course materials, this study also proposes a collaborative voting approach for adjusting course material difficulty. Experiment results show that applying Item Response Theory (IRT) to Web-based learning can achieve personalized learning and help learners to learn more effectively and efficiently.     

Adaptive item‐based learning environments based on the item response theory: possibilities and challenges

The popularity of intelligent tutoring systems (ITSs) is increasing rapidly. In order to make learning environments more efficient, researchers have been exploring the possibility of an automatic adaptation of the learning environment to the learner or the context. One of the possible adaptation techniques is adaptive item sequencing by matching the difficulty of the items to the learner's knowledge level. This is already accomplished to a certain extent in adaptive testing environments, where the test is tailored to the person's ability level by means of the item response theory (IRT). Even though IRT has been a prevalent computerized adaptive test (CAT) approach for decades and applying IRT in item‐based ITSs could lead to similar advantages as in CAT (e.g. higher motivation and more efficient learning), research on the application of IRT in such learning environments is highly restricted or absent. The purpose of this paper was to explore the feasibility of applying IRT in adaptive item‐based ITSs. Therefore, we discussed the two main challenges associated with IRT application in such learning environments: the challenge of the data set and the challenge of the algorithm. We concluded that applying IRT seems to be a viable solution for adaptive item selection in item‐based ITSs provided that some modifications are implemented. Further research should shed more light on the adequacy of the proposed solutions.     

Elo-based learner modeling for the adaptive practice of facts

We investigate applications of learner modeling in a computerized adaptive system for practicing factual knowledge. We focus on areas where learners have widely varying degrees of prior knowledge. We propose a modular approach to the development of such adaptive practice systems: dissecting the system design into an estimation of prior knowledge, an estimation of current knowledge, and the construction of questions. We provide a detailed discussion of learner models for both estimation steps, including a novel use of the Elo rating system for learner modeling. We implemented the proposed approach in a system for practising geography facts; the system is widely used and allows us to perform evaluation of all three modules. We compare the predictive accuracy of different learner models, discuss insights gained from learner modeling, as well as the impact different variants of the system have on learners’ engagement and learning.     

基于项目反应理论自适应测试系统的设计实现

根据自适应学习的特点,对项目反应理论的参数模型进行分析,提出极大似然估算法计算学习者能力参数。并在分析自适应测试系统的体系结构及系统主要功能模块的基础上,构建该测试系统;并提出了测试系统中首题估算策略、信息间隔量估算策略,从而提高出题速度和计算速度;并根据学习者的实际测试情况得出难度系数和特质水平,实现了难度系数算法和学习者特征水平计算算法。在此基础上开发了一个基于计算机应用基础课程的自适应测试系统。     

Personalized curriculum sequencing utilizing modified item response theory for web-based instruction

Curriculum sequencing is an important research issue for Web-based instruction systems because no fixed learning pathway will be appropriate for all learners. Therefore, many researchers focused on developing e-learning systems with personalized learning mechanism to assist on-line Web-based learning and adaptively provide learning pathways. However, although most personalized systems consider learner preferences, interests and browsing behavior in providing personalized curriculum sequencing services, these systems usually neglect to consider whether learner ability and the difficulty level of the recommended courseware are matched to each other or not. Generally, inappropriate courseware leads to learner cognitive overload or disorientation during learning, thus reducing learning effect. Besides, the problem of concept continuity of learning pathways also needs to be considered while implementing personalized curriculum sequencing. Smoother learning pathways increase learning effect, avoiding unnecessarily difficult concepts. This paper presents a prototype of personalized Web-based instruction system (PWIS) based on the proposed modified Item Response Theory (IRT) to perform personalized curriculum sequencing through simultaneously considering courseware difficulty level, learner's ability and the concept continuity of learning pathways during learning. In the proposed modified IRT, the information function is revised to consider the concept continuity of learning pathway as well as considering the difficulty level of courseware and individual learner ability. Experiment results indicate that applying the proposed modified IRT for Web-based learning can construct suitable learning pathway to learners for personalized learning, and help them to learn more effectively.     

Reducing Item Exposure in Computerized Adaptive Testing Systems Using Automatic Item Generation

This paper studies the technics of reducing item exposure by utilizing automatic item generation methods. Known test item calibration method uses item parameter estimation with the statistical data, collected during examinees prior testing. Disadvantage of the mentioned item calibration method is the item exposure; when test items become familiar to the examinees. To reduce the item exposure, automatic item generation method is used, where item models are being constructed based on already calibrated test items without losing already estimated item parameters. A technic of item model extraction method from the already calibrated and therefore exposed test items described, which can be used by the test item development specialists to integrate automatic item generation principles with the existing testing applications.     

双重两步迭代估计及其应用

两参数Logisitic模型(2PLM)是一个使用较广的项目反应模型．用专业软件BILOG估计2PLM中未知项目参数时,随着样本容量上升其估计精度改善很小．为了解决这一问题,本文改变BILOG的估计方案中迭代算法的初值,即建议使用本文给出的双重两步迭代估计(DTIE)作为迭代初值,所得新的估计方案记为MMLE/EM(DTIE)．大量Monte Carlo模拟结果显示,在样本容量不小于2000时,使用MMLE/EM(DTIE)进行项目参数估计,其估计精度超过BILOG．     

一种快速收敛的变步长自适应谐波检测算法

为解决传统固定步长LMS自适应算法在电网谐波检测中存在的收敛速度和稳态误差之间的矛盾,本文提出了一种快速收敛的变步长自适应谐波检测算法。该算法以误差反馈信号、误差信号在总误差信号中所占的比率以及负载电流的相邻两个采样值之差的和作为自适应反馈量,并通过自适应反馈量的相干平均估计来控制步长的更新;同时对系统权值迭代公式进行改进提高收敛速度;并改传统的固定步长变化范围为时变范围,使步长变化更加平滑。该方法在负载突变的情况下有很好的跟踪性能,可有效的提高初始收敛速度、减小稳态失调。仿真分析及实验证明了该算法在谐波检测中的有效性和准确性。     

Modeling dichotomous item responses with free-knot splines

Item response theory (IRT) models are a class of generalized mixed effect (GME) models used by psychometricians to describe the response behavior of individuals to a set of categorically scored items. The typical assumptions of IRT are Unidimensionality(U) of the random effect; Conditional (or Local) Independence (CI), the item responses are independent given the random effect; and Monotonicity (M), the probability of a correct response is a non-decreasing function of the random effect. The simple parametric models available in the psychometric literature have proved to be too restrictive in many data sets. Non-parametric regression models are a powerful tool for the estimation of non-linear curves, and have been used in IRT as a flexible way to model the item response function. This paper develops a new method for the non-parametric estimation of item response functions based on reversible-jump Markov Chain Monte Carlo, and demonstrates the practicality of the method by examining two data sets.     

软件规模估算方法的研究

要软件规模估算是整个项目的基础,是项目成功的核心。目前流行的软件规模估算方法主要有：专家小组判断法、类比法、基于代理的估算法。该文对以上三种方法进行了初步探究,并在此基础上进一步分析了每种方法的优缺点及适用情况,最后提出了软件规模估算方法下一步可能的发展方向。     

一种新的基于神经网络的IRT项目参数估计模型

探讨了一种新的基于广义回归神经网络（GRNN）的IRT（项目反应理论）项目参数估计建模方法，着重介绍了如何建立网络的输出模式及利用Monte Carlo方法建立网络的输入模式，提出了多种对模型进行改进的方法。模拟实验表明，利用GRNN可以以任意精度拟合CTT（经典测验理论）参数统计值和IRT参数值间隐含的非线性关系。与其他方法进行的比较表明，在小样本情况下，该方法的参数估计误差更小。     

基于神经网络的项目参数估计方法

针对题库建设中项目参数估计的实际问题,提出了一种全新的基于神经网络的参数估计方法;并以二值记分的3PLM为项目反应理论模型,以广义回归神经网络为网络模型,根据Monte Cado实验法进行了模拟实验研究,最后将神经网络方法与传统的数理统计估计方法进行了比较.结果表明,在小样本测验情况下,神经网络方法具有一定的优势,尤其是当去掉对项目参数的先验概率分布的限制时,神经网络方法的优势更加明显,说明本文提出的方法具有一定的价值. ,     

计算机自适应考试的理论模型研究

阐述了教育测量的两种主要理论,即经典测试理论（Qassical Test Theory,CTT）和项目反应理论（Item Response Theory,IRT）。其中IRT是计算机自适应测试CAT的理论基础。介绍了IRT和CTT相比的优点和几种IRT模型,并以三参数Logistic模型为典型代表论述了Logistic模型的基本理论。     

Personalized web-based tutoring system based on fuzzy item response theory

With the rapid growth of computer and Internet technologies, e-learning has become a major trend in the computer assisted teaching and learning field. Previously, many researchers put effort into e-learning systems with personalized learning mechanism to aid on-line learning. However, most systems focus on using learner’s behaviors, interests, and habits to provide personalized e-learning services. These systems commonly neglect to consider if learner ability and the difficulty level of the recommended courseware are matched to each other. Frequently, unsuitable courseware causes learner’s cognitive overload or disorientation during learning. To promote learning effectiveness, our previous study proposed a personalized e-learning system based on Item response theory (PEL-IRT), which can consider both course material difficulty and learner ability evaluated by learner’s crisp feedback responses (i.e. completely understanding or not understanding answer) to provide personalized learning paths for individual learners. The PEL-IRT cannot estimate learner ability for personalized learning services according to learner’s non-crisp responses (i.e. uncertain/fuzzy responses). The main problem is that learner’s response is not usually belonging to completely understanding or not understanding case for the content of learned courseware. Therefore, this study developed a personalized intelligent tutoring system based on the proposed fuzzy item response theory (FIRT), which could be capable of recommending courseware with suitable difficulty levels for learners according to learner’s uncertain/fuzzy feedback responses. The proposed FIRT can correctly estimate learner ability via the fuzzy inference mechanism and revise estimating function of learner ability while the learner responds to the difficulty level and comprehension percentage for the learned courseware. Moreover, a courseware modeling process developed in this study is based on a statistical technique to establish the difficulty parameters of courseware for the proposed personalized intelligent tutoring system. Experiment results indicate that applying the proposed FIRT to web-based learning can provide better learning services for individual learners than our previous study, thus helping learners to learn more effectively.     

Alternating imputation posterior estimation of models with crossed random effects

Generalized linear mixed models or latent variable models for categorical data are difficult to estimate if the random effects or latent variables vary at non-nested levels, such as persons and test items. Clayton and Rasbash (1999) suggested an Alternating Imputation Posterior (AIP) algorithm for approximate maximum likelihood estimation. For item response models with random item effects, the algorithm iterates between an item wing in which the item mean and variance are estimated for given person effects and a person wing in which the person mean and variance are estimated for given item effects. The person effects used for the item wing are sampled from the conditional posterior distribution estimated in the person wing and vice versa. Clayton and Rasbash (1999) used marginal quasi-likelihood (MQL) and penalized quasi-likelihood (PQL) estimation within the AIP algorithm, but this method has been shown to produce biased estimates in many situations, so we use maximum likelihood estimation with adaptive quadrature. We apply the proposed algorithm to the famous salamander mating data, comparing the estimates with many other methods, and to an educational testing dataset. We also present a simulation study to assess performance of the AIP algorithm and the Laplace approximation with different numbers of items and persons and a range of item and person variances.     

基于XML的计算机自适应测试技术的应用研究

测试是远程教育系统的重要组成部分之一。计算机自适应测试具有测试精度高、速度快等优点,但目前在远程教育中应用的还较少。文中在对自适应测试理论的研究基础上,指出了自适应测试在应用中存在一些主要问题,提出了利用XML按照教育资源规范进行IRT试题描述的方法,解决了IRT题库的建设和共享问题。结合Java技术较好地解决了自适应测试中的计算量大的问题,最后简单地描述了系统的实现过程,并给出了系统的设计模式。     

An adaptive testing system for supporting versatile educational assessment

With the rapid growth of computer and mobile technology, it is a challenge to integrate computer based test (CBT) with mobile learning (m-learning) especially for formative assessment and self-assessment. In terms of self-assessment, computer adaptive test (CAT) is a proper way to enable students to evaluate themselves. In CAT, students are assessed through a process that uses item response theory (IRT), a well-founded psychometric theory. Furthermore, a large item bank is indispensable to a test, but when a CAT system has a large item bank, the test item selection of IRT becomes more tedious. Besides the large item bank, item exposure mechanism is also essential to a testing system. However, IRT all lack the above-mentioned points. These reasons have motivated the authors to carry out this study. This paper describes a design issue aimed at the development and implementation of an adaptive testing system. The system can support several assessment functions and different devices. Moreover, the researchers apply a novel approach, particle swarm optimization (PSO) to alleviate the computational complexity and resolve the problem of item exposure. Throughout the development of the system, a formative evaluation was embedded into an integral part of the design methodology that was used for improving the system. After the system was formally released onto the web, some questionnaires and experiments were conducted to evaluate the usability, precision, and efficiency of the system. The results of these evaluations indicated that the system provides an adaptive testing for different devices and supports versatile assessment functions. Moreover, the system can estimate students’ ability reliably and validly and conduct an adaptive test efficiently. Furthermore, the computational complexity of the system was alleviated by the PSO approach. By the approach, the test item selection procedure becomes efficient and the average best fitness values are very close to the optimal solutions.     

用于二维源搜索的梯度自适应极值搜索算法

基于正弦扰动的二维源极值搜索算法存在着适应性差和快速性与准确性相互制约的缺点。针对这一问题,提出一种基于梯度估计的参数自适应极值搜索算法,该算法在传统极值搜索算法基础上,通过三个历史采样点估计当前区域的梯度,并依据当前区域梯度值自适应调整反馈增益参数。此外,利用平均值理论对所提算法进行了理论分析和收敛性证明。不同环境下的仿真对比表明本方法提高了源搜索效率和对复杂梯度环境的适应性。     

Scalable continual quality control of formative assessment items in an educational digital library: an empirical study

An essential component of any library of online learning objects is assessment items, for example, homework, quizzes, and self-study questions. As opposed to exams, these items are formative in nature, as they help the learner to assess his or her own progress through the material. When it comes to quality control of these items, their formative nature poses additional challenges. e.g., there is no particular time interval in which learners interact with these items, learners come to these items with very different levels of preparation and seriousness, guessing generates noise in the data, and the numbers of items and learners can be several orders of magnitude larger than in summative settings. This empirical study aims to find a highly scalable mechanism for continual quality control of this class of digital content with a minimalist amount of additional metadata and transactional data, while taking into account also characteristics of the learners. In a subsequent evaluation of the model on a limited set of transactions, we find that taking into account the learner characteristic of ability improves the quality of item metadata, and in a comparison to Item Response Theory (IRT), we find that the developed model in fact performs slightly better in terms of predicting the outcome of formative assessment transactions, while never matching the performance of IRT on predicting the outcome of summative assessment.