Strategies for controlling item exposure in computerized adaptive testing with the partial credit model

Exposure control research with polytomous item pools has determined that randomization procedures can be very effective for controlling test security in computerized adaptive testing (CAT). The current study investigated the performance of four procedures for controlling item exposure in a CAT under the partial credit model. In addition to a no exposure control baseline condition, the Kingsbury-Zara, modified-within-.10-logits, Sympson-Hetter, and conditional Sympson-Hetter procedures were implemented to control exposure rates. The Kingsbury-Zara and the modified-within-.10-logits procedures were implemented with 3 and 6 item candidate conditions. The results show that the Kingsbury-Zara and modified-within-.10-logits procedures with 6 item candidates performed as well as the conditional Sympson-Hetter in terms of exposure rates, overlap rates, and pool utilization. These two procedures are strongly recommended for use with partial credit CATs due to their simplicity and strength of their results.     

Distractors with information in multiple choice items: a rationale based on the Rasch model

There is a substantial literature on attempts to obtain information on the proficiency of respondents from distractors in multiple choice items. Information in a distractor implies that a person who chooses that distractor has greater proficiency than if the person chose another distractor with no information. A further implication is that the distractor deserves partial credit. However, it immediately follows from the Rasch model that if a distractor deserves partial credit, then the response to that distractor and other distractors should not be pooled into a single response with a single probability of an incorrect response. Using the partial credit parameterization of the polytomous Rasch model, the paper shows how an hypothesis can be formed, and tested, regarding information in a distractor. The hypothesis is formed by studying the shape of the distractor response curves across the continuum, and the hypothesis is tested by scoring the correct response 2, the hypothesized distractor 1, and other distractors 0, and then applying the polytomous Rasch model. Multiple pieces of evidence, including fit of the responses at the two thresholds and the order of the two threshold estimates, are used in deciding if a distractor has information. An example illustrating the theory and its application is provided.     

Comparing concurrent versus fixed parameter equating with common items: using the dichotomous and partial credit models in a mixed-item format test

There has been some discussion among researchers as to the benefits of using one calibration process over the other during equating. Although literature is rife with the pros and cons of the different methods, hardly any research has been done on anchoring (i.e., fixing item parameters to their pre-determined values on an established scale) as a method that is commonly used by psychometricians in large-scale assessments. This simulation research compares the fixed form of calibration with the concurrent method (where calibration of the different forms on the same scale is accomplished by a single run of the calibration process, treating all non-included items on the forms as missing or not reached), using the dichotomous Rasch (Rasch, 1960) and the Rasch partial credit (Masters, 1982) models, and the WINSTEPS (Linacre, 2003) computer program. Contrary to the belief and some researchers' contention that the concurrent run with larger n-counts for the common items would provide greater accuracy in the estimation of item parameters, the results of this paper indicate that the greater accuracy of one method over the other is confounded by the sample-size, the number of common items, etc., and there is no real benefit in using one method over the other in the calibration and equating of parallel tests forms.     

The impact of receiving the same items on consecutive computer adaptive test administrations

This study addresses item exposure in a Computerized Adaptive Test (CAT) when the item selection algorithm is permitted to present examinees with questions that they have already been asked in a previous test administration. The results indicate that the combined use of an adaptive algorithm to select items and latent trait theory to estimate person ability provides substantial protection from score contamination. The implications for constraints that prohibit examinees from seeing an item twice are discussed.     

Diagnosing a common rater halo effect using the polytomous Rasch model

The 'halo effect' may be unique to different raters or common to all raters. When common to all raters, halo is not detectable through standard fit indices of the three-facet Rasch model used to account for differences in rater severities. Using a formulation of halo as a violation of local independence, a halo effect common to all raters is simulated and shown to be diagnosable through contrasts between two-facet stack and rack Rasch analyses. In the former, the thresholds are clustered and the distribution of persons is multimodal; in the latter, all thresholds are close together and the distribution of persons is unimodal. In the former, the scale is stretched, and the person separation inflated, relative to the latter.     

Calculation of the properties of digital mammograms using a computer simulation

A Monte Carlo computer model of mammography has been developed to study and optimise the performance of digital mammographic systems. The program uses high-resolution voxel phantoms to model the breast, which simulate the adipose and fibroglandular tissues, Cooper's ligaments, ducts and skin in three dimensions. The model calculates the dose to each tissue, and also the quantities such as energy imparted to image pixels, noise per image pixel and scatter-to-primary (S/P) ratios. It allows studies of the dependence of image properties on breast structure and on position within the image. The program has been calibrated by calculating and measuring the pixel values and noise for a digital mammographic system. The thicknesses of two components of this system were unknown, and were adjusted to obtain a good agreement between measurement and calculation. The utility of the program is demonstrated with the calculations of the variation of the S/P ratio with and without a grid, and of the image contrast across the image of a 50-mm-thick breast phantom.     

Comparing traditional and Rasch analyses of the Mississippi PTSD Scale: revealing limitations of reverse-scored items

This study examined whether Rasch analysis could provide more information than true score theory (TST) in determining the usefulness of reverse-scored items in the Mississippi Scale for Posttraumatic Stress Disorder (M-PTSD). Subjects were 803 individuals in inpatient PTSD units at 10 VA sites. TST indicated that the M-PTSD performed well and could be improved slightly by deleting one item. Factor analysis using raw scores indicated that the reverse-scored items formed the second factor and had poor relationships with normally scored items. However, since item-total correlations supported their usefulness, they were kept. The subsequent Rasch analysis indicated that five of the seven worst fitting items were reverse-scored items. We concluded that using reversed items with disturbed patients can cause confusion that reduces reliability. Deleting them improved validity without loss of reliability. The study supports the use of Rasch analysis over TST in health research since it indicated ways to reduce respondent burden while maintaining reliability and improving validity.     

An examination of exposure control and content balancing restrictions on item selection in CATs using the partial credit model

The purpose of the present investigation was to systematically examine the effectiveness of the Sympson-Hetter technique and rotated content balancing relative to no exposure control and no content rotation conditions in a computerized adaptive testing system (CAT) based on the partial credit model. A series of simulated fixed and variable length CATs were run using two data sets generated to multiple content areas for three sizes of item pools. The 2 (exposure control) X 2 (content rotation) X 2 (test length) X 3 (item pool size) X 2 (data sets) yielded a total of 48 conditions. Results show that while both procedures can be used with no deleterious effect on measurement precision, the gains in exposure control, pool utilization, and item overlap appear quite modest. Difficulties involved with setting the exposure control parameters in small item pools make questionable the utility of the Sympson-Hetter technique with similar item pools.     

Reliability and true-score measures of binary items as a function of their Rasch difficulty parameter

This article provides formulas for expected true-score measures and reliability of binary items as a function of their Rasch difficulty when the trait (ability) distribution is normal or logistic. The proposed formulas have theoretical value and can be useful in test development, score analysis, and simulation studies. Once the items are calibrated with the dichotomous Rasch model, one can estimate (without further data collection) the expected values for true-score measures (e.g., domain score, true score variance, and error variance for the number-right score) and reliability for both norm-referenced and criterion-referenced interpretations. Thus, given a bank of Rasch calibrated items, one can develop a test with desirable values of population true-score measures and reliability or compare such measures for subsets of items that are grouped by substantive characteristics (e.g., content areas or strands of learning outcomes). An illustrative example for using the proposed formulas is also provided.     

Measuring positive and negative affect in older adults over 56 days: comparing trait level scoring methods using the partial credit model

Positive (PA) and negative affect (NA) are important constructs in health and well-being research. Good longitudinal measurement is crucial to conducting meaningful research on relationships between affect, health, and well-being across the lifespan. One common affect measure, the PANAS, has been evaluated thoroughly with factor analysis, but not with Racsh-based latent trait models (RLTMs) such as the Partial Credit Model (PCM), and not longitudinally. Current longitudinal RLTMs can computationally handle few occasions of data. The present study compares four methods of anchoring PCMs across 56 occasions to longitudinally evaluate the psychometric properties of the PANAS plus additional items. Anchoring item parameters on mean parameter values across occasions produced more desirable results than using no anchor, using first occasion parameters as anchors, or allowing anchor values to vary across occasions. Results indicated problems with NA items, including poor category utilization, gaps in the item distribution, and a lack of easy-to-endorse items. PA items had much more desirable psychometric qualities.     

A comparison between robust z and 0.3-Logit Difference procedures in assessing stability of linking items for the Rasch model

There are at least two procedures to assess item difficulty stability in the Rasch model: robust z procedure and ".3 Logit Difference" procedure. The robust z procedure is a variation of the z statistic that reduces dependency on outliers. The ".3 Logit Difference" procedure is based on experiences in Rasch linking for tests developed by Harcourt. Both methods were applied to archival data from two large-scale South Carolina assessment programs: HSEE 1986/1987 and PACT 2004/2005.The results of the analysis showed the ".3 Logit Difference" procedure identifies slightly more stable items (2.6%) for all items under study. In addition, approximately 93% of all items under consideration were identically classified as stable or unstable for both procedures. This very high level of agreement between the two methods indicates that either procedure can be safely used to identify stable items for use in a common-item linking design. The advantage of the robust z procedure lies in its foundation of robust statistical inference. The procedure takes into account well-accepted models for identifying outliers and permits critical values set at a specified Type I error.     

Modelling the biomechanical properties of DNA using computer simulation

Duplex DNA must remain stable when not in use to protect the genetic material. However, the two strands must be separated whenever genes are copied or expressed to expose the coding strand for synthesis of complementary RNA or DNA bases. Therefore, the double stranded structure must be relatively easy to take apart when required. These conflicting biological requirements have important implications for the mechanical properties of duplex DNA. Considerable insight into the forces required to denature DNA has been provided by nanomanipulation experiments, which measure the mechanical properties of single molecules in the laboratory. This paper describes recent computer simulation methods that have been developed to mimic nanomanipulation experiments and which, quite literally, 'destruction test' duplex DNA in silico. The method is verified by comparison with single molecule stretching experiments that measure the force required to unbind the two DNA strands. The model is then extended to investigate the thermodynamics of DNA bending and twisting. This is of biological importance as the DNA must be very tightly packaged to fit within the nucleus, and is therefore usually found in a highly twisted or supercoiled state (in bacteria) or wrapped tightly around histone proteins into a densely compacted structure (in animals). In particular, these simulations highlight the importance of thermal fluctuations and entropy in determining the biomechanical properties of DNA. This has implications for the action of DNA processing molecular motors, and also for nanotechnology. Biological machines are able to manipulate single molecules reliably on an energy scale comparable to that of thermal noise. The hope is that understanding the statistical mechanisms that a cell uses to achieve this will be invaluable for the future design of 'nanoengines' engineered to perform new technological functions at the nanoscale.     

热处理数学模型与计算机模拟

热处理计算机模拟是热处理智能化的基础。采用三维非线性有限元模型和界面条件突变的处理方法实现了几种复杂形状零件在复杂的淬火操作过程中温度场、相变、应力和应变的模拟，模拟结果与实测结果基本相符。将气体渗碳的计算机模拟与计算机控制技术相结合，研制成功智能型密封多用炉自动生产线，已处理渗碳件3 000炉次以上，全部合格。计算机模拟技术的应用明显提高渗碳和渗氮零件的承载能力和可靠性，减少热处理变形，缩短渗碳时间。建议将机械产品的CAD、选材与热处理的CAE和产品可靠性的动力评估技术相结合，进行高可靠而又无冗余的产品设计。     

Exploring differential item functioning (DIF) with the Rasch model: a comparison of gender differences on eighth grade science items in the United States and Spain

Traditionally, women and minorities have not been fully represented in science and engineering. Numerous studies have attributed these differences to gaps in science achievement as measured by various standardized tests. Rather than describe mean group differences in science achievement across multiple cultures, this study focused on an in-depth item-level analysis across two countries: Spain and the United States. This study investigated eighth-grade gender differences on science items across the two countries. A secondary purpose of the study was to explore the nature of gender differences using the many-faceted Rasch Model as a way to estimate gender DIF. A secondary analysis of data from the Third International Mathematics and Science Study (TIMSS) was used to address three questions: 1) Does gender DIF in science achievement exist? 2) Is there a relationship between gender DIF and characteristics of the science items? 3) Do the relationships between item characteristics and gender DIF in science items replicate across countries. Participants included 7,087 eight grade students from the United States and 3,855 students from Spain who participated in TIMSS. The Facets program (Linacre and Wright, 1992) was used to estimate gender DIF. The results of the analysis indicate that the content of the item seemed to be related to gender DIF. The analysis also suggests that there is a relationship between gender DIF and item format. No pattern of gender DIF related to cognitive demand was found. The general pattern of gender DIF was similar across the two countries used in the analysis. The strength of item-level analysis as opposed to group mean difference analysis is that gender differences can be detected at the item level, even when no mean differences can be detected at the group level.     

Rehabilitation of the Parkinson's tremor by using robust adaptive sliding mode controller: a simulation study

One of the efficient methods in controlling the Parkinson''s tremor is Deep Brain Stimulation (DBS) therapy. The stimulation of Basal Ganglia (BG) by DBS brings no feedback though the existence of feedback reduces the additional stimulatory signal delivered to the brain. So this study offers a new adaptive architecture of a closed‐loop control system in which two areas of BG are stimulated simultaneously to decrease the following three indicators: hand tremor, the level of a delivered stimulation signal in the disease condition, and the level of a delivered stimulation signal in health condition to the disease condition. One area (STN: subthalamic nucleus) is stimulated with an adaptive sliding mode controller and the other area (GPi: Globus Pallidus internal) with partial state feedback controller. The simulation results of stimulating two areas of BG showed satisfactory performance.Inspec keywords: bioelectric phenomena, diseases, variable structure systems, brain models, biomedical electrodes, adaptive control, closed loop systems, state feedback, feedback, neurophysiology, brain, robust controlOther keywords: DBS, additional stimulatory signal, adaptive architecture, closed‐loop control system, hand tremor, delivered stimulation signal, disease condition, health condition, partial state feedback controller, Parkinson''s tremor, robust adaptive sliding mode controller, simulation study, efficient methods, Deep Brain Stimulation therapy, Basal Ganglia     

Analysis of adaptive shell structures using a refined laminated model

This work presents the development of a shell conical panel finite element model, which has the possibility of having embedded piezoelectric actuators and/or sensors patches. A mixed laminated theory is used, which combines an equivalent single layer higher order shear deformation approach for the mechanical behavior with a layerwise representation in the thickness direction to describe the distribution of the electric potential in each of the piezoelectric layers of the finite element. The electrical potential function is represented through a linear variation across the thickness with two electric potential nodes for each piezoelectric layer. Based in this model an active damping scheme applied to laminated shell structures is presented and discussed.     

Fabrication,process simulation and testing of a thick CFRP component using the RTM process

The present article introduces the case of a CFRP con-rod beam, and describes many aspects regarding its production with the Resin Transfer Moulding (RTM) process.The objective was to find the best process parameters of the injection and curing stages in order to manufacture the 20 mm thick CFRP part. The results are analysed in terms of the aesthetic aspect, the porosity and the mechanical properties of the final component.For the resin injection stage, results obtained from production experiences are presented, which have been performed with different set-ups, and simulations of the resin flow are used to analyse them. The results show that the resin flow during injection could be rather unpredictable, probably because of the fibre rearrangement and race tracking effects. Improvements in terms of aesthetic aspect and porosity of the part could be achieved by a process which included final compaction of the cavity by means of compressed air.Regarding the curing stage, the article presents the simulation results of a curing cycle, and it’s validation through DSC analysis of specimens obtained from the finished component.Finally, results of tensile mechanical tests are provided, performed on finished components produced by RTM and compared to others produced with the method of hand lay-up of pre-impregnated plies and curing in autoclave (Prepreg + Autoclave). The results confirm that it is possible to achieve components through RTM with comparable mechanical performance to those produced with the Prepreg + Autoclave process.