Self-adapted testing: A performance-improving variant of computerized adaptive testing.

We conducted an experiment that contrasted a variant of computerized adaptive testing, self-adapted testing, with two traditional tests, a relatively difficult one and a relatively easy one, that were constructed from the same bank of verbal ability items. In a self-adapted test, the examinee, rather than a computerized algorithm, chooses the difficulty of the next item to be presented. Participants completed a self-report of text anxiety and were randomly assigned to take one of the three tests of verbal ability. Analyses of variance using Rasch estimates of ability and the standard error of those estimates as dependent measures demonstrated that the self-adapted test led to higher ability estimates and minimized the effect of test anxiety without any overall loss of measurement precision. Analysis of the item choices in the self-adapted test suggested that, in general, participants chose more difficult items as the test progressed. Anxiety was negatively associated with the difficulty of the initial choice but not associated with the rate of progress to higher difficulty items. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Computerized adaptive personality testing: A review and illustration with the MMPI-2 computerized adaptive version.

Computerized adaptive testing in personality assessment can improve efficiency by significantly reducing the number of items administered to answer an assessment question. Two approaches have been explored for adaptive testing in computerized personality assessment: item response theory and the countdown method. In this article, the authors review the literature on each and report the results of an investigation designed to explore the utility, in terms of item and time savings, and validity, in terms of correlations with external criterion measures, of an expanded countdown method-based research version of the Minnesota Multiphasic Personality Inventory-2 (MMPI-2), the MMPI-2 Computerized Adaptive Version (MMPI-2-CA). Participants were 433 undergraduate college students (170 men and 263 women). Results indicated considerable item savings and corresponding time savings for the adaptive testing modalities compared with a conventional computerized MMPI-2 administration. Furthermore, computerized adaptive administration yielded comparable results to computerized conventional administration of the MMPI-2 in terms of both test scores and their validity. Future directions for computerized adaptive personality testing are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Factor structure of the PHQ-9 screen for depression across time since injury among persons with spinal cord injury.

Objective: The overlap of somatic-depressive symptoms and physical sequelae of spinal cord injury (SCI) has raised concerns regarding the validity of depression screens used within the SCI population. The Patient Health Questionnaire-9 (PHQ-9) measure parallels Diagnostic and Statistical Manual of Mental Disorders (4th ed., text rev.; DSM-IV-TR; American Psychiatric Association, 2000) criteria of major depression. The authors investigated PHQ-9 factor structure among persons with SCI at various times postinjury. Design: Data from 2,570 participants at 1 year (N = 682), 5 years (N = 517), 15 years (N = 653), and 25 years (N = 718) postinjury were used. Emergent factors were correlated with satisfaction with life. Results: A 2-factor solution emerged for all groups, with 3 affective referenced items (feeling depressed/hopeless, feeling bad about self/failure, and suicidal ideation) and 3 somatic referenced items (sleep disturbance, low energy/fatigue, and appetite disturbance) loading consistently on Affective and Somatic factors, respectively, at all time points. Factor scores negatively correlated with satisfaction with life. Conclusions: Dual factor structure of the PHQ-9 is present at various times postinjury in the SCI population. It remains unclear whether somatic item endorsement reflects depressive symptomatology per se; however, endorsement is still associated with satisfaction with life. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Computerized adaptive personality assessment: An illustration with the Absorption scale.

This article introduces the theory behind and applications of adaptive personality assessment based on the item response theory. Two adaptive testing strategies were compared: (a) fixed test length and (b) clinical decision. Real-data simulations, based on the item responses from 1,000 subjects who had previously taken the 34-item Absorption scale (A. Tellegen, 1982) by means of paper-and-pencil format, were used to illustrate these strategies. Results suggest that computerized adaptive personality assessment works impressively well. With the fixed-test-length strategy, a 50% savings in administered items was achieved with little loss of measurement precision. In the clinical-decision testing strategy, individuals who were extreme on the Absorption trait were identified with perfect accuracy using, on average, 25% of the available items. The implications of these results for personality research and assessment are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The effect of item pool restriction on the precision of ability measurement for a Rasch-based CAT: comparisons to traditional fixed length examinations

This paper describes a method for examining the precision of a computerized adaptive test with a limited item pool. Standard errors of measurement ascertained in the testing of simulees with a CAT using a restricted pool were compared to the results obtained in a live paper-and-pencil achievement testing of 4494 nursing students on four versions of an examination of calculations of drug administration. CAT measures of precision were considered when the simulated examine pools were uniform and normal. Precision indices were also considered in terms of the number of CAT items required to reach the precision of the traditional tests. Results suggest that regardless of the size of the item pool, CAT provides greater precision in measurement with a smaller number of items administered even when the choice of items is limited but fails to achieve equiprecision along the entire ability continuum.     

A real-data simulation of computerized adaptive administration of the MMPI.

A real-data simulation of computerized adaptive administration of the Minnesota Multiphasic Personality Inventory (MMPI) was conducted with data obtained from two personnel-selection samples and two clinical samples. A modification of the countdown method was tested to determine the usefulness, in terms of item administration savings, of several different test administration procedures. Substantial item administration savings were achieved for all four samples, though the clinical samples required administration of more items to achieve accurate classification and/or full-scale scores than did the personnel-selection samples. The use of normative item endorsement frequencies was found to be as effective as sample-specific frequencies for the determination of item administration order. The role of computerized adaptive testing in the future of personality assessment is discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Is depression taxonic, dimensional, or both?

The current investigation examines the potential importance of item selection in the ongoing discussion regarding the taxonicity of depression. Following J. C. Coyne (1994), we contrast the taxonicity of "distress" with the taxonicity of a syndrome more focused on somatic symptoms (i.e., the involuntary defeat syndrome). Using 4 samples of 984 undergraduates, we fast replicate the J. Ruscio and A. M. Ruscio (2000) results, by showing that distress item indicators are dimensional. We then demonstrate taxonicity using items focused on somatic symptoms and reflective of disruption of multiple homeostatic mechanisms. Results suggest that item selection is central to the determination of taxonicity in depression. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Validation of the MMPI-2 Computerized Adaptive version (MMPI-2-CA) in a correctional intake facility.

Computerized adaptive testing in personality assessment can improve efficiency by significantly reducing the number of items administered to answer an assessment question. The time savings afforded by this technique could be of particular benefit in settings where large numbers of psychological screenings are conducted, such as correctional facilities. In the current study, item and time savings, as well as the test–retest and extratest correlations associated with an audio augmented administration of all the scales of the Minnesota Multiphasic Personality Inventory (MMPI)-2 Computerized Adaptive (MMPI-2-CA) are reported. Participants include 366 men, ages 18 to 62 years (M = 33.04, SD = 10.40), undergoing intake into a large Midwestern state correctional facility. Results of the current study indicate considerable item and corresponding time savings for the MMPI-2-CA compared to conventional administration of the test, as well as comparability in terms of test–retest and correlations with external measures. Future directions of adaptive personality testing are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Computerized adaptive assessment with the MMPI-2 in a clinical setting.

Comparability, validity, and impact of loss of information of a computerized adaptive administration of the Minnesota Multiphasic Personality Inventory-2 (MMPI-2) were assessed in a sample of 140 Veterans Affairs hospital patients. The countdown method (J. N. Butcher et al, see record 1986-10763-001) was used to adaptively administer Scales L (Lie) and F (Frequency), the 10 clinical scales, and the 15 content scales. Participants completed the MMPI-2 twice, in 1 of 2 conditions: computerized conventional test–retest, or computerized conventional–computerized adaptive. Mean profiles and test–retest correlations across modalities were comparable. Correlations between MMPI-2 scales and criterion measures supported the validity of the countdown method, although some attenuation of validity was suggested for certain health-related items. Loss of information incurred with this mode of adaptive testing has minimal impact on test validity. Item and time savings were substantial. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Effects and underlying mechanisms of self-adapted testing.

Undergraduates participated in 3 experiments related to self-adapted testing. Experiment 1 demonstrated that, in comparison with computerized adaptive testing, self-adapted testing reduced the influence of anxiety on performance but took longer and was less efficient. Experiment 2 indicated that benefits of self-adapted testing cannot be attributed solely to item ordering. Instead, active choice of item difficulty seems to be necessary. Experiment 3 demonstrated that the provision of feedback increased the efficiency of the test but had no effect on estimates of ability derived. The potential of self-adapted testing to reduce the influence of extraneous sources of variation in test performance is discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Posttraumatic stress disorder and major depression in veterans with spinal cord injury.

Objective: To explore the relationship between posttraumatic stress disorder (PTSD) and major depressive disorder (MDD) in veterans with spinal cord injury and to compare those results with results found in veterans who had sustained other traumatic injuries. Method: To investigate the relationship between PTSD and MDD in persons with spinal cord injury, the authors examined whether individuals endorsed overlapping items on measures of both disorders, evaluated the contribution of overlapping items to comorbid diagnosis, and conducted an exploratory factor analysis. Results: The overlapping symptoms between the 2 disorders did not fully explain the high rate of comorbidity, although participants who endorsed a symptom common to MDD and PTSD on 1 measure were likely to endorse the corresponding item on another measure. In both samples, items loaded on separate PTSD and MDD factors. Conclusion: MDD and PTSD appear to represent independent reactions to trauma in those individuals who had experienced either a nonspinal cord injury or a spinal cord injury. This research also provides an initial investigation of some of the possible ways that MDD and PTSD are related by addressing psychometric issues inherent in their measurement. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Mini-Mental State Examination item scores as predictors of Alzheimer's disease: incidence data from the Kungsholmen Project, Stockholm

BACKGROUND: The present study examined the power of individual Mini-Mental State Examination (MMSE) items in predicting incidence of Alzheimer's disease (AD). In addition, 3-year longitudinal changes in MMSE items were contrasted between incident AD and nondemented persons. METHODS: A population-based group of very old adults, 75-95 years of age, were followed longitudinally. Of the original 327 participants, 32 were diagnosed with probable or possible AD after a 3-year follow-up interval and 189 remained nondemented. Cognitive performance was indexed by the individual item scores from the MMSE. These sample from multiple domains of cognitive functioning, including visuospatial skill, recent memory, orientation to time and place, language, and the ability to sustain attention. RESULTS: Items dealing with delayed episodic memory and orientation to time were significant predictors of AD incidence, independent of age, gender, and years of education, as determined by logistic regression analyses. Longitudinally, changes in performance were largest among individuals diagnosed as incident AD, although the magnitude of change across items was highly variable. In particular, decline was relatively small for the delayed memory item, whereas most other measures showed dramatic decline in performance among individuals with incident AD. CONCLUSIONS: Individual MMSE items, especially those with some type of episodic memory referent, were the best predictors of incident cases of AD. Moreover, MMSE items displayed differential rates of changes, particularly for the incident AD participants.     

Adaptive testing by computer.

Describes problems with conventional tests, in which all examinees take the same items; the advantages of adaptive tests are also described. Computer simulation results and results of other live-testing studies by J. G. Thompson and the present author (1980) are reviewed. Adaptive testing based on item response theory is also discussed. Results support theoretical predictions that adaptive tests can decrease testing time by about 50% while resulting in more precise measurements in comparison with conventional tests. Item pool and computer hardware and software requirements for adaptive testing are specified, and a microcomputer-based system for implementing adaptive testing in clinical environments is briefly described. (26 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Screening for depression in primary care patients: Dimensionality of the Short Form of the Beck Depression Inventory.

In an attempt to discover factors that might explain the high misclassification rate of the short form of the Beck Depression Inventory (BDI) as a screening tool for depression in primary health care patients (primarily due to false positives), confirmatory factor analysis was used to test the dimensionality of the instrument with data from a sample of 598 family practice outpatients. A dual-factor specification provided a superior fit, with 2 moderately correlated factors (r?=?.722), identified and labeled Nonsomatic Symptoms and Somatic Symptoms. The items work inhibition and fatigue loaded together on the Somatic Symptoms factor, whereas the anorexia item loaded on both factors, and it is hypothesized that these symptoms are elevated in otherwise nondepressed, medically ill patients. Caution is urged in interpreting positive screens of the shorter BDI where somatic symptoms are elevated in the absence of nonsomatic symptoms. Alternative screening and diagnostic testing algorithms are suggested. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Depression without sadness: functional outcomes of nondysphoric depression in later life

OBJECTIVES: We hypothesized that depressive symptoms not meeting full standard criteria for Major Depression would be associated with significant functional impairment among older adults over the course of a 13-year follow-up interval. Specifically, we developed criteria for a form of depression whose core symptoms did not include sadness or dysphoria. DESIGN: Population-based 13-year follow-up survey. SETTING: Community-dwelling adults living in East Baltimore in 1981. PARTICIPANTS: Subjects were the 1612 participants of the Baltimore sample of the Epidemiologic Catchment Area Program aged 50 years and older at the initial interview in 1981. MEASUREMENTS: The subjects were sorted into four categories based on their responses at baseline: (1) persons meeting standard criteria for Major Depression; (2) persons meeting alternative criteria for depression with dysphoria or (3) without dysphoria; and (4) a comparison category of persons not meeting any criteria for depression ("noncases"). The mortality and functional status of each group were compared after a 13-year follow-up interval. RESULTS: Compared with non-cases, participants aged 50 years and older who reported depressive symptoms but who denied sadness or dysphoria (nondysphoric depression) were at increased risk for death (relative risk (RR) = 1.70; 95% confidence interval (CI) (1.09, 2.67)), impairment in activities of daily living (RR = 3.76; 95% CI (1.73, 8.14)), impairment in instrumental activities of daily living (RR = 5.07; 95% CI (2.24, 11.44)), psychologic distress (RR = 3.68; 95% CI (1.47, 9.21)), and cognitive impairment (RR = 3.00; 95% CI (1.31, 6.89)) after a 13-year follow-up interval. The findings were not wholly explained by potentially influential baseline characteristics such as age, education, selected comorbid medical conditions, and functional status. CONCLUSION: Among adults aged 50 years and older, nondysphoric depression may be as important as Major Depression in relation to the development of functional disability and other long-term outcomes.     

Cluster-branching methodology for adaptive testing and the development of the Adaptive Category Test.

Adaptive testing involves the adjustment of a set of test items, in accordance with an individual's characteristics, to minimize items that do not yield useful information. The best known methodology used to develop adaptive tests, item response theory (IRT), cannot be used with most psychological instruments. The authors propose using cluster analysis to develop a branching logic that would allow the adaptive administration of such instruments. The proposed methodology is described in detail and is used to develop an adaptive version of the Halstead Category Test (W. Halstead & P. Settlage, 1943) from archival data. Real-data simulations show the Adaptive Category Test to yield scores that are not significantly different from the scores actually obtained on the original version of the test. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Family caregivers of persons with spinal cord injury: Predicting caregivers at risk for probable depression.

Objective: To determine the percentage of family caregivers of persons with spinal cord injury (SCI) with probable depression and to test the hypothesis that dysfunctional problem-solving abilities would be significantly predictive of risk status after taking into account important demographic characteristics and caregiver health. Design: Correlational and logistic regression analyses of data collected in a cross-sectional design. Participants: Eighteen men and 103 women caregivers (mean age of caregivers = 45.66 years, SD = 12.88) of individuals with SCI. Main Outcome Measure: The Inventory to Diagnose Depression. Results: Nineteen caregivers (15.7%) met criteria on the Inventory to Diagnose Depression for a major depressive disorder. A dysfunctional problem-solving style was significantly predictive of caregiver depression, regardless of the severity of physical impairment of the care recipient or the physical health of the caregiver and caregiver demographic variables. Conclusions: The percentage of caregivers with probable depressive disorder may parallel that observed among persons with SCI, using a more conservative self-report measure designed to assess symptoms associated with a depressive syndrome. Family caregivers with a dysfunctional problem-solving style and assisting individuals with more severe injuries may have probable depression. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Bifactor and item response theory analyses of interviewer report scales of cognitive impairment in schizophrenia.

A psychometric analysis of 2 interview-based measures of cognitive deficits was conducted: the 21-item Clinical Global Impression of Cognition in Schizophrenia (CGI-CogS; Ventura et al., 2008), and the 20-item Schizophrenia Cognition Rating Scale (SCoRS; Keefe et al., 2006), which were administered on 2 occasions to a sample of people with schizophrenia. Traditional psychometrics, bifactor analysis, and item response theory methods were used to explore item functioning and dimensionality and to compare instruments. Despite containing similar item content, responses to the CGI-CogS demonstrated superior psychometric properties (e.g., higher item intercorrelations, better spread of ratings across response categories) relative to the SCoRS. The authors argue that these differences arise mainly from the differential use of prompts and how the items are phrased and scored. Bifactor analysis demonstrated that although both measures capture a broad range of cognitive functioning (e.g., working memory, social cognition), the common variance on each is overwhelmingly explained by a single general factor. Item response theory analyses of the combined pool of 41 items showed that measurement precision is peaked in the mild to moderate range of cognitive impairment. Finally, simulated adaptive testing revealed that only about 10 to 12 items are necessary to achieve latent trait level estimates with reasonably small standard errors for most individuals. This suggests that these interview-based measures of cognitive deficits could be shortened without loss of measurement precision. (PsycINFO Database Record (c) 2011 APA, all rights reserved)     

Age-related associative deficits are absent with nonwords.

Words and nonwords were used as stimuli to assess item and associative recognition memory performance in young and older adults. Participants were presented with pairs of items and then tested on both item memory (old/new items) and associative memory (intact/recombined pairs). For words, older participants performed worse than young participants on item and associative tests but to a greater extent on the latter. In contrast, for nonwords, older participants performed equally worse than young participants on item and associative tests. This is the first study to demonstrate that a manipulation of stimulus novelty can alter age-related associative deficits. (PsycINFO Database Record (c) 2011 APA, all rights reserved)