A Monte Carlo evaluation of three formula estimates of cross-validated multiple correlation.

To determine the stability of regression equations, researchers have typically employed a cross-validation design in which weights are developed on an estimation subset of the sample and then applied to the members of a holdout sample. The present study used a Monte Carlo simulation to ascertain the accuracy with which the shrinkage in R–2 could be estimated by 3 formulas developed for this purpose. Results indicate that R. B. Darlington's (see record 1968-08053-001) and F. M. Lord (1950) and G. E. Nicholson's (1960) formulas yielded mean estimates approximately equal to actual cross-validation values, but with smaller standard errors. Although the Wherry estimate is a good estimate of population multiple correlation, it is an overestimate on population cross-validity. It is advised that the researcher estimate weights on the total sample to maximize the stability of the regression equation and then estimate the shrinkage in R–2 that he/she can expect when going to a new sample with either the Lord-Nicholson or Darlington estimation formulas. (17 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Generalizability of a checklist for assessment of psychopathy.

Suggests that there are problems associated with assessments of psychopathy in prison populations that use self-report inventories and global diagnostic procedures. In response to these problems, the authors developed a behavioral checklist for psychopathy. The psychometric qualities of the checklist were evaluated using generalizability theory and classical test score indices of reliability. In each of 5 yrs, 2 raters (usually different each year) rated prison inmates (N?=?301; mean age 26.9 yrs) on 22 items. The generalizability coefficients were .85, .86, and .89 for the years 1977–1981, respectively. The generalizability coefficient for a test–retest study was .89. Classical indices of reliability (alpha coefficients and inter- and intrarater reliability) ranged from .82 to .93. Results indicate that the checklist is a highly reliable and generalizable instrument when used with prison populations. It is highly correlated with global ratings of psychopathy and criteria from the DSM-III for antisocial personality disorder. (14 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Evaluation of Wechsler Adult Intelligence Scale—Revised premorbid IQ formulas in clinical populations.

The use of demographic variables in the prediction of premorbid IQ has been noted to show some promise (R. S. Wilson et al; see record 1979-26429-001). With the revision of the Wechsler Adult Intelligence Scale (WAIS—R) and subsequent development of a regression formula for the WAIS—R (A. Barona et al; see record 1985-04035-001), some authors have suggested that continued use of the Wilson formulas is inappropriate. The present study examined the efficacy of both Wilson and Barona formulas in the prediction of concurrently obtained IQ levels in neurologically normal psychiatric (n?=?77) and brain-damaged (n?=?64) patients. The results demonstrated that neither formula differed significantly in terms of classifications of IQ range, with both formulas performing essentially at chance levels. Qualitative and quantitative differences between the formulas are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Marital interactions of drug-abusing patients and their partners: Comparisons with distressed couples and relationship to drug-using behavior.

Couples with a drug-abusing husband (n?=?17) and non-substance-abusing, distressed couples (n?=?17) completed several self-report measures of relationship adjustment and participated in videotaped marital conflict-resolution discussions that were coded with the Clinician Rating of Adult Communication (CRAC). Although no differences were found between the couple types on the self-report inventories, drug-abusing couples had higher scores (indicating more frequent use of dysfunctional communication behaviors) on 3 of the 5 CRAC subscales (i.e., Abusiveness, Problem-Solving Skills, and Attribution of Blame) and a higher CRAC total score (a global measure of communication skill) than distressed couples. In addition, the CRAC total score for drug-abusing couples was negatively related to husbands' percentage of days abstinent during the year before entering substance abuse treatment. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Two forms of minority-group test bias as psychometric artifacts with an animal model (Rattus norvegicus).

Controversy abounds over attributing group differences on tests to nature, nurture, or test bias. Limitations of correlational sampling from natural populations necessitate experimental methods to resolve underlying issues. In classicial psychometrics test items are selected from a larger item pool through analysis of item responses in a sample of subjects. Rats of six inbred strains (n?=?366) were tested in multiple mazes to provide a large item pool. Six populations were created, each with differing proportions of each strain. Items selected through independent item analyses within each population yielded six tests. An independent cross-validation sample (n?=?146) provided scores on all six items. This sample was also tested in another set of maze problems defined as the criterion to be predicted. Strain means and intrastrain predictive validities for the six tests varied with strain representation in the population used for item selection (p?     

Factor structure of mental health measures for older adults.

This article examines the factor structure of mental health self-reports among 246 older adults, ages 60 to 80 years, who were either recently physically disabled (n?=?62), recently bereaved (n?=?61), or matched control subjects (n?=?123). Confirmatory factor analyses were carried out on the Mental Health Inventory and the Psychiatric Epidemiology Research Interview (PERI) Demoralization Composite to test whether factor structures obtained in previous studies would fit the data for this older adult sample and whether those structures would be equivalent among groups that differ in degree of life stress. The structure of these two inventories was reorganized as a result of these analyses, resulting in 9 subscales that varied somewhat from the original subscale structure. The Bradburn Positive Affect Scale was added, and a second order confirmatory factor analysis was performed on these 10 scales. Two highly correlated superordinate factors emerged: Psychological Distress and Psychological Well-Being. Although the factor structure was generally similar across groups, there were some notable exceptions that could be attributed to between-groups differences in exposure to life stress. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

An examination of nonresponse in a work environment questionnaire mailed to psychiatric health care personnel.

Nonresponse to surveys, which seems to plague work environment studies and studies of health care personnel in particular, may pose problems of generalizability and validity. The aim of this study was to provide an estimate of nonresponse error in a self-administered survey concerning the work environment of psychiatric health care personnel. A random sample of 10% of the original survey population (N?=?693) was selected to participate in a telephone follow-up of a postal survey that had a response rate of 51%. There were no differences between the responders and nonresponders to the postal survey on the exposure or outcome variables. There was no evidence of nonresponse bias in this study, although recall bias may have been a problem. In those cases in which generalizability is deemed important, it is recommended that nonresponse studies be regularly carried out when response rates are less than 100%. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Personality measurement, faking, and detection: An assessment method for use in personnel selection.

Usual methods for personality assessment have been found unsuitable for use in personnel selection contexts. An alternative method of item construction and of scoring key and detection scale development for personality inventories is proposed. Results of a double cross-validation study based on 456 male Ss using 3 newly developed forced-choice inventories indicate that (a) over 90% of the test performance can be correctly identified as self-report or faked, (b) mean score profiles under the 2 conditions for the 5 personality variables under study are virtually congruent and variances under the faking condition are uniformly smaller, (c) validities against peer-nomination criteria are in the moderate range for all 5 variables and (d) Kuder-Richardson Formula 20 reliabilities between .83 and .92 were obtained. (27 ref.) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Cultural fairness in aptitude testing revisited: A cross-cultural parallel.

Explored some interesting parallels between the cultural fairness controversy and related research in the US, which peaked in the late 1970s, and the ongoing debate and empirical research on test fairness in a cultural site different from the US—namely, Israel. Specifically, I tested for cultural fairness and equity of various scholastic aptitude exams routinely used in Israel for prediction and selection purposes among school-age pupils and college applicants alike. In the 1st study I tested for differential predictive validity in verbal and nonverbal aptitude test scores, for a representative sample of middle-class majority (n?=?144) and lower-class minority (n?=?144) Israeli schoolchildren, with grade point average (GPA) as criterion. In the 2nd study I tested the degree of differential validity in college admissions aptitude test scores, as predictors, for 824 Jewish majority and 364 Arab minority students enrolled at a major Israeli campus; 1st-year GPA served as the criterion measure. Overall, the results of both studies lend cross-cultural generalizability to much previous research conducted in the US, providing some evidence of intercept bias; test scores were overpredictive of the scores of minority group students. However, there was little evidence of predictive slope bias by sociocultural group. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Structure of the Weinberger Adjustment Inventory Self-Restraint Scale and its relation to problem behaviors in adolescence.

The authors examined the structure of the Weinberger Adjustment Inventory (WAI) Self-Restraint scale in derivation (n?=?1,286) and cross-validation (n?=?1,154) samples of mostly African American 6th-graders in 3 urban schools. Four models were compared: (1) a 1-factor model; (2) a hierarchical model in which factors representing Impulse Control, Suppression of Aggression, Responsibility, and Consideration of Others were subsumed by a higher order factor; (3) a model that represented these 4 factors as correlated but distinct constructs; and (4) a model that excluded Consideration of Others from the higher order factor. Consistent support was found for the last model based on confirmatory factor analyses and latent-variable analyses examining the relations among self-restraint scales, drug use, delinquency, and aggression. These findings have implications for using the WAI, particularly in studies of adolescent problem behaviors. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Are indigenous personality dimensions culture specific? Philippine inventories and the five-factor model.

The authors addressed the culture specificity of indigenous personality constructs, the generalizability of the 5-factor model (FFM), and the incremental validity of indigenous measures in a collectivistic culture. Filipino college students (N=508) completed 3 indigenous inventories and the Filipino version of the Revised NEO Personality Inventory (NEO-PI-R). On the basis of the factor and regression analyses, they concluded that (a) most Philippine dimensions are well encompassed by the FFM and thus may not be very culture specific; (b) a few indigenous constructs are less well accounted for by the FFM; these constructs are not unknown in Western cultures, but they may be particularly salient or composed somewhat differently in the Philippines; (c) the structure of the NEO-PI-R FFM replicates well in the Philippines; and (d) Philippine inventories add modest incremental validity beyond the FFM in predicting selected culture-relevant criteria. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Integrating laboratory and field study for improving selection: Development of a battery for predicting air traffic controller success.

Describes an example of combining laboratory- and field-based study to develop a selection battery for field implementation. The procedure provides advantages in comparison with sole use of field-based experimentation, namely, greater control over examinee samples, collection of construct validity data, and fewer field demands for cross-validation. Two experiments were conducted that converge on development of a test battery for selection of air traffic controllers (ATCs). The lab study (N?=?112) used an ATC simulator (terminal radar approach control, or TRACON) for initial development and evaluation of the selection battery. The field study of 206 Federal Aviation Administration ATC trainees provided cross-validation data as a precursor to implementation of the battery. Implications for developing ability-based and self-efficacy-based selection measures for complex job performance are discussed, as are general issues for new selection research and application. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Validity and generalizability of student ratings from between-classes and within-class data.

Examined the generalizability and validity of student (N?=?485) ratings by studying within-class and between-classes correlations of ratings with other variables for regular faculty teaching lecture courses as well as for graduate assistants teaching recitation sections. Results indicate that most ratings were highly generalizable but only some were related to learning and that certain aspects of both generalizability and validity varied with instructor's role and with level of data. The implications of these findings for the evaluation of teaching are discussed with reference to 2 alternative paradigms: construct validity and criterion development. (22 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

White racial identity attitude development: A psychometric examination of two instruments.

The purpose of this investigation was to provide an independent examination of the psychometric properties of the White Racial Identity Attitude Scale (WRIAS; J. E. Helms & R. T. Carter, 1990) and the Oklahoma Racial Attitude Scale—Preliminary Form (ORAS–P; S. Choney & J. Behrens, 1996). After completing both inventories, 387 students in introductory education and psychology classes from a Midwest university were split randomly into 2 groups: Sample 1 (n?=?199) was used for exploratory factor analysis, and Sample 2 (n?=?199) for confirmatory factor analysis. Four factors reflecting attitudes were identified: Degree of Racial Comfort (Factor 1), Attitudes Toward Racial Equality (Factor 2), Attitudes of Racial Curiosity (Factor 3), and Unachieved Racial Attitudes (Factor 4). Each instrument was represented by 3 of the 4 factors. The WRIAS and ORAS–P overlapped on Factors 1 and 2, the WRIAS loaded only on Factor 3, and the ORAS–P loaded only on Factor 4. Recommendations are made for future investigations. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The Variable Responding Scale for detection of random responding on the Multidimensional Pain Inventory.

This study developed a scale for detecting random responding on the Multidimensional Pain Inventory (MPI). Ninety-five undergraduates (derivation sample) completed the MPI randomly, as did 2 cross-validation samples, 34 chronic pain patients (pain) and 115 health care professionals (health care). Up to 71% of random profiles appeared valid. For comparison in validity scale development, a clinical MPI sample (N?=?507) was split into derivation and cross-validation samples. Given that responses to similar items should be consistent in nonrandom protocols, 8 pairs of highly intercorrelated items were selected. Absolute differences between pairs were summed into a variable responding (VR) scale; scores were contrasted across clinical and random groups. On the basis of derivation sample results, VR scale cut scores (from 12 to 17) were tested and found to discriminate accurately (p?     

The WAIS—R index for estimating premorbid intelligence: Cross-validation and clinical utility.

Barona, Reynolds, and Chastain (1984) proposed new demographic information formulas for estimating premorbid Wechsler Adult Intelligence Scales-Revised IQ-scores. Our study was directed toward the following issues: (a) cross-validation of this index on a neurologically normal but clinically relevant criterion group, and (b) determining the neuropsychological utility of the formulas to discriminate between neurologically normal and brain-impaired populations. Our subjects included 80 neurologically normal patients and 83 brain-impaired patients. Cross-validation results generally support the estimation accuracy of the formulas within a neurologically normal clinical sample, although by all three formulas, scores are significantly overestimated. To determine the neuropsychological usefulness of the formulas, we derived a difference score (D-score) cut-point method. D-score diagnostic discrimination of patients was slightly greater than obtained IQs, but not at a statistically significant level. Suggestions for clinical use and further research with the Barona Index are provided. (10 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Estimates of cross-validity for stepwise regression and with predictor selection.

The effects of preselection of predictors (e.g., stepwise regression) on formula estimates of cross-validity were examined. Three actual data sets were used to generate populations of varying sample size, population validity, and number of predictors. No formula estimate provided an unbiased estimate of the population cross-validity, although some formula estimates were less biased than others. More important, having an adequate sample size (relative to number of predictors) was the issue most affecting the utility of the formula estimates. Another conclusion was that adjusted R2 provided by at least some popular software programs can provide gross overestimates of cross-validity and should not be used as such. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Delayed-nonmatching-to-sample performance is impaired by extensive, but not by limited, lesions of the thalamus in the rat.

Two experiments were conducted to determine whether lesions affecting limited areas of the thalamus can impair the performance of rats on a spatial delayed-nonmatching-to-sample (DNMTS) task trained before surgery. In Exp 1, DNMTS was not affected by lesions produced by injecting 5 μl of 1 mM N-methyl-D-aspartate (NMDA) into either the midline thalamus (n?=?16) or bilaterally 1.0 mm from the midline (n?=?16). In Exp 2, radio-frequency lesions were made 1.0 mm lateral from the midline at 3 anterior–posterior locations that destroyed the full rostral–caudal extent of the lateral internal medullary lamina (L-IML; n?=?8), or at single anterior–posterior locations that destroyed either the anterior (n?=?8) or posterior (n?=?8) portions of the L-IML site. Although complete L-IML lesions disrupted DNMTS performance to an extent comparable to that of another study (R. G. Mair and D. M. Lacourse; see record 1992-42024-001), lesions that were restricted to either the anterior or posterior portion of the L-IML site had no significant effect on this task. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The shortened workweek: A field experiment.

Examined the effects of the 4-day, 40-hr workweek by comparing 2 experimental groups (n?=?97, n?=?111) and a comparison group (n?=?94) of operating employees in a medium-sized manufacturing company. Comparisons were made on dimensions of self-actualization, autonomy, personal worth, social affiliation, job security, pay and overall job satisfaction, anxiety-stress, absenteeism, and performance over a 13- and a 25-mo period. Analysis of 13-mo data indicated that Ss in the 4-day, 40-hr groups were (a) more satisfied with autonomy, personal worth, job security, and pay; (b) experienced less anxiety-stress; and (c) performed better with regard to productivity than did the comparison group. However, these improvements were not found with the 25-mo data. It is suggested that uncontrolled variables may account for the differences in the short- and long-term effects of the shortened workweek. (PsycINFO Database Record (c) 2010 APA, all rights reserved)