Use of the Rorschach test as a diagnostic criterion.

In his American Psychologist article, Joseph Lerner (see record 1964-01189-001) kindly ascribed to me words which properly belong to Samuel J. Beck. Beck does refer to my Perceptanalysis (Piotrowski & Lewis, 1957), but not on the same page. His words express my past belief. At present my attitude is more complex. It changed after I checked some "blind" Rorschach diagnoses and clinical psychiatric diagnoses on the same patients (Piotrowski, 1950, p. 363), and read published reviews of the reliability and validity of clinical psychiatric diagnoses. These revealed that a considerable percentage of first admission patients, discharged as psychoneurotics, are rediagnosed as schizophrenics after a re-examination several years later. In fact, some schizophrenic conditions escape detection through clinical observations for as long as 10 years, despite intermittent clinical examinations. The Rorschach test definitely is highly sensitive to schizophrenia even though at times some remitted or much improved schizophrenics produce test records failing to give any indication of the psychosis, let alone of the past acute psychotic episodes Lerner stated that "the Rorschach alone is of little assistance unless it is an integral part of the total evaluation." Well, if the Rorschach is never used as an independent diagnostic criterion, we shall never know how good or bad a diagnostic criterion it is. Using it as a part source of information, is to contaminate it (that is why "blind" diagnoses are important). The second point is: It seems advisable to follow the rule that if clinical observations or the Rorschach test--or both--suggest schizophrenia, this diagnosis is likely to be valid. This rule is compatible with Lerner's conclusion that an evaluation based on all available sources of information is better than one which utilizes only one diagnostic criterion, be it test, anamnesis, or clinical examination. To be certain that the Rorschach test is a dependable diagnostic criterion in neuropsychiatry we must have first highly reliable diagnostic test procedures. A digital computer program of Rorschach interpretation, including numerous diagnostic formulae, has been written to achieve objective and perfectly reliable application of the diagnostic test rules to individual cases. The computer program will be submitted to a stringent test of validity. We shall then be in possession of a test which will yield independent and uncontaminated diagnoses. These, in turn, will be available for use independently or as part of a "total evaluation." (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Determinants of differential test validity.

In response to discussions by W. Mischel (see PA, Vol. 48:Issue 6) and D. McClelland (see PA, Vol. 48:Issue 6), attention is invited to the desirability of explicitly identifying general circumstances under which indirect assessment is superior to direct methods for assessing the same trait. Among the variables that might be considered as possibly affecting differential validity are characteristics of the criterion, including the degree to which it is subject to distortion through biased sampling of the intended trait. The susceptibility of a judge to deception by S's filtered self-presentation constitutes an appealing potential determinant of differential validity, but its effect remains to be demonstrated empirically. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Subgroup differences in predictor and criterion variances and differential validity.

Investigated whether findings of differential validity could be explained by taking account of subgroup differences in predictor and/or criterion variances. The Thorndike equations for restriction in range were used to "correct" subgroup validities for both predictor and criterion for all race and sex subgroup data sets in which subgroup validities and standard deviations were available. The data indicate that corrections for differences in predictor variances had little effect on differences in subgroup validities. Equating criterion variances in those cases in which female validities exceeded male validities had the effect of virtually eliminating the difference in validity. It is suggested that findings of differential validity may be due to methodological problems associated with the criteria used, although in the data reported here corrections for differences in variances would increase the difference in male and female validities. (12 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Biased test items and differential validity.

Tested some effects of one type of biased test items through the use of a simple psychometric model. Particular emphasis was placed on the magnitude of the difference in majority and minority group validity coefficients, and it was hypothesized that ability distributions for minority and majority (White) groups are identical, but tests are biased. An example was tested to illustrate the consequences of a profoundly biased test, but only a small difference in validity coefficients was found. The probability of detecting this difference in a standard differential validity study or by the aggregation method of J. E. Hunter et al (see record 1979-22609-001) is near the alpha level of the significance test. Thus, it cannot be assumed that minority and majority group validity coefficients will differ when a test is biased. The effects of subtest reliability on bias are discussed. Even though the test can predict criterion performance as well for minorities as for majorities, it may be inappropriate to compare the 2 types of scores. (13 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Further support for the criterion validity of the Alcohol Use Inventory.

Reports evidence for the criterion validity of the Alcohol Use Inventory (AUI) by showing that the instrument accurately identifies expected differences in a specific condition (alcohol severity) in 2 contrasting groups of alcohol abusers. 200 male veterans admitted to an inpatient alcohol program were categorized as to whether or not they had a father who abused alcohol. Ss with a father who abused alcohol scored significantly higher on AUI scales measuring alcohol abuse severity and alcohol-related problems than veterans with no such history. Differences in patterns of and benefits from alcohol use were not found. Results provide further evidence of the validity of the AUI. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Use of job information as a criterion in employment decisions of interviewers.

Conducted 2 factorially designed experiments to investigate the extent to which job information was used by interviewers as a criterion for their decisions. In Exp I 48 experienced and 48 unexperienced job interviewers received identical job information. Applicant information was either relevant or irrelevant to the job information. Ss' judgments were based on composites of relevant and irrelevant information segments. Each of the segments was either favorable or unfavorable. Exp II replicated Exp I, except that the 48 Ss did not receive job information. Findings indicate extensive use of job information. Availability of such information reduced the effect of irrelevant attributes on decisions but did not eliminate it. Use of job information was not enhanced by experience in interviewing. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Single-group validity, differential validity, and differential prediction.

The controversial topics of single-group and differential validity have been the subject of several recent articles. In line with the critiques of the original articles, it is argued that differential validity is too narrow a focus. Differential prediction and considerations of bias in selection procedures are more critical issues and require more than a comparison of correlation coefficients. (16 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Empirical test of the validity of seniority as a factor in staffing decisions.

Evaluated the appropriateness of basing staffing decisions on an employee's length of service. Policy-capturing regression models were utilized to simulate a variety of promotional policies involving seniority. Data were collected on the job changes of 162 female sewing machine operators. Past performance, seniority, and interjob similarity were examined for their ability to predict performance in the new job. The results support policies that prescribe promotion on the basis of past performance. No evidence for the validity of seniority was obtained. Interjob similarity, which was expected to moderate the validity of seniority, was found to be a significant predictor of future performance. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Enhancing the construct validity of differential response latencies for personality test items.

Evaluated the effects of increasing scale reliability and of priming dimensional activation on the construct validity of personality test item differential response latencies. 93 undergraduates were computer-administered items from the 3 lengthened scales of the Personality Research Form. Findings show that differential response latencies can demonstrate remarkably strong evidence for their construct validity. In addition, the priming of personality traits may enhance the construct validity of their corresponding differential response latencies. Results support a model of test item responding where differential item response latencies reflect the interaction of an individual's schema with test item content. (French abstract) (PsycINFO Database Record (c) 2011 APA, all rights reserved)     

Replicability as a publication criterion.

From time to time various suggestions have been made on how to reduce publication lag. It has been pointed out that raising the methodological standards by which articles are judged would help solve the lag problem and, at the same time, increase the quality of published articles. One particular methodological criterion which might help the present publication lag to disappear is that of replicability--requiring every author to satisfy the editor that his major results have been repeated before the article is considered for publication. Articles using replication designs which are not satisfactory to the editor could be given lowest publication priority. Articles with no attempt at replication would be rejected. There is good reason to anticipate that use of the repeatability criterion might do more than simply increase rejections and reduce publication lag. It seems likely that, under such a built-in reward system, more studies would be designed with replication in mind, the quality of replication (and generalization) designs would improve, and a great deal of overelaborate statistical analysis will disappear. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The effects of response instructions on situational judgment test performance and validity in a high-stakes context.

This study fills a key gap in research on response instructions in situational judgment tests (SJTs). The authors examined whether the assumptions behind the differential effects of knowledge and behavioral tendency SJT response instructions hold in a large-scale high-stakes selection context (i.e., admission to medical college). Candidates (N = 2,184) were randomly assigned to a knowledge or behavioral tendency response instruction SJT, while SJT content was kept constant. Contrary to prior research in low-stakes settings, no meaningfully important differences were found between mean scores for the response instruction sets. Consistent with prior research, the SJT with knowledge instructions correlated more highly with cognitive ability than did the SJT with behavioral tendency instructions. Finally, no difference was found between the criterion-related validity of the SJTs under the two response instruction sets. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Homogeneity of test validity.

In the present investigation we examined the extent to which the validity of three cognitive ability tests varied across test score range with respect to predicting job performance. This study was a partial replication and extension of R. Lee and P. P. Foley (see record 1987-09206-001). The 24,219 participants in the current study were obtained from the General Aptitude Test Battery (GATB) data base. Measures included in the analyses were general cognitive ability, numerical ability, and verbal ability. Results did not confirm Lee and Foley's findings. Specifically, the validity of the tests used in this study did not appreciably vary at different points along the test score range. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

On differential validity and bias.

Responds to J. E. Hunter and F. L. Schmidt's criticism (see record 1979-22562-001) that the present authors underestimated Type I bias and overestimated Type II bias in their analysis of studies of ethnic differences in employment test validity (see record 1978-02212-001). It is suggested that it is premature to dismiss the issue of differential validity. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

A field study of frame-of-reference effects on personality test validity.

As part of a test validation study at a major U.S.-based airline, the authors tested the effects of providing an "at work" frame-of-reference on the validity of the NEO Five-Factor Inventory among a sample of customer service supervisors (N = 206). Frame-of-reference moderated the validity of the Extraversion and Openness to Experience subscales after controlling for cognitive ability. In addition, the frame-of-reference personality test showed incremental validity over cognitive ability (ΔR2 = .16), but the standard personality test did not (ΔR2 = .05). The authors' discussion focuses on implications for personality theory and research and on implications for increasing the validity of personality tests in organizational settings. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Testwiseness as a factor in readiness test performance of young Mexican-American children.

Assessed the effects of training in test-taking skills on kindergarten children's performance on a school readiness test with a sample of 131 Spanish-speaking children from low socioeconomic backgrounds. Half of the Ss were trained in content-free test-taking skills, and half were provided comparable time in nondirected paper/crayon activities. All Ss were tested in both Spanish and English, with order of administration balanced. Trained Ss performed better than untrained Ss in both languages. Males were more influenced by training than were females, but overall females outperformed males. Spanish administration scores were higher than English administration scores. Order of test administration was significant in that performance in English was improved when the test was taken first in Spanish; no influence of English on subsequent performance was found. Findings support the hypothesis that test-wiseness is an important influence on test performance of young children from nonmajority backgrounds and should be taken into account in readiness assessment programs. (14 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Validity of peer nominations in predicting a distant performance criterion.

This paper reports the follow-up phase of a study of peer nominations begun in 1955 at the Naval OCS in Newport, Rhode Island. Over 700 trainees completed several peer nomination forms at various stages of training, 1 in particular on "success as a future Naval Officer" (FO). Subsequently, 639 trainees were identified who had gone on to duty as officers for about 3 yr. The average grade they secured on a key portion of the fitness report ratings assigned by their direct superior officers was used as a performance criterion; it had a split-half reliability of .90 In the prediction of this criterion, the FO peer nomination score from the 3rd wk. of training gave a validity of .40 which was as high as that for later FO scores and which was only slightly diminished after academic grades and popularity were partialed. The findings support the use of early peer nominations as a valid supplemental measure in predicting performance after training. (17 ref.) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Procedural justice as a criterion in allocation decisions.

336 undergraduates considered allocations made within 16 representative types of interpersonal relationships that varied with regard to social-emotional or task basis, cooperative or competitive style of interaction, causal or rule-directed behavior, and power distribution. Ss also rated the importance of 8 allocation criteria, such as objectivity and practicality, and of 6 criteria for judging procedural fairness, such as consistency and correctability. Results suggest that procedural justice was a more important criterion in decisions concerning resource allocations than were a variety of nonfairness criteria and that it was equal in importance to distributive justice. The meaning of procedural justice in allocations and the relation of procedural justice concerns to social allocation goals are explored. (41 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Hysterics and dysthymics as criterion groups in the study of introversion-extraversion.

The E scale (extraversion) and N scale (neuroticism) of the MPI (Mandsley Personality Inventory) were given to an hysteric-psychopathic group of 27 Ss and a dysthymic group of 25 Ss selected on the basis of unanimous diagnostic agreement among 3 psychologists. "The results suggest that either hysterics and dysthymics cannot be used in the described manner, or that the E and N scales do not measure introversion-extraversion and neuroticism, or that both statements are true." 18 references. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The check list as a criterion of proficiency.

Four typical tasks were administered to a sample of 15 naval aviation structural mechanics, whose performances in process were rated on specially-constructed check lists by one or more examiners. The end-products were then independently ranked in over-all quality from best to worst by five experts. The median correlation (rho) between pairs of expert rankings was .89 and .95 for two of the tasks, and .29 and .37 for the other two. Three of the four median correlations between rankings of the end products were not significantly different from the correlations of the expert rankings with rankings of the scores obtained from the check lists. (PsycINFO Database Record (c) 2010 APA, all rights reserved)