Validity of personnel decisions: A conceptual analysis of the inferential and evidential bases.

Issues common to both the process of building psychological theories and validating personnel decisions are examined. Inferences linking psychological constructs and operational measures of constructs are organized into a conceptual framework, and validation is characterized as the process of accumulating various forms of judgmental and empirical evidence to support these inferences. The traditional concepts of construct-, and content-, and criterion-related validity are unified within this framework. This unified view of validity is then contrasted with more conventional views (e.g., Uniform Guidelines, 1978), and misconceptions about the validation of employment tests are examined. Next, the process of validating predictor constructs is extended to delineate the critical inferences unique to validating performance criteria. Finally, an agenda for programmatic personnel selection research is described, emphasizing a shift in the behavioral scientist's role in the personnel selection process. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

A meta-analysis of the situational interview: An enumerative review of reasons for its validity.

The premise underlying the situational interview (SI) is that intentions predict behaviour. The distinguishing characteristics of the SI include the depiction of a dilemma in the interview question, the use of an item scoring guide, and summative scoring of items to yield a total interview score. A meta-analysis of SI studies indicated that the weighted mean observed criterion-related validity of the SI is .35 (n?=?1010), and the mean corrected criterion-related validity is .47. An enumerative review of the literature revealed that the SI is applicable across a diversity of jobs, participants, performance criteria, and countries. The criterion-related validation evidence for the SI is explained in terms of content, construct, and incremental validity, freedom from interviewer biases, and practicality. The article concludes with a comparison of the SI to interviews that measure either past behaviours or future intentions where the interview question does not contain a dilemma or scoring guide, and performance dimensions rather than each individual item are scored. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

On Selecting Psychology Graduate Students: Validity Evidence for a Test of Tacit Knowledge.

This study provides initial evidence for the criterion-related validity of tacit knowledge (TK) as an alternative measure for selecting psychology graduate students and adds insight to the construct of TK by evaluating its factor structure, assessing convergent relationships with other variables, and exploring alternative reasons for why TK predicts performance. TK provided incremental validity beyond GRE scores, undergraduate GPA, and 7 personality dimensions in predicting graduate school performance. Moreover, TK scores computed from experts at different universities yielded nearly identical criterion-related validity coefficients, thus strengthening the generalizability of the predictor. Although the results of the present study demonstrate convergent relationships between TK and conceptually related variables, factor analyses conducted on the TK measure did not support the subdimensions proposed by the authors of the scale. Limitations of the present study, issues involving revision of the TK measure, and recommendations for future research are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Validity issues in clinical assessment.

The data investigators should provide to support the validity of inferences they make based on scores from a measure depend on (a) whether the measure is assumed to assess a hypothetical construct or behavior and (b) the purposes for which the measure is intended. The authors distinguish between the representational phase of validity assessment, which establishes that a measure produces scores that reflect the construct or behavior it purports to assess, and the elaborative validity phase, in which the meaning and utility of scores are examined. Key issues relevant to convergent, discriminant, and criterion-related validity are examined for measures of latent traits or constructs and then for measures of behaviors or response classes. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Concurrent validity and screening efficiency of The Dyslexia Screener.

The concurrent criterion-related validity and screening efficiency of a recently developed scale, The Dyslexia Screener (TDS), was investigated by examining its relation to the Reading Cluster of the Woodcock-Johnson Psychoeducational Assessment Battery (WJ) in a sample of 100 nonreferred children participating in a longitudinal study. Results supported the validity of the TDS; TDS scores correlated moderately with the WJ Reading Cluster and its subtests, p?     

Polygraph testing: Thoughts of a skeptical legislator.

Following a review of federal policies on polygraph testing for investigations and preemployment screening in the 1960s and 1970s, the events following a 1983 Reagan directive requiring polygraph testing in the investigation of security leaks are detailed. The General Accounting Office and the Office of Technology Assessment (OTA) were asked by Congress to examine the validity and reliability of polygraph testing. OTA found serious limitations and widely differing results in its review of polygraph validity and reliability research. As a result legislation was introduced to limit polygraph use in criminal investigation and prohibit its use for any screening purposes. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Faking and the validity of conscientiousness: A Monte Carlo investigation.

The article reports the findings from a Monte Carlo investigation examining the impact of faking on the criterion-related validity of Conscientiousness for predicting supervisory ratings of job performance. Based on a review of faking literature, 6 parameters were manipulated in order to model 4,500 distinct faking conditions (5 [magnitude] × 5 [proportion] × 4 [variability] × 3 [faking-Conscientiousness relationship] × 3 [faking-performance relationship] × 5 [selection ratio]). Overall, the results indicated that validity change is significantly affected by all 6 faking parameters, with the relationship between faking and performance, the proportion of fakers in the sample, and the magnitude of faking having the strongest effect on validity change. Additionally, the association between several of the parameters and changes in criterion-related validity was conditional on the faking-performance relationship. The results are discussed in terms of their practical and theoretical implications for using personality testing for employee selection. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Open a new window: Validities and values in psychological measurement.

Discusses the issue of validity in testing, with reference to new Federal equal employment regulations. Definitions of testing terms are presented and illustrated with examples from a study of packers. Definitions include criterion-related validity, construct validity, and job-relatedness. Validity is not equated with job-relatedness, and criterion-related validity is only conditional evidence of job-relatedness. The importance of considering values in hypothesis determination is stressed. Organizational climate and information content are also considered. (19 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Psychometric properties of the Drug Abuse Screening Test in psychiatric outpatients.

Individuals with a severe mental illness frequently have substance abuse and dependence problems, placing them at increased risk for poor treatment outcome. However, the reliability and validity of self-report measures assessing substance abuse and dependence remains understudied in this population. This investigation evaluates 2 versions of the Drug Abuse Screening Test (DAST; H. Skinner, 1982) as screening tools for an outpatient psychiatric sample. Participants were 73 men and 24 women who had been receiving treatment at a public psychiatric facility. All participants completed the DAST along with other measures of substance use and psychiatric status. The DAST demonstrated adequate internal consistency and temporal stability in this sample. Factor analysis supports a multidimensional scale. We evaluated criterion-related, concurrent and discriminant evidence for validity, and we have concluded that both tested versions of the DAST have sound psychometric properties when used with psychiatric outpatients. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Mentors' perceptions of negative mentoring experiences: Scale development and nomological validation.

Three studies were conducted to develop a psychometrically sound, multidimensional measure of mentors' perceptions of negative experiences with their protégés. In Study 1, items were developed, and content-related validity was established. In Study 2, CFA was used to establish the dimensionality of the new measure. Construct-related (convergent and discriminant) and criterion-related validity evidence were also obtained by using data from matched mentor-protégé dyads. Study 3 replicated the factor structure of the instrument and provided additional validity evidence by using a sample of female academic mentors. The findings are discussed in terms of broadening the scope of mentoring research to consider the mentor's perspective of relationship problems and dyadic processes in mentoring relationships. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Personality scale validities increase throughout medical school.

Admissions and personnel decisions rely on stable predictor–criterion relationships. The authors studied the validity of Big Five personality factors and their facets for predicting academic performance in medical school across multiple years, investigating whether criterion-related validities change over time. In this longitudinal investigation, an entire European country’s 1997 cohort of medical students was studied throughout their medical school career (Year 1, N = 627; Year 7, N = 306). Over time, extraversion, openness, and conscientiousness factor and facet scale scores showed increases in operational validity for predicting grade point averages. Although there may not be any advantages to being open and extraverted for early academic performance, these traits gain importance for later academic performance when applied practice increasingly plays a part in the curriculum. Conscientiousness, perhaps more than any other personality trait, appears to be an increasing asset for medical students: Operational validities of conscientiousness increased from .18 to .45. In assessing the utility of personality measures, relying on early criteria might underestimate the predictive value of personality variables. Implications for personality measures to predict work performance are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

On Trinitarian doctrines of validity.

The 3 conventionally listed aspects of validity—criterion-related, content, and construct—are examined from a dual perspective: aiding in the understanding of a construct and establishing a basis for comparison between evaluations of the validity of measurement and evaluations of the validity of a hypothesis. The unifying nature of the validity of measurement is found in the degree to which results of measurement (the numbers or scores) represent magnitudes of the intended attribute. Validity is an evaluative judgment based on a variety of considerations, including the structure of the measurement operations, the pattern of correlations with other variables, and the results of confirmatory and disconfirmatory investigations. Validity in this sense is close to the concept of construct validity but without the theoretical implications of that term; like construct validity, the evaluation cannot be expressed with a single research result. It is suggested that evaluations of the validity of hypotheses should be based on multiple considerations rather than on single coefficients. (23 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The use of person-organization fit in employment decision making: An assessment of its criterion-related validity.

Because measures of person-organization (P-O) fit are accountable to the same psychometric and legal standards used for other employment tests when they are used for personnel decision making, the authors assessed the criterion-related validity of P-O fit as a predictor of job performance and turnover. Meta-analyses resulted in estimated true criterion-related validities of .15 (k = 36, N = 5,377) for P-O fit as a predictor of job performance and .24 (k = 8, N = 2,476) as a predictor of turnover, compared with a stronger effect of .31 (k = 109, N = 108,328) for the more commonly studied relation between P-O fit and work attitudes. In contrast to the relations between P-O fit and work attitudes, the lower 95% credibility values for the job performance and turnover relations included zero. In addition, P-O fit's relations with job performance and turnover were partially mediated by work attitudes. Potential concerns pertaining to the use of P-O fit in employment decision making are discussed in light of these results. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Comparison of factor structures and criterion-related validity coefficients for two measures of personality based on the five factor model.

The factor structure and criterion-related validity for 2 types of personality measures that are based on 5 factor models were studied. Confirmatory factor analysis was used to compare a 5 versus 6 factor model in an applied setting. In addition, criterion-related validity coefficients were examined for the 2 inventories. Two Big Five measures were used—1 was an adjective-based bipolar inventory and the other was a questionnaire (NEO-Personality Inventory; PI)—to shed light on the relationships between inventory characteristics, factor structure, and criterion-related validity. The sample consisted of 423 flight attendant trainees. Results indicated that the 6 factor model provided a better fit for both measures compared with the 5 factor model. Scales from the NEO-PI were significantly correlated with measures of training success, whereas scales from the bipolar inventory were not. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The Team Role Test: Development and validation of a team role knowledge situational judgment test.

The main objectives in this research were to introduce the concept of team role knowledge and to investigate its potential usefulness for team member selection. In Study 1, the authors developed a situational judgment test, called the Team Role Test, to measure knowledge of 10 roles relevant to the team context. The criterion-related validity of this measure was examined in 2 additional studies. In a sample of academic project teams (N = 93), team role knowledge predicted team member role performance (r = .34). Role knowledge also provided incremental validity beyond mental ability and the Big Five personality factors in the prediction of role performance. The results of Study 2 revealed that the predictive validity of role knowledge generalizes to team members in a work setting (N = 82, r = .30). The implications of the results for selection in team environments are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The development and validation of the Workplace Ostracism Scale.

This article outlines the development of a 10-item measure of workplace ostracism. Using 6 samples (including multisource and multiwave data), the authors developed a reliable scale with a unidimensional factor structure that replicated across 4 separate samples. The scale possessed both convergent and discriminant validity, and criterion-related validity was demonstrated through the scale's relation with basic needs, well-being, job attitudes, job performance, and withdrawal. Overall, the present study suggests that the Workplace Ostracism Scale is a reliable and valid measure and that the workplace ostracism construct has important implications for both individuals and organizations. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Impact of Elaboration on Socially Desirable Responding and the Validity of Biodata Measures.

The current study investigated the impact of requiring respondents to elaborate on their answers to a biodata measure on mean scores, the validity of the biodata item composites, subgroup mean differences, and correlations with social desirability. Results of this study indicate that elaborated responses result in scores that are much lower than nonelaborated responses to the same items by an independent sample. Despite the lower mean score on elaborated items, it does not appear that elaboration affects the size of the correlation between social desirability and responses to biodata items or that it affects criterion-related validity or subgroup mean differences in a practically significant way. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Use of situational judgment tests to predict job performance: A clarification of the literature.

Although situational judgment tests have a long history in the psychological assessment literature and continue to be frequently used in employment contexts, there has been virtually no summarization of this literature. The purpose of this article is to review the history of such tests and present the results of a meta-analysis on criterion-related and construct validity. On the basis of 102 coefficients and 10,640 people, situational judgment tests showed useful levels of validity (p?=?.34) that were generalizable. A review of 79 correlations between situational judgment tests and general cognitive ability involving 16,984 people indicated that situational judgment tests typically evidence relationships with cognitive ability (p?=?.46). On the basis of the literature review and meta-analytic findings, implications for the continued use of situational judgment tests are discussed, particularly in terms of recent investigations into tacit knowledge. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Moving beyond content validation of nursing diagnoses

TOPIC: Studies to establish construct and criterion-related validity of nursing diagnoses. PURPOSE: The overwhelming majority of previous studies addressed content validation by nurse experts. This paper describes strategies to move beyond content validation research to construct and criterion-related validation. SOURCES: The range of studies that should be conducted for development of nursing diagnoses are reviewed with examples drawn from the field of psychology. Existing studies on the diagnoses of ineffective breathing pattern, ineffective airway clearance and impaired gas exchange are used as examples. CONCLUSIONS: Many types of studies are needed for each nursing diagnosis. Increased funding and support for nursing diagnosis research will be facilitated by attention to the accuracy of nurses' diagnoses and outcomes of the diagnostic process.