Validity of Army recruiter behavioral assessment: Does the assessor make a difference?

Assessment ratings of 450 soldiers by 13 US Army recruiters correlated .32 with soldiers' subsequent performance in recruiter school. Although there were significant differences between assessors in mean assessment ratings, no differences in assessment validities were found. Possible reasons for the validity findings are that the randomization test used lacks statistical power to detect actual differences in this setting, that assessor training smooths out differences in abilities related to accurate assessment, and that assessment exercises uniformly provide all assessors with good (and equal) opportunity to view behavior relevant to criterion performance. (14 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Validity of performance assessment in mathematics for early adolescents.

Reports validity evidence for a large-scale and "low-stakes" performance assessment involving 905 Grade 8 students. A subsample of 198 students was used to study the relationship of performance measures with conventional achievement and affective measures. Confirmatory factor analysis indicated that the 8 math performance tasks were unidimensional. Generalizability and dependability coefficients were .72 and .68, respectively. Also provided is other empirical validity evidence. Performance scores produced large and significant correlations with the achievement variables. Gender differences were significant for the total performance score as well as for the 2 components: concepts, procedures, and relationships, and applications and problem solving. Results are related to theory and practice. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Understanding and predicting work performance in the Canadian military.

Six empirical studies involving the application of industrial-organizational psychology in the Canadian Forces (CF) are summarized and critiqued within the framework of the performance model provided by J. P. Campbell (1990) and his associates (Campbell et al, 1993). In their model, performance is multidimensional, consisting mainly of task performance, contextual performance, counterproductive behaviour and adaptability. It is concluded that a fuller appreciation of the multidimensionality of performance is critical if the CF wishes to improve personnel staffing decisions and organizational culture. In particular, it is recommended that the CF give greater attention to post-training performance criteria, contextual (non task-specific) performance, adaptability and counterproductive behaviour. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Validity of an assessment center for identifying future female officers in the military.

Focused on a comparative evaluation of traditional selection procedures (e.g., selection interviews) with static and dynamic assessment techniques of an assessment center. The latter were designed to identify officer potential of 193 18–19 yr old women applicants for commissioned service in the military. All were admitted to officer's school, regardless of the assessment achievements obtained prior to arrival at the assessment center. The final score attained in officer training served to evaluate the predictive validity of all selection procedures compared. A high multiple correlation was found between assessment center predicting measures and final score in officer training. This validity was substantially higher than that attained in relation to the same Ss using traditional predictors (e.g., selection interviews). (29 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Validity of peer nominations in predicting a distant performance criterion.

This paper reports the follow-up phase of a study of peer nominations begun in 1955 at the Naval OCS in Newport, Rhode Island. Over 700 trainees completed several peer nomination forms at various stages of training, 1 in particular on "success as a future Naval Officer" (FO). Subsequently, 639 trainees were identified who had gone on to duty as officers for about 3 yr. The average grade they secured on a key portion of the fitness report ratings assigned by their direct superior officers was used as a performance criterion; it had a split-half reliability of .90 In the prediction of this criterion, the FO peer nomination score from the 3rd wk. of training gave a validity of .40 which was as high as that for later FO scores and which was only slightly diminished after academic grades and popularity were partialed. The findings support the use of early peer nominations as a valid supplemental measure in predicting performance after training. (17 ref.) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Validity and fairness of physical ability tests for predicting performance in craft jobs.

Problems relating to performance, accidents, and turnover in outdoor telephone craft jobs stimulated 2 experiments aimed at developing and validating a physical test battery. Based on job analysis results, a battery of 9 measures was administered to a sample of 128 Ss (83 males and 45 females) in Exp I. A 2-test battery (dynamic arm strength and reaction time), valid for predicting job task performance and turnover, was selected. Regression equations for males and females were not significantly different. Exp II included a sample of 210 Ss (132 males and 78 females). A 3-test battery consisting of a body density measure, a balance test, and a static strength test was selected based on relationships with training performance. No significant differences were found in the regression equations for males compared to females. The Exp II battery was also significantly related to field performance, training completion, and accidents and was valid for the Exp I criteria. (18 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Validity of assessment center ratings for the prediction of performance ratings and school climate of school administrators.

Describes the validation of an assessment center used to select school administrators on the basis of ratings of 153 school administrators who had participated in the center between 1976 and 1981. Behaviorally anchored rating scales were obtained from supervisors, teachers, and support staff; and measures of 7 school climate dimensions were provided by teachers, students, and support staff. Results show a significant relationship between an overall assessment center placement recommendation and supervisory, teacher, and support staff ratings on most performance dimensions. Climate measures were significantly related to few assessment-center ratings regardless of the source of the climate measure. It is suggested that the assessment center proved valid for a sample from widely distant geographic areas and school districts of differing sizes and levels. (16 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Validity issues in clinical assessment.

The data investigators should provide to support the validity of inferences they make based on scores from a measure depend on (a) whether the measure is assumed to assess a hypothetical construct or behavior and (b) the purposes for which the measure is intended. The authors distinguish between the representational phase of validity assessment, which establishes that a measure produces scores that reflect the construct or behavior it purports to assess, and the elaborative validity phase, in which the meaning and utility of scores are examined. Key issues relevant to convergent, discriminant, and criterion-related validity are examined for measures of latent traits or constructs and then for measures of behaviors or response classes. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Validity of expectancy theory as a within-person behavioral choice model for sales activities.

Using 62 female life insurance sales representatives (mean age 36.5 yrs) in Japan, the present study hypothesized, based on an expectancy model of choice, that when choosing among 6 types of policies to sell, Ss would more frequently choose the policy associated with the larger force score. Ss were required to fill out a questionnaire containing measures of the expectancy of sales success, the valence for 4 self-generated outcomes, and the instrumentality of sales success for those outcomes. The number of policies sold was used as an index of performance for each policy. An overall contingency table based on the item ranks indicated significant relationships between force and performance. The average within-person Pearson product-moment correlation coefficient for force and performance for those 6 policy types was .50. Implications for expectancy theory as a within-person choice model and some methodological problems involved in the present study are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Validity of an operational management assessment program.

Examined the validity of an assessment program by reviewing the career progress of 1,086 employees in sales, service, and administrative functions after they were assessed. Although Ss participating in the program were nominated on the basis of being promotable, raters found that more than 1/4 were unqualified, and the others were widely differentiated. Ratings were used to move Ss into 1st-line management, but the relationship of ratings to 1st promotions was moderate enought to reduce fears of "crown prince" or "kiss of death" effects. Participation did not seem to demotivate these Ss. Assessment ratings substantially correlated with 2 major criteria, 2nd-level promotions and demotions from 1st-line management. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Evaluation of a peer rating system for predicting subsequent promotion of senior military officers.

Evaluated the validity of 1,236 peer ratings for predicting 886 US Army senior officers' promotion. A nomination technique within career field groups was used and produced split-half (group) reliabilities primarily in the high 80s and low 90s. Although there was a moderate degree of rater resistance to their operational use, the peer ratings were highly predictive of promotion to general, both for the total group and when education level was partialed out. Results extend the generality of peer rating beyond the previous restrictive training environment for junior officers into the realistic work situation and for senior level individuals. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Validity and fairness of physical ability tests for predicting performance in craft jobs: Correction to Reilly, Zedeck, and Tenopyr.

Reports an error in the original article by Richard R. Reilly, Sheldon Zedeck, and Mary L. Tenopyr (Journal of Applied Psychology, 1979, Vol. 64, No. 3, pp. 262-274). In the Results section of Experiment 2 several results were reported incorrectly. The corrected results are provided. (The following abstract of this article originally appeared in record 1980-26872-001.) Problems relating to performance, accidents, and turnover in outdoor telephone craft jobs stimulated 2 experiments aimed at developing and validating a physical test battery. Based on job analysis results, a battery of 9 measures was administered to a sample of 128 Ss (83 males and 45 females) in Exp I. A 2-test battery (dynamic arm strength and reaction time), valid for predicting job task performance and turnover, was selected. Regression equations for males and females were not significantly different. Exp II included a sample of 210 Ss (132 males and 78 females). A 3-test battery consisting of a body density measure, a balance test, and a static strength test was selected based on relationships with training performance. No significant differences were found in the regression equations for males compared to females. The Exp II battery was also significantly related to field performance, training completion, and accidents and was valid for the Exp I criteria. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The assessment of anxiety by physiological behavioral measures.

There is little research to support the contention that clearly defined patterns of physiological-behavioral responses associated with anxiety arousal can be distinguished from other arousal patterns. Intercorrelations among physiological measures of anxiety obtained under either resting states or under stress are generally low. No studies have been reported in which several measures of behavior were obtained simultaneously with a variety of physiological measures under conditions likely to be fear arousing. From Psyc Abstracts 36:02:2HN34M. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Validity and utility of alternative predictors of job performance.

Meta-analysis of the cumulative research on various predictors of job performance showed that for entry-level jobs there was no predictor with validity equal to that of ability, which had a mean validity of .53. For selection on the basis of current job performance, the work sample test, with mean validity of .54, was slightly better. For federal entry-level jobs, substitution of an alternative predictor would cost from $3.12 (job tryout) to $15.89 billion/year (age). Hiring on ability had a utility of $15.61 billion/year but affected minority groups adversely. Hiring on ability by quotas would decrease utility by 5%. A 3rd strategy—using a low cutoff score—would decrease utility by 83%. Using other predictors in conjunction with ability tests might improve validity and reduce adverse impact, but there is as yet no database for studying this possibility. (89 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The role of mindfulness in predicting individual performance.

Mindfulness refers to an individual difference variable regarding the degree to which a person is in the present moment (K. W. Brown & R. M. Ryan, 2003). Despite a growing interest in the benefits of mindfulness in health and clinical outcomes, little research has explored whether mindfulness relates to individual performance. The authors examined whether mindfulness was related to performance among a group of MBA students (N = 149). The results show that mindfulness interacted with gender to predict performance. Specifically, the positive association between mindfulness and performance was stronger for women than for men. Implications and future directions are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Self-monitoring as an assessment strategy in behavioral medicine.

This article describes assessment issues in behavioral medicine and a typology of uses of self-monitoring in behavioral medicine assessment. Illustrative examples of the use of self-monitoring in behavioral medicine assessment are presented. Current issues, and methodological concerns, in self-monitoring for behavioral medicine assessment are described along with some ideas for solutions of these issues and concerns. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Ability–personality interactions when predicting job performance.

The general proposition that performance is a multiplicative function of ability and motivation has a long-standing history. Three recent studies have reported results that suggest that shifting from an additive model to a multiplicative model may improve efforts to predict performance. This article represents an extensive examination of this multiplicative proposition when motivation is conceptualized in terms of personality characteristics. The Project A database, the Management Continuity Study database, and 2 additional data sets were brought together to facilitate a systematic investigation concerning whether ability and personality interact when predicting performance. Contrary to expectations, the results indicate that ability–personality interactions are not detected at above chance levels. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The validity of behavioral rating scale items for the assessment of individual creativity.

In the study of creativity, how would it do to have statements about creative people obtained in an unstructured interview form the basis for rating research personnel? After trial with 78 Ss, the data was subjected to linkage analysis. 8 types of items were identified. "It was suggested that the items presented may be valid discriminators between relatively more or less creative persons in a wide variety of research areas." (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

School psychologists' acceptability of behavioral and traditional assessment procedures for externalizing problem behaviors.

A national survey of school psychologists' acceptability of psychological assessment techniques in the area of externalizing problem behaviors was examined. School psychologists' ratings of 2 procedures, Behavioral Assessment (BA) and Traditional Assessment (TA), were compared using an analog case study methodology. Using a random sample of 500 National Association of School Psychologists members from the 1991–1992 membership directory, 339 Ss completed the Assessment Rating Profile after reading a hypothetical case summary describing a student with suspected externalizing problem behaviors. Results indicated statistically significant differences between assessment methods and ratings of acceptability. BA procedures were found to be more acceptable than TA procedures. Implications related to the acceptability of psychological assessment techniques, as well as implications for school psychology practitioners are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

An integrative typology of personality assessment for aggression: Implications for predicting counterproductive workplace behavior.

This study presents an integrative typology of personality assessment for aggression. In this typology, self-report and conditional reasoning (L. R. James, 1998) methodologies are used to assess 2 separate, yet often congruent, components of aggressive personalities. Specifically, self-report is used to assess explicit components of aggressive tendencies, such as self-perceived aggression, whereas conditional reasoning is used to assess implicit components, in particular, the unconscious biases in reasoning that are used to justify aggressive acts. These 2 separate components are then integrated to form a new theoretical typology of personality assessment for aggression. Empirical tests of the typology were subsequently conducted using data gathered across 3 samples in laboratory and field settings and reveal that explicit and implicit components of aggression can interact in the prediction of counterproductive, deviant, and prosocial behaviors. These empirical tests also reveal that when either the self-report or conditional reasoning methodology is used in isolation, the resulting assessment of aggression may be incomplete. Implications for personnel selection, team composition, and executive coaching are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)