Male-female performance on U.S. Air Force pilot selection tests

BACKGROUND: Recently, the U.S. Air Force conducted several studies to examine sex differences on pilot selection tests and training performance. METHODS: Research has focused on mean score performance, the structure of ability, the predictive utility of pilot selection tests, and the causal role of ability and prior flying knowledge in the acquisition of additional flying knowledge and flying skills. DISCUSSION: Despite male-female mean score differences on pilot selection tests, confirmatory factor analyses indicated that the same factors were being measured for both groups. In studies of predictive bias, no evidence of differential validity was found for male vs. female pilot trainees. An examination of causal models of ability and prior flying knowledge on the acquisition of additional flying knowledge and flying skills showed similar structure for men and women.     

Impact of job experience and ability on job knowledge, work sample performance, and supervisory ratings of job performance.

Based on data from 4 independent studies reported by R. Vineberg and E. N. Taylor (1972) with a total sample size of 1,474, path analysis was used to examine the causal impact of job experience on job knowledge, performance capability as measured by job sample tests, and supervisory ratings of job performance. Findings support the conclusion that (1) when mean job experience is 2–3 yrs, there is substantial variance in job experience and (2) when the jobs are of an intermediate complexity level, job experience has a substantial direct impact on job knowledge and a smaller direct impact on performance capabilities as assessed by job sample measures. Job experience also has a substantial indirect effect on work sample performance through its effect on job knowledge, which, in turn, was found to be the strongest determinant of work sample performance. The pattern and magnitude of causal effects of general mental ability were similar to those of job experience. The effect of job knowledge on supervisory ratings was several times stronger than the effect of job sample performance, confirming the findings of J. E. Hunter (1983). When job experience was held constant, the direct impact of ability on the acquisition of job knowledge increased substantially, and this, in turn, increased the indirect effect of ability on job sample performance. (28 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Motivation and cognitive abilities: An integrative/aptitude-treatment interaction approach to skill acquisition.

Two central constructs of applied psychology, motivation and cognitive ability, were integrated within an information-processing (IPR) framework. This framework simultaneously considers individual differences in cognitive abilities, self-regulatory processes of motivation, and IPR demands. Evidence for the framework is provided in the context of skill acquisition, in which IPR and ability demands change as a function of practice, training paradigm, and timing of goal setting (GS). Three field-based lab experiments were conducted with 1,010 US Air Force trainees. Exp 1 evaluated the basic ability–performance parameters of the air traffic controller task and GS effects early in practice. Exp 2 evaluated GS later in practice. Exp 3 investigated the simultaneous effects of training content, GS and ability–performance interactions. Results support the theoretical framework and have implications for notions of ability–motivation interactions and design of training and motivation programs. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Predicting job performance: Not much more than g..

The roles of general cognitive ability (g) and specific abilities or knowledge (s) were investigated as predictors of work sample job performance criteria in 7 jobs for US Air Force enlistees. Both g and s (the interaction of general ability and experience) were defined by scores on the first and subsequent principal components of the enlistment selection and classification test (the Armed Services Vocational Aptitude Battery). Multiple regression analyses, when corrected for range restriction, revealed that g was the best predictor of all criteria and that s added a statistically significant but practically small amount to predictive efficiency. These results are consistent with those of previous studies, most notably Army Project A (J. J. McHenry et al; see record 1990-27146-001). The study also extends the findings to other jobs and uses traditionally more acceptable estimates of g, application of effective sample size in cross-validation estimation, and new performance criteria. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Adaptability measures as predictors of performance ratings.

How well do measures of temperamental and motivational characteristics in training predict success of performance of officers as indicated by Officer Effectiveness Ratings? S were selected from 666 cadets at one base and 69 at another. Variables used were 3 global indicies of adaptability and 10 variables representing the common factors. "(a) measures of adaptability to training and Air Force Life are more highly related to later officer performance than are measures of aptitude or ability; (b) assessments of a man's functioning involving personal judgments of peers, superiors, and experts are predictive of later performance as an Air Force officer." (PsycINFO Database Record (c) 2011 APA, all rights reserved)     

The stability of validity coefficients over time: Ackerman's (1988) model and the General Aptitude Test Battery.

This study examined P. L. Ackerman's (1988) model of skill acquisition within an applied setting. Differences were examined between jobs on the basis of task consistency: changes in performance variability across experience, learning curves, and stability of ability–performance correlations across experience. Results showed the degree of task consistency influenced the shape of learning curves, with jobs composed of primarily consistent tasks improving more rapidly and reaching asymptote sooner. In addition, trends in ability–performance correlations were moderated by the degree of task consistency within a job. Specifically, for jobs with primarily consistent tasks, general cognitive ability best predicted early performance whereas psychomotor ability best predicted later performance. In contrast, general cognitive ability was the strongest predictor across experience for jobs with primarily inconsistent tasks. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

An interview approach to work sample criterion measurement.

A variety of measurement techniques have been used to assess job proficiency, including hands-on work sample tests, job knowledge tests, and ratings. The current research effort examined the viability of an interview approach to work sample criterion measurement. Hands-on and interview work sample measures were developed and administered to 1,491 enlisted personnel in 8 Air Force specialties. Job-level analyses yielded moderate (.46) to high (.84) correlations but produced significant mean differences (6 of 8 specialties) between the 2 techniques. In addition, the patterns of relationships between each work sample test and a series of related performance, aptitude, and experience measures were found to be quite similar, and the magnitudes of these correlations were similar to those in studies reported previously. In contrast, task-level analyses yielded considerable variability in correlational values (rs ranged from .19 to .97) and mean differences (39 of 64 mean comparisons were significantly different). Results are discussed in terms of the usefulness of interview testing as a work sample measurement methodology for different personnel functions. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Team effectiveness: Beyond skills and cognitive ability.

On the basis of job analysis results, the validity of using measures of general cognitive ability, job-specific skills, and personality traits jointly at both the individual level and the group level to predict the performance of 79 four-person, human resource work teams was evaluated. Team member trait and job skill scores were aggregated with a conjunctive model of task performance. At the individual level of analysis, measures of personality (i.e., Agreeableness and Conscientiousness) predicted peer ratings of team member performance beyond measures of job-specific skills and general cognitive ability. Similarly, at the group level of analysis, both Agreeableness and Conscientiousness predicted supervisor ratings of work team performance, objective measures of work team accuracy, and work completed. At both the individual and group levels, the trait of Agreeableness predicted Interpersonal Skills. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Intelligence, training, and employment.

Validity coefficients of cognitive ability tests for predicting job performance are impressive. Nevertheless, a need exists to supplement cognitive ability tests with additional predictors to maximize validity and utility and to minimize adverse impact, even for tests that are unbiased according to psychometric criteria. An agenda for research and practice is presented that includes building on the existing validity of cognitive ability tests for job selection by expanding both the set of constructs used to predict job performance and the set of performance criteria used to quantify job performance. Even greater benefits are likely to be associated with advances in knowledge about effective training. Such advances will require substantial investment in well-designed, large-scale studies and facile adaptation of lessons from other domains, such as cognitive psychology, to the organizational-training context. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Relationship of early work challenge to job performance, professional contributions, and competence of engineers.

Investigated the relationship of the work challenge experienced by 85 engineers at the beginning of their career to their subsequent job performance, professional contributions, and competence. Engineers whose initial job experience involved challenging technical work tended to contribute to knowledge relatively early and to maintain professional competence and good job performance during their careers. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Construct models of task performance.

A covariance structure model was tested that related 3 classes of predictors (ability, experience, and supervisor support) to several classes of criteria of task performance (rated performance by self, supervisor, and peers, a work sample test, and training success). The model was tested for 3 categories of tasks (installation of engine parts, inspection of components, and forms completion), using data for 201 first-term U.S. Air Force jet engine mechanics. Ability explained significant variance in training success for 3 categories, in rated performance for 2 categories, and in work sample performance for 1 category. Experience explained substantial variance in rated performance for 3 categories, and in work sample performance for 2 categories. Supervisor support did not explain significant variance in any category. The task models have potential usefulness as the beginnings of a generalizable task taxonomy. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Toward an integrative theory of training motivation: A meta-analytic path analysis of 20 years of research.

This article meta-analytically summarizes the literature on training motivation, its antecedents, and its relationships with training outcomes such as declarative knowledge, skill acquisition, and transfer. Significant predictors of training motivation and outcomes included individual characteristics (e.g., locus of control, conscientiousness, anxiety, age, cognitive ability, self-efficacy, valence, job involvement) and situational characteristics (e.g., climate). Moreover, training motivation explained incremental variance in training outcomes beyond the effects of cognitive ability. Meta-analytic path analyses further showed that the effects of personality, climate, and age on training outcomes were only partially mediated by self-efficacy, valence, and job involvement. These findings are discussed in terms of their practical significance and their implications for an integrative theory of training motivation. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Phonological awareness and reading: A chicken and egg problem?

Investigated the causal relationship between metaphonological skills (sound blending and segmentation) and reading/spelling in 2 experiments. In Exp I, data on 46 dyslexic and 44 control Ss in Grades 1–6 were drawn from a longitudinal study in Sweden of 723 Ss so that a number of causal models could be tested by LISREL. Results show that early spelling ability is primarily dependent on metaphonological abilities and only indirectly affected by linguistic and cognitive development. The causal model for reading obtaining empirical support was similar with the exception that it also included a direct causal path from cognitive development to reading. In Exp II, 38 1st graders were assigned to metaphonological training for 8 sessions or to general language activities (control). Ss were pre- and posttested on reading, spelling, sound blending, and segmentation. Results show that training had significant effects on metaphonological skills. Among Ss with the lowest metaphonological pretest performance, specific metaphonological training improved spelling performance more than did general language activities. Findings show that metaphonological abilities have a causal effect on reading and spelling, but no support was found for the reverse causal influence. (26 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Learning from instructional text: Test of an individual-differences model.

An individual-differences model of learning from instructional text was tested on 211 Air Force recruits. The results strongly supported the model, with a high level of fit for a structural equation modeling analysis using LISREL 8 (K. J?reskog & D. S?rbom, 1993). According to the model, learning from text is determined by making connections among ideas, including ideas from the text and from prior knowledge. Making the connections depends in part on 4 variables that differ among individuals: metacognition, specifically the ability to sense that one's mental representation is not coherent and consequently needs extra connections; inference-making ability, which spreads activation among the elements within working memory, which provides the arena in which the connections are made; and domain knowledge, which provides prior knowledge elements for use by the inference-making process to make connections. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Job experience correlates of job performance.

Although measures of job experience are frequently-used screening devices in the selection of employees, personnel psychologists have devoted little attention to their usefulness. This article quantitatively summarizes data on the relation between job experience and job performance from a total sample of 16,058. The correlation between job experience and job performance was found to be moderated by two variables: length of experience and job complexity. The highest correlations were obtained in populations with low mean levels of job experience and for jobs that place low levels of cognitive demands on employees. Results appear to be consistent with the causal model of job performance proposed by Schmidt, Hunter, and Outerbridge (1986). (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Interpersonal facilitation and job dedication as separate facets of contextual performance.

This study attempts to refine the construct of contextual performance by dividing it into 2 narrower constructs, interpersonal facilitation and job dedication. Supervisors rated 975 U.S. Air Force mechanics on at least 1 of 4 aspects of job performance (different supervisors rated each aspect of performance), and 515 of these mechanics also completed self-report individual difference measures. Correlations between performance ratings and individual difference variables support distinguishing task performance from interpersonal facilitation but not from job dedication. Thus this study suggests the need to redefine task performance to include motivational elements of job dedication. Then task performance would include task proficiency and motivation to perform one's own tasks effectively, and contextual performance would include interpersonal skills, the motivation to maintain good working relationships and help others perform their tasks. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Development and validation of minicourses in the telecommunication industry.

Beginning in the early 1970s, American Telephone and Telegraph (AT&T) and the associated Bell companies have used minicourses for selecting personnel in a variety of jobs involving new technologies. The minicourse approach uses material representative of training content to assess the candidate's ability to acquire knowledge relevant to full-scale training and the job. A major advantage of the minicourse approach is the ability to implement a selection procedure concurrent with the introduction of a new training program. Although minicourses are initially validated using a content-oriented strategy, the AT&T model has included criterion-related validation where possible. The experience of AT&T with seven minicourses is discussed, and validities for training criteria and job performance are reported. Minicourses are a specific example of the general class of predictors known as trainability tests. The results of a meta-analysis indicate that validities for trainability tests are comparable with validities found for ability tests and work sample tests by Hunter and Hunter (1984). (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Evaluating potential officer effectiveness in a training situation.

A situational performance test was developed for evaluating potential Air Force officer effectiveness during the officer training program. For the 480 members of an AF Candidate School graduating class reliability of scoring (examiner-observer agreements on a behavior check list) was .75. Correlations between the total test score and Officer Candidate School evaluation measures were in the range .21—.25. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Psychomotor abilities tests as predictors of training performance.

Examined the predictive and incremental validity of three psychomotor ability measures (manual dexterity, finger dexterity, and motor coordination) in 209 Canadian Forces personnel being trained in technical and mechanical occupations. For both types of occupations combined, manual dexterity predicted training performance (r=.18), as it did for the Technical (r=.22) and Mechanical (r=.17) families, separately Finger Dexterity (r=.02) and Motor Coordination (r=.05) did not predict training performance for either the combined group or for each family by itself. The addition of the three psychomotor measures increased validity beyond what was predicted by cognitive ability in the combined occupations (δ R2=.05); however, only Manual Dexterity (β=.26) made a significant contribution to the regression model. Similar, though nonsignificant, increases in predictive validity occurred within each family. The results from this study suggest that specific abilities, when determined through a job analysis, improve predictions based solely on cognitive ability. (PsycINFO Database Record (c) 2010 APA, all rights reserved)