Evaluating frame-of-reference rater training effectiveness using performance schema accuracy.

Frame-of-reference training has been shown to be an effective intervention for improving the accuracy of performance ratings (e.g., Woehr & Huffcutt, 1994). Despite evidence in support of the effectiveness of frame-of-reference training, few studies have empirically addressed the ultimate goal of such training, which is to teach raters to share a common conceptualization of performance (Athey & McIntyre, 1987; Woehr, 1994). The present study tested the hypothesis that, following training, frame-of-reference–trained raters would possess schemas of performance that are more similar to a referent schema, as compared with control-trained raters. Schema accuracy was also hypothesized to be positively related to rating accuracy. Results supported these hypotheses. Implications for frame-of-reference training research and practice are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Ratings of leadership potential at the U. S. Naval Academy and subsequent officer performance.

For 324 graduates of the class of 1951, fitness report ratings were correlated with a variety of undergraduate performance measures, including aptitude-for-service ratings, class standings in various courses, and Officer Classification Battery Test Scores. "It is concluded that the aptitude-for-service ratings reflect significant correlates of successful officer performance following graduation." The other measures were found to bear an insignificant or much lower relationship to post graduation performance measures. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The effect of training in verbal self-guidance on performance effectiveness in a MBA program.

The present field experiment examined the effects of training in verbal self-guidance (VSG) and a motivational intervention, goal setting, on the performance effectiveness of students in a MBA program. Performance effectiveness was assessed in terms of outcome (i.e., GPA) and behavioural (i.e., interpersonal skills) measures as well as a composite criterion (performance effectiveness). MBA students (n = 126) were randomly assigned to a 2 (VSG training/control) × 2 (goal setting/do your best) factorial design. The results revealed a main effect for VSG on performance effectiveness. There was an interaction effect such that participants who were trained in VSG and set goals had the highest level of performance effectiveness. A subsequent analysis revealed that the main effect for VSG on performance was spurious due to an ordinal interaction effect. Hence, VSG may be most effective when combined with goal setting. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Validity of peer nominations in predicting a distant performance criterion.

This paper reports the follow-up phase of a study of peer nominations begun in 1955 at the Naval OCS in Newport, Rhode Island. Over 700 trainees completed several peer nomination forms at various stages of training, 1 in particular on "success as a future Naval Officer" (FO). Subsequently, 639 trainees were identified who had gone on to duty as officers for about 3 yr. The average grade they secured on a key portion of the fitness report ratings assigned by their direct superior officers was used as a performance criterion; it had a split-half reliability of .90 In the prediction of this criterion, the FO peer nomination score from the 3rd wk. of training gave a validity of .40 which was as high as that for later FO scores and which was only slightly diminished after academic grades and popularity were partialed. The findings support the use of early peer nominations as a valid supplemental measure in predicting performance after training. (17 ref.) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Use of analytical information concerning task requirements to increase the effectiveness of skill training.

How can complex tasks in training be related to student ability in such a way that time is an improved application of theory to practice? 324 university students, divided into 3 groups, were engaged in learning how to stay on a target appearing on a radar picture tube. In this tracking they manipulated aircraft type controls. The experimental group of 60 students was given a usual type of training program, plus "standard instructions derived from the analysis of task requirements." This group showed superiority throughout the course of practice. "… changes in learning occurred… rather than just changes in performance." From Psyc Abstracts 36:05:5LE95P. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Frame-of-reference training effectiveness: Effects of goal orientation and self-efficacy on affective, cognitive, skill-based, and transfer outcomes.

Empirical evidence supporting frame-of-reference (FOR) training as an effective intervention for calibrating raters is convincing. Yet very little is known about who does better or worse in FOR training. We conducted a field study of how motivational factors influence affective, cognitive, and behavioral learning outcomes, as well as near transfer indexed by achieving professional certification. Relying on goal orientation theory, we hypothesized effects for 3 goal orientations: learning, prove performance, and avoid performance. Results were generally supportive across learning outcomes and transfer. Findings further supported a hypothesized interaction between learning self-efficacy and avoid performance goal orientation, such that higher levels of learning self-efficacy mitigated the negative effects of higher performance avoid tendencies. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Test anxiety and blame assignment in a failure situation.

It was found that high anxiety Ss tended to blame the self for failure to a significantly greater extent than did the low anxiety Ss. Also, high anxiety Ss proved more consistent in the direction of blame assignment than did the low anxiety Ss. A discussion of the order of presentation of test materials reveals a possible source of significant difference between groups. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Evaluating nicotine replacement therapy and stage-based therapies in a population-based effectiveness trial.

Pharmacological interventions for smoking cessation are typically evaluated using volunteer samples (efficacy trials) but should also be evaluated in population-based trials (effectiveness trials). Nicotine replacement therapy (NRT) alone and in combination with behavioral interventions was evaluated on a population of smokers from a New England Veterans Affairs Medical Center. Telephone interviews were completed with 3,239 smokers, and 2,054 agreed to participate (64%). Participants were randomly assigned to one of four conditions: stage-matched manuals (MAN); NRT plus manuals (NRT = MAN); expert system plus NRT and manuals (EXP = NRT = MAN); and automated counseling plus NRT, manuals, and expert system (TEL = EXP = NRT = MAN). Assessments were completed at baseline, 10, 20, and 30 months. The point prevalence cessation rates at final follow-up (30 months) were MAN, 20.3%; NRT = MAN, 19.3%; EXP = NRT = MAN, 17.6%; and TEL = EXP = NRT = MAN, 19.9%. Stage-matched manuals provided cessation rates comparable with previous studies. The addition of NRT, expert system interventions, and automated telephone counseling failed to produce a further increase in intervention effectiveness. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Prediction of an intermediate criterion of combat effectiveness with a biographical inventory.

If, according to biographical data, an individual has tended to engage in risky activities such as contact sports, playing poker, etc., will he also tend to be more effective in combat? Using a Risk Scale and a Peer Rating Scale, 57 seniors enrolled in Army ROTC courses were tested. The criterion was selection of combat and noncombat assignments. For combat effectiveness and leadership, the Risk Scale showed a significant difference in the expected direction. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Personality characteristics related to leadership behavior in two types of small group situational problems.

Using male United States Air Force Officer Candidate School Ss several hypotheses implied in the "great man" theory of small group leadership were focused on small group behavior and situational testing. "The results… personality traits associated with successful performance in two types of small group activity do not differ in relative importance." (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Treatment effectiveness of parent management training in Norway: A randomized controlled trial of children with conduct problems.

This study was a randomized control trial (RCT) of Parent Management Training--The Oregon Model (PMTO) in Norway. A sample representing all health regions of Norway and consisting of 112 children with conduct problems and their families participated in the study. Families were randomly assigned to either PMTO or a regular services comparison group. PMTO was delivered via existing children's services, and families were recruited using the agencies' regular referral procedures, making this the first effectiveness study of PMTO and the first RCT of PMTO conducted outside of the United States. Using a multiagent-multimethod approach, results showed that PMTO was effective in reducing parent-reported child externalizing problems, improving teacher-reported social competence, and enhancing parental discipline. Age level and gender modified the effects of PMTO treatment on other outcomes. In a path model, participation in PMTO was associated with improved parental discipline, and effective discipline predicted greater child compliance, fewer child-initiated negative chains, and lower levels of child externalizing problems. Findings are presented along with a discussion of the implications for practice and research and the challenges accompanying effectiveness trials. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Rating formats and rater training redux: A context-specific approach for enhancing the effectiveness of performance management.

In this paper, we critically examine previous research on rating formats and rater training in the context of performance appraisal. Historically, the goal of this body of research has been to search for ways of maximizing the psychometric quality of performance evaluation data. Our central thesis is that there are a number of avenues for broadening this research. Accordingly, we propose a conceptual model that hopefully serves as a conceptual framework for future work in these 2 traditional performance appraisal research streams. For example, both rating formats and rater training research may be useful for facilitating and improving the feedback and employee development process, as well as reducing the potential existence of rater biases. In addition, format and training research may focus upon ways of enhancing both rater and ratee reactions to the appraisal system. A key feature of our model is the integration of national culture as a moderator of the relations between specific formats, training programs, and various outcomes. We consider both the national culture of raters and ratees, and focus specifically on comparisons between Western and East Asian cultures. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

A narrative and meta-analytic review of helping skills training: Time to revive a dormant area of inquiry.

The authors review previous narrative and meta-analytic reviews on the effectiveness of overall helping skills training programs. The authors then review narrative reviews and conduct a new meta-analysis of specific methods used to teach helping skills within these programs. Our meta-analysis found that, in the aggregate, training methods substantially outperformed no training conditions, and that effect sizes did not vary as a function of trainee educational level (graduate vs. undergraduate students) or the type of criterion measure (interview-based vs. analogue-based empathy measures). Direct comparison of the training methods revealed that modeling outperformed instruction and feedback, and multimethod outperformed single-method training. The authors critique the literature and suggest that the studies in the helping skills literature generally fail to meet contemporary methodological standards, thereby limiting the conclusions that can be drawn. The authors appeal for better research on helping skills training, especially as it is currently practiced. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Risk for suicidal ideation in the U.S. Air Force: An ecological perspective.

Objective: Suicidal members of the U.S. military often fail to disclose their suicidal urges and behaviors. Military suicide prevention efforts may therefore be enhanced if they also target less stigmatized psychosocial factors that may decrease risk of suicidality. In keeping with Bronfenbrenner's (1977, 1994) model, this study simultaneously examined 4 ecological levels (i.e., individual, family, workplace, and community) of factors variously associated with increased or decreased risk for suicidal ideation. Method: Active-duty U.S. Air Force members (N = 52,780; 79.3% male; 79.2% non-Hispanic White; mean age = 31.78 years, SD = 7.38) completed the 2006 Community Assessment survey (a biennial, anonymous survey conducted at 82 U.S. Air Force bases worldwide), including the Centers for Disease Control and Prevention's (2008) 5-item measure of past-year suicidality along with scales assessing an array of potential predictors. Results: The 1-year rate of suicidal ideation, defined as (a) more than rarely thinking about suicide or (b) ever seriously considering suicide, was approximately 4%. In multivariate models, for men and women, individual- (depressive symptoms and alcohol problems), family- (relationship satisfaction and intimate partner victimization), workplace- (hours worked), and community-level (social support) variables were retained in the final model. However, some sex differences in retained predictors were noted (e.g., men: dissatisfaction with the U.S. Air Force way of life; women: workplace relationship satisfaction and financial stressors). Conclusions: Addressing depressive symptoms and alcohol use, facilitating healthy relationship functioning, and increasing job satisfaction and social support may aid military suicide prevention efforts. These findings illustrate the importance of attending to multiple levels of potential influence when designing integrated suicide prevention and intervention programs. (PsycINFO Database Record (c) 2011 APA, all rights reserved)     

Implicit policies about relations between personality traits and behavioral effectiveness in situational judgment items.

This paper develops the concept of implicit trait policy (ITP), which is a variant of the accentuation effect described by Tajfel (1957). ITPs are implicit beliefs about causal relations between personality traits and behavioral effectiveness. Studies reported here tested the hypotheses (a) that personality traits affect ITPs so that agreeable people, for instance, believe the relation between agreeableness and effectiveness is more strongly positive than disagreeable people do and (b) that ITPs can predict behavior that expresses associated personality traits. Two studies with undergraduate research participants supported the first hypothesis for traits of agreeableness and extraversion (the average correlation between traits and associated ITPs was .31 for agreeableness and .37 for extraversion) but not for conscientiousness. A 3rd study with student participants found that individual differences in ITPs for agreeableness predicted agreeable behavior (the average correlation was .33) in simulated work settings. These results suggest that ITPs may be useful for predicting work behavior that expresses personality traits even though ITPs may not be strongly correlated with the personality traits themselves. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Stimulus and response fidelity in team training.

Transfer performance of 2-man teams was observed in a simulated radar-controlled aerial intercept task following either high or low stimulus (S-) fidelity and either high or low response (R-) fidelity training treatments. Both high S- and high R-fidelity training treatments resulted in superior transfer task performance; however, the effects of high R-fidelity training were relatively brief. It was concluded that whereas both are desirable, it is less important to provide high R-fidelity training at least for tasks where the major output requires verbal communication skills. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Prevalences of intimate partner violence in a representative U.S. Air Force sample.

Objective: Intimate partner violence (IPV) is a serious health concern, but little is known about prevalence of IPV in the armed forces, as military members cope with the pressures of long-standing operations. Furthermore, previous prevalence studies have been plagued by definitional issues; most studies have focused on acts of aggression without consideration of impact (clinically significant [CS] IPV). This is the first large-scale study to examine prevalences of IPV, CS-IPV, and clinically significant emotional abuse (CS-EA) for men and women. Method: A United States Air Force-wide anonymous survey was administered across 82 bases in 2006 (N = 42,744) to assess IPV, CS-IPV, and CS-EA. Results: The adjusted prevalence of CS-IPV perpetration was 4.66% for men and 3.54% for women. Prevalences of IPV perpetration were 12.90% for men and 15.14% for women. CS-EA victimization was 6.00% for men and 8.50% for women. Sociodemographic differences in risk for violence were found for gender, race/ethnicity, pay grade, religious faith, marital status, and career type even after controlling for other demographic variables. Conclusions: Partner maltreatment is widespread in military (and civilian) samples. Men were more likely to perpetrate CS-IPV, whereas women were more likely to perpetrate IPV. Specific demographic risk factors were identified for different types of partner maltreatment (e.g., lower rank predicted higher risk for both perpetration and victimization across men and women). Other sociodemographic differences varied across severity (IPV vs. CS-IPV) and across gender. (PsycINFO Database Record (c) 2011 APA, all rights reserved)     

Consulting in the military context. Implications of the revised training principles.

The author considers the revised education and training principles in consulting psychology (R. L. Lowman et al; see record 2003-04049-003) from the specific perspective of military psychology. Consulting psychologists in military settings must adopt a cross-cultural framework for consultation, recognizing unique facets such as the distinctive military mission and concerns regarding multiple roles and confidentiality. The revised principles are well-conceived and likely to facilitate consultation in the military. The author recommends further consideration of issues unresolved by the Principles, including the clinical dilemma in organizational psychology (blending clinical and nonclinical roles with organizational clients), the need for more stringent minimal training and experience requirements, and clarification of the essential characteristics of consulting psychologists. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Ability grouping in army basic training.

"This study investigated the effectiveness of heterogeneous ability grouping as a method of increasing proficiency in Army Basic Combat Training. In each of two companies, low-ability trainees were trained under three conditions of ability grouping… . In spite of a system of competition that made privileges dependent on squad performance, a proficiency test given at the end of eight weeks of training failed to show a significant difference between the learning of low-ability men who had high aptitude men in their squads and those who did not. Achievement at all ability levels was unusually high, but low men who were trained in squads by themselves were just as proficient as low men who were trained in squads with higher ability men." (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Cognitive consistency and performance in the autokinetic situation.

An experiment was conducted testing certain implications of a balance or cognitive consistency model of behavior. It was hypothesized that a person (P) would respond to another person's (O's) judgments of light movement in the autokinetic situation as a function of the attraction relationship between them and the initial similarity in their judgments. By using an assistant playing the role of another S, it was possible to manipulate experimentally P's liking or disliking of O, and the initial discrepancy between P's and O's judgments of light movement. As compared with a condition of cognitive balance, under the conditions of cognitive imbalance—where either P likes O and initially is dissimilar to O or P dislikes O and initially is similar to O—we find a greater tendency for P to change his own judgments of the light movement, and to feel anxious or nervous while making these judgments. In addition, we find the direction of P's judgmental change is such as to increase his similarity to O when he likes O and to decrease this similarity when he dislikes O. These findings are taken as supporting a cognitive consistency model of interpersonal behavior and as having broad implications for the area of social influence and conformity. (18 ref.) (PsycINFO Database Record (c) 2010 APA, all rights reserved)