A note on the economic utility of generalized validity coefficients in personnel selection.

This study compared economic utility estimates that were based on noninteractive, interactive, independent multiplicative, and Taylor Series Approximation (TSA) 1 and 2 validity generalization results for clerical selection procedures at a large international manufacturing company. On the basis of estimates of the mean true validity and lower bound 90% credibility value, magnitude and percentage differences in resulting utility estimates across validity generalization procedures were relatively small for almost all comparisons. Regardless of the specific validity generalization parameter estimate used in estimating a utility value, the change in economic utility, going from the organization's current selection procedure (i.e., a verbal ability test) to an alternative procedure, was sizable in most cases. These results clearly demonstrate the practical similarity in utility terms of alternative validity generalization procedure results as well as the sizable economic value of minimum-level generalized validity coefficients. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Quantifying the effects of psychological interventions on employee job performance and work-force productivity.

Quantifying the economic impact of psychological programs in organizations requires determination of (a) the size and variability of the resulting increase in job performance and (b) the economic value of the increase in job performance. New methods of meta-analysis allow attainment of the 1st of these; and in relation to the 2nd, utility analysis methods provide the ability to translate job performance increases into estimates of the economic value of the program. In the area of personnel selection, many meta-analytic studies have resulted in precise and generalizable estimates of the validity of cognitive ability tests and other selection procedures. Utility analyses show that the job performance increases resulting from use of valid selection methods have substantial economic value. Valid selection produces major increases in work-force productivity. It is concluded that the combined effects of selection and nonselection interventions can be expected to produce substantial increases in workforce productivity. (29 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Choosing the best method for local validity estimation: Relative accuracy of meta-analysis versus a local study versus Bayes-analysis.

This study assessed the relative accuracy of 3 techniques--local validity studies, meta-analysis, and Bayesian analysis--for estimating test validity, incremental validity, and adverse impact in the local selection context. Bayes-analysis involves combining a local study with nonlocal (meta-analytic) validity data. Using tests of cognitive ability and personality (conscientiousness) as predictors, an empirically driven selection scenario illustrates conditions in which each of the 3 estimation techniques performs best. General recommendations are offered for how to estimate local parameters, based on true population variability and the number of studies in the meta-analytic prior. Benefits of empirical Bayesian analysis for personnel selection are demonstrated, and equations are derived to help guide the choice of a local validity technique (i.e., meta-analysis vs. local study vs. Bayes-analysis). (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

I/O psychology in the Canadian federal government: The Personnel Psychology Centre of the Public Service Commission.

Suggests that hiring in the Canadian federal government is based on merit and that tools such as selection and language standards and assessment techniques foster the assessment of merit. Legislation entrusts the Public Service Commission of Canada with the authority to develop and administer these selection tools, and this authority is, in turn, entrusted to the Personnel Psychology Centre (PPC). The scope of the PPC mandate and each divisional area of responsibility are described to illustrate the state of industrial/organizational psychology within the federal government. Implications for the PPC of the increasing concern with test bias issues are also discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Inventoried vocational interests of cartographers.

Studied the interests of 466 male cartographers to develop a cartographer scale for the Strong Vocational Interest Blank (SVIB). Results show that (a) cartographers liked science, mathematics, and religious activities; (b) they were masculine and diverse and tended to dislike activities involving other people; and (c) their interests were most similar to those of computer programers, public administrators, Air Force officers, and physical therapists though none of those SVIB keys adequately portrayed the interests of the cartographers. The cartographer key developed was cross-validated on 2 cartographer samples drawn from other populations (n = 89) and was found to discriminate adequately between cartographers and men in general. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The search for predictors with high validity and low adverse impact: Compatible or incompatible goals?

Many researchers and personnel selection specialists appear to believe that validity must often be sacrificed to reduce adverse impact. This belief may be bolstered by an interpretation of the Uniform Guidelines on Employee Selection Procedures (Equal Employment Opportunity Commission, 1978) that alternative selection methods should be sought in an effort to reduce adverse impact as long as the accompanying reduction in validity is not too large. The authors show that, contrary to popular belief, within the universe of fair tests (as defined by T. A. Cleary, 1968), the most valid selection method will necessarily produce the least adverse impact. Although a less valid selection method can have less adverse impact than the most valid fair method, such an alternative necessarily fails to meet Cleary's fairness criterion. Thus, for fair tests, maximizing validity also minimizes adverse impact. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Long-range impact of "fair selection" standards on minority employment.

Reviews alternate statistical standards of "fair selection," including the standard that seems to be implicit in the federal employee selection guidelines. A human resource planning model is developed to simulate the impact of each standard on minority employment. In the model, the racial composition of the flows of employees into and through the organization is determined by the "fair selection" standard chosen for simulation. The model is applied to staffing data from an existing organization. Data show that the fairness standard implicit in the federal selection guidelines would have a worse effect on Black employment than would the implicit fairness standard typically used by employers. (60 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The National Center for Health Care Technology: Assessment of psychotherapy for policymaking.

Discusses the importance of empirical evaluations of health and mental health procedures to providing the highest quality of care and to reimbursing the most appropriate and efficient techniques. The National Center for Health Care Technology, during its brief existence, provided the federal government with a mechanism for making such assessments. The now-abandoned plans for the assessment of psychotherapy are described. (8 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Corrections for restriction of range and attenuation in criterion-related validation studies.

The results of 1,253 Navy enlisted personnel on the Navy Basic Test Battery and the Navy Enlisted Advancement Exam were used as predictor and criterion, respectively. Groups were selected according to 5 selection ratios, .10 to .50. Results were then compared to the true validity. Except for the selection ratio of .10, all of the corrected validity coefficients slightly over-estimated the true validity. Triple and 2-step correction procedures yielded equivalent results. In all cases, the corrected coefficients better estimated the unrestricted true validity. (4 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Employment testing: Old theories and new research findings.

Claims that cognitive ability tests of the kind generally used in personnel selection are valid predictors of successful performance for jobs in all settings. This controversial stance is supported by analyses that recast findings of invalid tests as instances of Type I error. Ideally, if an employer has large enough samples, perfectly reliable tests, and an unrestricted range of ability in the applicant pool, the most widely used types of standardized tests should be valid in all job situations, and the notion of job-specific validity would no longer hold. The authors argue against previous reservations about the suitability of cognitive ability tests for employee selection that were made on the basis of their supposed limited applicability, their bias, and their ultimate contribution to workforce productivity. (56 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Validity and utility of alternative predictors of job performance.

Meta-analysis of the cumulative research on various predictors of job performance showed that for entry-level jobs there was no predictor with validity equal to that of ability, which had a mean validity of .53. For selection on the basis of current job performance, the work sample test, with mean validity of .54, was slightly better. For federal entry-level jobs, substitution of an alternative predictor would cost from $3.12 (job tryout) to $15.89 billion/year (age). Hiring on ability had a utility of $15.61 billion/year but affected minority groups adversely. Hiring on ability by quotas would decrease utility by 5%. A 3rd strategy—using a low cutoff score—would decrease utility by 83%. Using other predictors in conjunction with ability tests might improve validity and reduce adverse impact, but there is as yet no database for studying this possibility. (89 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings.

This article summarizes the practical and theoretical implications of 85 years of research in personnel selection. On the basis of meta-analytic findings, this article presents the validity of 19 selection procedures for predicting job performance and training performance and the validity of paired combinations of general mental ability (GMA) and the 18 other selection procedures. Overall, the 3 combinations with the highest multivariate validity and utility for job performance were GMA plus a work sample test (mean validity of .63), GMA plus an integrity test (mean validity of .65), and GMA plus a structured interview (mean validity of .63). A further advantage of the latter 2 combinations is that they can be used for both entry level selection and selection of experienced employees. The practical utility implications of these summary findings are substantial. The implications of these research findings for the development of theories of job performance are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Stress and the American worker.

Stress and stress-related disease are considered as they relate to American worker productivity and global competitiveness. Occupational trends, current and future federal government efforts to address stress in the workplace, as well as external factors that contribute to stress are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Employees may love flextime, but what does it do to the organization's productivity?

57–63 computer programmers (CPs) and 16–20 data entry operators (DEOs) in a government agency with flextime were compared with 59–69 CPs and 33–42 DEOs in a government agency without flextime. Direct measures of productivity were collected for each group over a 2-yr period and analyzed. CPs shared a limited physical resource (the computer), while DEOs did not. Findings support the hypothesis that flextime has a positive effect on productivity when there are limited physical resources being shared by a work group. (20 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Negotiating the indirect cost of research: A review of the process.

Notes that a considerable portion of the cost of federally sponsored research is used in the reimbursement of the indirect costs of the research. The percentage of indirect costs has been increasing over the last decade. This increase has led both investigators and federal agencies to seek procedures that would reduce the share of the research budget devoted to indirect costs. Under the assumption that any attempt to achieve this goal will be enhanced by an understanding of the process, the present authors review the procedures used as the indirect costs are negotiated between representatives of the federal government and a university. Attention is drawn to those stages of the process at which investigators can assess their university's indirect cost policy. It is concluded that the evaluation should focus on the total cost bill presented by the university rather than on the so-called indirect cost rate that is used to distribute the indirect cost recovery within the university. Factors that may account for the large variability across universities in the indirect cost rate are reviewed. (14 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Using personality tests in preemployment screening: Issues raised in Soroka v. Dayton Hudson Corporation.

Soroka v Dayton Hudson Corporation was the first case in which, on the basis of individuals' right to privacy, the use of psychological tests by a private sector employer was challenged. This article reviews the factual evidence and documentation from the case, discussing a number of technical and professional issues. Test users should consider the potential implications of using tests containing invasive items, and the relevance of personality constructs and evidence of validity for their use in preemployment screening. The legitimacy and legality of clinical personality tests for preemployment screening are questioned and balanced against the utility of such data for the selection or exclusion of job applicants. Finally, the job relevance of personality constructs, technical requirements for job analytic data, the impact of federal guidelines on employment testing, the face validity of items, and assumptions for generalizing across jobs are examined. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Economic impact on physicians and hospitals of proposed changes in Medicare reimbursement for coronary interventions

BACKGROUND: The federal government is implementing changes in reimbursement for angioplasty and coronary stenting. These include reductions in physician reimbursement and a redesignation of intracoronary stents to a different diagnosis-related group than other methods of intracoronary intervention. OBJECTIVE: The aim of this study was to examine the financial impact on physicians and hospitals of proposed federal reimbursement policies for percutaneous coronary revascularization procedures. METHODS: We modeled the financial effects of 3 different stenting strategies: strategy I is the most conservative, with stents reserved for addressing lab complications; strategy II stents are used for suboptimal results after attempts at conventional percutaneous transluminal coronary angioplasty (PTCA); strategy III is the most aggressive, with initial stenting of all accessible lesions. We used economic data on PTCA and stent costs from a 1996 dataset and made assumptions about PTCA and stent success rates and restenosis rates based on published data. RESULTS: Under current reimbursement policies, physician revenues and profits are approximately equal under all 3 stenting strategies. After the proposed changes, there is a slight financial incentive for physicians to pursue the more aggressive strategy III, but the major financial effect is a substantial overall decline in revenues with any of the 3 strategies. For hospitals, the present situation strongly favors the more conservative strategies, but after the proposed changes the more aggressive stenting strategies will be more profitable, thus realigning physician and hospital incentives. Health care delivery organizations that combine physician and hospital income streams achieve the greatest financial stability. CONCLUSIONS: Current reimbursement policies for angioplasty and stenting have created misaligned incentives between physicians and hospitals. Proposed changes do not present physicians with large economic incentives to pursue aggressive versus conservative stent strategies but substantially address the current disparity in hospital financial incentives.     

Concurrent and predictive validity designs: A critical reanalysis.

Historical and contemporary discussions of test validation cite 4 major criticisms of concurrent validity that are assumed to seriously distort a concurrent validity coefficient. These include "missing persons," restriction of range, motivational and demographic differences between present employees and job applicants, and confounding by job experience. As a result, predictive validity has been espoused as a scientifically superior strategy for the validation of personnel selection tests. However, it is argued here that frequently the conceptual distinction between predictive and concurrent validity has been exaggerated. More importantly, the differences that may exist have never been shown to render concurrent validity inaccurate as an estimate of predictive validity. Existing data suggest that these differences, if present, have a minimal impact on the magnitude of an obtained validity coefficient. (29 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Substance versus style: A new look at social desirability in motivating contexts.

Although there is an emerging consensus that social desirability does not meaningfully affect criterion-related validity, several researchers have reaffirmed the argument that social desirability degrades the construct validity of personality measures. Yet, most research demonstrating the adverse consequences of faking for construct validity uses a fake-good instruction set. The consequence of such a manipulation is to exacerbate the effects of response distortion beyond what would be expected under realistic circumstances (e.g., an applicant setting). The research reported in this article was designed to assess these issues by using real-world contexts not influenced by artificial instructions. Results suggest that response distortion has little impact on the construct validity of personality measures used in selection contexts. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Validity generalization and situational moderation in the life insurance industry.

Investigated the effect of company differences on the validity generalization of the Aptitude Index Battery, a noncognitive, weighted, biographical inventory used in the selection of life insurance agents. Data analyzed for 12,453 agents contracted by 12 companies indicate that although the validity could be generalized across the 12 companies, up to 38% of the variance in coefficients across companies was due to company differences. Further analyses of 2 a-priori company groups supported the hypothesis of situational moderation of validity coefficients. Utility estimates suggested that the net gain in productivity for one company group was about 50% above that expected in the other group. (12 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)