A note on the economic utility of generalized validity coefficients in personnel selection.

This study compared economic utility estimates that were based on noninteractive, interactive, independent multiplicative, and Taylor Series Approximation (TSA) 1 and 2 validity generalization results for clerical selection procedures at a large international manufacturing company. On the basis of estimates of the mean true validity and lower bound 90% credibility value, magnitude and percentage differences in resulting utility estimates across validity generalization procedures were relatively small for almost all comparisons. Regardless of the specific validity generalization parameter estimate used in estimating a utility value, the change in economic utility, going from the organization's current selection procedure (i.e., a verbal ability test) to an alternative procedure, was sizable in most cases. These results clearly demonstrate the practical similarity in utility terms of alternative validity generalization procedure results as well as the sizable economic value of minimum-level generalized validity coefficients. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Negro-white differences in validity of employment and training selection procedures: Summary of research evidence.

Examined 13 research studies dealing with differences between negroes and caucasians in the validity of employment and training selection procedures. It was found that 100 of the 160 validity coefficients computed were not significant for either group, indicating the need for validation prior to operational use. Significant differences in the coefficients between groups were found only in 7 instances. Statistical significance of single group validity occurred in 33 instances. The combination of ethnic groups yielded lower validity than for either group separately on only 3 out of 120 instances. It is concluded that there is very little evidence of differential validity. Single-group validity appears linked to the use of ratings (rather than more objective criterion measures), and small samples. (25 ref.) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Comprehensive meta-analysis of integrity test validities: Findings and implications for personnel selection and theories of job performance.

Conducted a comprehensive meta-analysis based on 665 validity coefficients across 576,460 data points to investigate whether integrity test validities are generalizable and to estimate differences in validity due to potential moderating influences. Results indicate that integrity test validities are substantial for predicting job performance and counterproductive behaviors on the job, such as theft, disciplinary problems, and absenteeism. The estimated mean operational predictive validity of integrity test for predicting supervisory ratings of job performance is .41. Results from predictive validity studies conducted on applicants and using external criterion measures (i.e., excluding self-reports) indicate that integrity tests predict the broad criterion of organizationally disruptive behaviors better than they predict employee theft alone. Despite the influence of moderators, integrity test validities are positive across situations and settings. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

A survey on return to work and fitness for work in Japan: the systems provided in corporate regulations and their application

We conducted a survey to evaluate the current situation and the application of systems for return to work (RTW) and fitness for work (FFW) in Japanese companies. Self-administered questionnaires were mailed to 351 occupational health (OH) physicians listed in the membership directory of the Japan Society for Occupational Health published in 1990. The item of information sought were the size and industrial type of the company, the organization of OH staff, the process of assessment of RTW, the provision for sick leave in corporate regulations, the FFW criteria, and job training systems for RTW. Of the 145 companies from which questionnaire replies were received from OH physicians (response rate: 41%), 123 (85%) were manufacturing industries. When classified according to the number of employees, 41 (28%) had 3000 or more, 66 (46%) had between 1000 and 2999, 26 (18%) had 999 or fewer and 12 (8%) were unknown. 144 companies (99%) had provisions on temporary retirement in their corporate regulations. As for who makes compensation during temporary retirement, 66 (48%) and 61 (44%) companies answered "both company and health insurance society" and "health insurance society alone," respectively. 136 companies (94%) carried out assessment of RTW, and involved OH professionals and other related staff in the process of assessment. In the majority of cases, the OH physicians were in charge of the decision-making process on RTW and were authorized to make the final decision. In the companies with 3000 or more employees, the cases of longer-term sick leave were assessed in a more complete process than those of shorter-term sick leave. 119 companies (88%) conducted on additional health examination to assess the employee's FFW. Sixty-four companies (47%) had standardized criteria on FFW. One hundred and eighteen companies (83%) had job training systems for employees' RTW, and they mainly introduced restricting one's job and/or reducing one's work-load. It was suggested that complete systems for RTW and FFW were more available among companies with 3000 or more employees than among companies with 2999 or fewer employees.     

A discriminant analysis of organizational performance variables.

Divided 98 groups participating in a gaming simulation (each group representing a company) into quartiles on the basis of their return on investment. A discriminant function analysis was utilized to test the following hypotheses: (a) that return on investment would divide the companies into the same quartiles as 12 other performance variables used in evaluating the groups and (b) that the most significant of the 12 performance variables identified by the discriminant function in Hypothesis 1 would remain constant over time. The 1st hypothesis was supported but the 2nd was not. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Organizational Divisions in BIM-Enabled Commercial Construction

Proponents claim that the adoption of building information modeling (BIM) will lead to greater efficiencies through increased collaboration. In this paper, we present research that examines the use of BIM technologies for mechanical, electrical, plumbing, and fire life safety systems (often referred to as MEP) coordination and how the introduction of BIM influences collaboration and communication. Using data from over 12 months of ethnographic observations of the MEP coordination process for two commercial construction projects and interviews with 65 industry leaders across the United States, we find that BIM-enabled projects are often tightly coupled technologically, but divided organizationally. This means that while BIM makes visible the connections among project members, it is not fostering closer collaboration across different companies. We outline the competing obligations to scope, project, and company as one cause for this division. Obligations to an individual scope of work or to a particular company can conflict with project goals. Individual leadership, especially that of the MEP coordinator in the teams we studied, often substitutes for stronger project cohesion and organization. Organizational forces and structures must be accounted for in order for BIM to be implemented successfully.     

Adding a dimension: Time as a factor in the generalizability of predictive relationships.

Trends in predictive validity coefficients across time or repeated performance assessments show significant trends. Commonly used predictors show decreasing validities for the prediction of temporally more remote performance assessments. Within-study corrections for statistical artifacts across performance assessments increased the negative slopes of the regressions of predictive validities on ordinal position of performance assessment. The average corrected validity decrement from initial to final performance assessment was –.60; the trimmed average decrement was –.45; the median decrement was –.29. The average within-study correlation between predictive validity and time of performance assessment was –.80. An analysis of stability coefficients of Time Period?×?Time Period performance assessments revealed similar findings. Several theoretical explanations are discussed as reasons for predictive validity decrements. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Relationships between subjective ratings and objective measures of performance in speechreading sentences

Ninety-six participants with normal hearing and 63 with severe-to-profound hearing impairment viewed 100 CID Sentences (Davis & Silverman, 1970) and 100 B-E Sentences (Bernstein & Eberhardt, 1986b). Objective measures included words correct, phonemes correct, and visual-phonetic distance between the stimulus and response. Subjective ratings were made on a 7-point confidence scale. Magnitude of validity coefficients ranged from .34 to .76 across materials, measures, and groups. Participants with hearing impairment had higher levels of objective performance, higher subjective ratings, and higher validity coefficients, although there were large individual differences. Regression analyses revealed that subjective ratings are predictable from stimulus length, response length, and objective performance. The ability of speechreaders to make valid performance evaluations was interpreted in terms of contemporary word recognition models.     

A validation study of the Worthington Personal History Blank.

The validity of the Worthington PH blank was compared with the validity of standardized tests (Wonderlic Personnel Test, The Personnel Laboratory's Employment Test, Thurstone Temperament Schedule, and the "Dominance" and "Self-Sufficiency" scales of the Bernreuter PI) for a sample of 47 publishing company employees. The criterion was associates' ratings on traits common to those measured by the PH and the other tests. Neither the PH nor the test results were significantly related to the criterion ratings. However, the difference (in favor of the tests) between the contingency coefficients comparing the tests with the ratings and the coefficients comparing the PH with the ratings was significant at the .06 level of confidence. It was concluded that the objective tests were at least as good as the PH, and that use of the latter is not warranted in terms of cost. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Employment test validation for minority and nonminority clerks with work sample criteria.

10 tests of intellectual ability and perceptual speed and accuracy were validated against specially developed clerical work samples of telephone company employees, using a newly hired sample of 143 Blacks, 74 Spanish-surnamed, and 195 Whites. Most validity coefficients were statistically significant, and in only 2 out of 36 comparisons were ethnic sample validity coefficients significantly different. A combination of 4 tests was significantly predictive of clerical proficiency for each ethnic sample and for the total combined sample. Comparisons of ethnic sample regression equations indicated that the slopes were essentially the same but that the intercepts differed significantly. The total sample regression equation did not underpredict prospective proficiency levels for minority clerks. A composite predictor found to be appropriate for minority and nonminority applicants for clerical jobs is recommended for employment office use. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Biographical data in employment selection: Can validities be made generalizable?

The hypothesis was examined that organizational specificity of biodata validity results from the methods typically used to select and key items. In this study, items were initially screened for job relevance, keying was based on large samples from multiple organizations, and items were retained only if they showed validity across organizations. Cross-validation was performed on approximately 11,000 first-line supervisors in 79 organizations. The resulting validities were meta-analyzed across organizations, age levels, sex, and levels of education, supervisory experience, and company tenure. In all cases, validities were generalizable. Validities were also stable across time and did not appear to stem from measurement of knowledge, skills, or abilities acquired through job experience. Finally, these results provide additional evidence against the hypothesis of situational specificity of validities, the first large-sample evidence in a noncognitive domain. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Biased test items and differential validity.

Tested some effects of one type of biased test items through the use of a simple psychometric model. Particular emphasis was placed on the magnitude of the difference in majority and minority group validity coefficients, and it was hypothesized that ability distributions for minority and majority (White) groups are identical, but tests are biased. An example was tested to illustrate the consequences of a profoundly biased test, but only a small difference in validity coefficients was found. The probability of detecting this difference in a standard differential validity study or by the aggregation method of J. E. Hunter et al (see record 1979-22609-001) is near the alpha level of the significance test. Thus, it cannot be assumed that minority and majority group validity coefficients will differ when a test is biased. The effects of subtest reliability on bias are discussed. Even though the test can predict criterion performance as well for minorities as for majorities, it may be inappropriate to compare the 2 types of scores. (13 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Gastro-oesophageal reflux disease

Using emerging international guidelines, stringent procedures were used to develop and evaluate Canadian-French, German and UK translations/adaptions of the 50 item, parent-completed Child Health Questionnaire (CHQ-PF50). Multitrait analysis was used to evaluate the convergent and discriminant validity of the hypothesized item sets across countries relative to the results obtained for a representative sample of children in the US. Cronbach's alpha coefficient was used to estimate the internal consistency reliability for each of the health scales. Floor and ceiling effects were also examined. Seventy-nine percent of all the item-scale correlations achieved acceptable internal consistency (0.40 or higher). The tests of the item convergent and discriminant validity were successful at least 87% of the time across all scales and countries. Equal item variance was observed 90% of the time across all countries. The reliability coefficients ranged from a low of 0.43 (parental time impact, Canadian English) to a high of 0.97 (physical functioning index, Canadian French) across all scales (median 0.80). Negligible floor effects were observed across countries. Noteworthy ceiling effects were observed, as expected, for the hypothesized physical scales (mean effect 73%). Conversely, fewer ceiling effects were observed for the psychosocial scales (range 3-17% behaviour-parental emotional impact). The item-scaling results obtained in these pilot studies support the psychometric properties of the American-English CHQ-PF50 and its respective translations.     

Image of industrial psychology among personnel administrators.

Results of a survey of 319 personnel administrators showed 11% of all companies and 20% of the largest ones employ a psychologist full time, and 25% employ 1 or more as a consultant. 50% felt it would be desirable to have a psychologist in the company and 75% felt it would increase productivity and satisfacton. Ratings of perceived past contributions, future usefulness, and need for further research in 12 areas of specialization are presented. Comparisons are made with previous surveys over a 20-yr period. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Single-group validity: Fact or fallacy?

13 studies, including 1,291 white and 1,308 black Ss, were examined with regard to single-group validity. Each study employed approximately equal sample sizes for both the black and the white samples. Results indicate that the significance of validity coefficients was independent of racial group membership for the entire sample of studies (p > .30). Further, the significance of the validity coefficients remained independent of racial group membership for studies employing only subjective criteria (p > .25) and for studies employing only objective criteria (p > .50). Findings provide clear and direct confirmation of the results of F. L. Schmidt et al (see record 1974-10042-001) demonstrating the nonexistence of true single-group validity. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Presentation mode, task importance, and cue order in experimental research on expert judges.

Used 140 23–56 yr old government auditors and 140 23–52 yr old auditors from private companies in 2 experiments investigating presentation mode, task importance, and cue order. In Exp I, 120 of these expert judges (60 from each group) made significantly different probability judgments depending on whether experimental task materials were presented in writing (visual mode), orally (auditory mode), or both (visual/auditory mode). Exp II, using 80 Ss from each group, was designed to partially explain why judgments differed across presentation modes. Results show that materiality (an auditor's term for task importance) interacted with presentation mode but that qualitative/quantitative cue order, a surrogate for time span of recall, did not. Findings provide evidence that research on expert judges should consider the potential costs of sacrificing external validity when the real world equivalent of an experimental task is likely to communicate information verbally or to vary presentation mode. (22 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Simultaneous study of individual differences and relationship effects in social behavior in groups.

The behavior of 35 graduate students, divided into 4 experiential training groups as part of a course in group psychotherapy, was observed over 12 weekly meetings to assess whether relationship and individual effects in dyads in groups are associated with specific types of behavior. The authors focused on 2 relevant verbal behaviors for small group settings: self-disclosure and questions asked. Self-disclosure was represented by sharing of personal information with another and by feeling statements; questioning was represented by asking for information or for a specific behavior from another. Self-disclosure comprised 40% and questioning 20% of all behavior coded. Round-robin ANOVAs of total verbal behavior, self-disclosing, and questioning during the 1st 6 wks and the last 6 wks revealed several indications of relationship effects and actor and partner individual difference effects. Results demonstrate that self-disclosing behavior in small groups is a function of the relationships among members in the group, in addition to individual differences in self-disclosing behavior. Findings support the incremental validity of relationship effects as in the study by L. La Voie (unpublished dissertation) and extend the external validity of relationship effects to important social behavior in small group settings. (21 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Comparison of Cultures in Construction and Manufacturing Industries

This paper investigates and compares the culture found in a construction company with that found in two manufacturing companies because large differences in culture might reduce the potential for the construction sector to adopt management tools developed in the manufacturing sector. For the comparisons to be valid, the culture within the physical production process was observed. The results were obtained by use of questionnaires and intensive interviews with individual members of staff using a U.K. government sponsored questionnaire. A vertical section of each company was sampled such that equivalent organizational levels could be directly compared. The main findings are that the culture existing in construction has significant differences to that found within the manufacturing industries. It will not be possible to transfer management tools from one industrial sector to the other without substantial redesign. The culture within construction was found to be a “project culture” in comparison to manufacturing, which was found to be a “company culture.”     

Performance Evaluating Model for Construction Companies: Egyptian Case Study

The dynamic nature of today’s construction industry compels construction partners to seek strategies in order to improve performance. Current research introduces a performance evaluation model for construction companies in order to provide a proper tool for a company’s managers, owners, shareholders, and funding agencies to evaluate the performance of construction companies. The model developed helps a company’s management to make the right decisions. Financial, economical, and industrial data are collected from Egyptian construction companies for nine consecutive years (1992–2000). Five indices (models) are developed: company performance score, economy performance score, industry performance score, performance index, and performance grade. The models developed consider companies in four construction sectors: general building, heavy, special trade, and real estate. These models accommodate the effect of macroeconomic and industry related factors and company size on the performance evaluation. The final outcome of current research is a performance grade, which provides the performance of a construction company. The developed model is validated, which shows robust results.