Faking and the validity of conscientiousness: A Monte Carlo investigation.

The article reports the findings from a Monte Carlo investigation examining the impact of faking on the criterion-related validity of Conscientiousness for predicting supervisory ratings of job performance. Based on a review of faking literature, 6 parameters were manipulated in order to model 4,500 distinct faking conditions (5 [magnitude] × 5 [proportion] × 4 [variability] × 3 [faking-Conscientiousness relationship] × 3 [faking-performance relationship] × 5 [selection ratio]). Overall, the results indicated that validity change is significantly affected by all 6 faking parameters, with the relationship between faking and performance, the proportion of fakers in the sample, and the magnitude of faking having the strongest effect on validity change. Additionally, the association between several of the parameters and changes in criterion-related validity was conditional on the faking-performance relationship. The results are discussed in terms of their practical and theoretical implications for using personality testing for employee selection. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Meta-analysis for integrating study outcomes: A Monte Carlo study of its susceptibility to Type I and Type II errors.

AMonte Carlo study was conducted to determine Types I and II error rates of the Schmidt and Hunter (S&H) meta-analysis method and the U statistic for assessing homogeneity within a set of correlations. One thousand samples of correlations were generated randomly to fill each of 450 cells of an 18?×?5?×?5 (Underlying Population Correlations?×?Numbers of Correlations Compared?×?Sample Size Per Correlation) design. To assess Type I error rates, correlations were drawn from the same population. To assess power, correlations were drawn from two different populations. As compared with U, which was uniformly robust, the Type I error rate for the S&H method was unacceptably high in many cells, particularly when the criterion for determining homogeneity was set at a highly conservative level. Power for the S&H method increased with increasing size of population differences, sample size per correlation, and in some cases, number of correlations compared. The U statistic did more poorly in most conditions in protecting from Type II errors. (14 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Appropriate critical percentages for the Schmidt and Hunter meta-analysis procedure: Comparative evaluation of Type I error rate and power.

Recent investigations have shown that the Schmidt and Hunter 75% meta-analysis procedure (S&H–75%) does not have adequate control of the Type I error rate. This lack of control has caused two problems: First, the S&H–75% displays an erratic relationship between the likelihood to detect moderators and the number of studies included in the meta-analysis. Second, it has precluded meaningful power comparisons of the S&H–75% procedure with alternate procedures. In the present study we first determine appropriate critical percentages for the Schmidt and Hunter procedure that maintain a fixed Type I error rate. Then we compare this procedure, using the appropriate percentages, with an alternative statistic, U. When the correct percentages are used, the Schmidt and Hunter procedure shows equivalent power to the U statistic. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Estimation accuracy of Thomas's likelihood-based procedure of meta-analysis: A Monte Carlo simulation.

Tested the accuracy of the likelihood-based validity generalization procedure proposed by H. Thomas (1990) for estimating the number of population correlations and their respective sizes and proportions in a mixed population. The simulation results showed that the Thomas procedure was highly accurate when there was only one correlation in the population. It was also accurate in estimating sizes and proportions of the population correlations when the correct number of population correlations was identified. However, its accuracy in estimating the actual number of population correlations decreased when they were close together and when the number of population correlations increased. The power of the procedure in identifying the existence of population correlations of magnitude zero and its accuracy in estimating means and standard deviations of population correlations are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Self-monitoring and sex as moderator variables in the statistical explanation of self-reported marijuana and alcohol use.

Examined self-monitoring (SM) and sex as moderators of predictors of self-reported use of marihuana and alcohol. SM theory implies that in comparisons between Ss high and low in SM, dispositional predictors account for more of the variance in reported use among low self-monitors and environmental predictors account for more of the variance among high self-monitors. With the 2 classes of predictors established via principal-components analysis and with subgroups defined by scores on Snyder's Self-Monitoring Scale, data from 807 college students showed that SM did not affect the action of either dispositional or environmental predictors. Thus, the SM construct promises explanatory leverage that in these tests the scale is not able to deliver. When subgroups were defined by sex, analyses of reported alcohol use yielded small but significant moderator effects: Women's use was more predictable from environmental variables, whereas men's use was more predictable from dispositional variables. These effects are consistent with the conclusions of recent meta-analyses of sex differences in conformity to group pressures. (19 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

On the robustness, bias, and stability of statistics from meta-analysis of correlation coefficients: Some initial Monte Carlo findings.

Each of several Monte Carlo simulations generated 100 sets of observed study correlations based on normal, heteroscedastic, or slightly nonlinear bivariate distributions, with one population correlation coefficient and true variance of 0. A version of J. E. Hunter and F. L. Schmidt's (1990b) meta-analysis was applied to each study set. Within simulations, ρ? was accurate on average. σ?2ρ was biased; one would correctly conclude more than half the time that no moderator effects existed. However, cases of variation in ρ? and especially in σ?2ρ indicated that results from individual meta-analyses could deviate substantially from what was found on average. Findings for these no-moderator cases offer applied psychologists some guidelines and cautions when drawing conclusions about true population correlations and true moderator effects (e.g., situational specificity, validity generalization) from meta-analytic results. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Type I error in counseling research: A plea for multivariate analyses.

Maintains that a Type I error becomes inflated beyond conventional acceptable levels when a researcher performs individual univariate statistics (such as t tests or ANOVAs) on each of several dependent variables within a single research project. The present article examines the prevalence of inflated Type I error in counseling research and recommends wider use of multivariate statistics to correct the problem. (13 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Aggregation, moderator variables, and the validity of personality tests: A peer-rating study.

A study with 85 college fraternity members investigated the effects of aggregation and moderator variables on the validity of personality tests. Aggregation over items and raters yielded an average self–peer correlation of .44 for ratings on 4 personality dimensions. The combination of social communication skill and self-knowledge produced significant moderating effects. Ss high on the Acting subscale and high on a composite of Private Self-Consciousness and Personal Identity scales had stronger agreement between self- and peer ratings than did those low on these measures. Results for 4 specific moderator variables (the importance, variability, and observability of each rating dimension) were in the predicted direction, although weaker than expected. It is concluded that aggregation and the use of moderator variables are both important techniques for obtaining convincing validity coefficients in personality research. (75 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Multivariate analysis and the problem of Type I error.

Discusses the recommendation by M. R. Leary and E. M. Altmaier (see record 1981-02539-001) that MANOVA should be used with several dependent variables in order to control for overall Type I error rate. This paper calls attention to limitations and dangers in routine application of MANOVA, describes some alternative procedures, and laments the necessarily pervasive character of the problem of statistical error. (20 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

A Monte Carlo investigation of three estimates of treatment magnitude: Epsilon squared, eta squared, and omega squared.

The effectiveness of 3 estimators of treatment magnitude were compared numerically for samples from a normal and exponential distribution. The estimators were compared for J. Cohen's (1969) definitions of small, medium, and large population treatment effects. It was found that omega squared was a more accurate estimator, while eta squared had the smallest sampling variability. (17 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The role of the media in body image concerns among women: A meta-analysis of experimental and correlational studies.

Research suggests that exposure to mass media depicting the thin-ideal body may be linked to body image disturbance in women. This meta-analysis examined experimental and correlational studies testing the links between media exposure to women's body dissatisfaction, internalization of the thin ideal, and eating behaviors and beliefs with a sample of 77 studies that yielded 141 effect sizes. The mean effect sizes were small to moderate (ds = -.28, -.39, and -.30, respectively). Effects for some outcome variables were moderated by publication year and study design. The findings support the notion that exposure to media images depicting the thin-ideal body is related to body image concerns for women. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Sampling variance in the correlation coefficient under range restriction: A Monte Carlo study.

Validity generalization methods require accurate estimates of the sampling variance in the correlation coefficient when the range of variation in the data is restricted. This article presents the results of computer simulations examining the accuracy of the sampling variance estimator under sample range restrictions. Range restriction is assumed to occur by direct selection on the predictor. Sample sizes of 25, 60, and 100 are used, with the selection ratio ranging from .10 to 1.0 and the population correlation ranging from .10 to .90. The estimator is found to have a slight negative bias in unrestricted data. In restricted data, the bias is substantial in sample sizes of 60 or less. In all sample sizes, the negative bias increases as the selection ratio becomes smaller. Implications of the results for studies of validity generalization are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Sex and susceptibility as moderator variables in the relationship of creativity and hypnotic susceptibility.

Studied 36 male and 36 female undergraduates, rectangularly distributed across the entire range of the harvard group scale of hypnotic susceptibility. Ss were given a battery of tests including the holtzman inkblot test, free association test, consequences test, remote associate test, and revised art preference test. Relationships among hypnotic susceptibility, creativity, and spontaneous trance-like experiences were found for women but not for men. Moreover, only women showed a dramatic increase in these correlations as a function of susceptibility level. Thus, both sex and susceptibility seem to operate as moderator variables in the correlates of hypnotic susceptibility. Women were significantly superior on a susceptibility scale that included more items requiring profound distortions in perception. It is suggested that the sex differences may reflect basic differences in the organization of imagination. It is hypothesized that women's imagination is more stimulus incited, whereas men's imagination is more impulse incited. (33 ref.) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Effect of the number of scale points on reliability: A Monte Carlo approach.

Discusses the contradictions and confusion in the literature on determining the optimal number of scale points in a rating scale, and suggests a mathematical model that allows for the simulation of the rating situation. The model involves generating data with different item variance-covariance structures and with different numbers of scale points. Such data were generated and used to calculate 3 reliability measures. The effects of different numbers of scale points and different covariance structures upon these reliability measures are examined, and the results help explain a large number of empirical studies exploring the "optimal number of scale points" problem. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Meta-analysis of correlation coefficients: A Monte Carlo comparison of fixed- and random-effects methods.

The efficacy of the Hedges and colleagues, Rosenthal-Rubin, and Hunter-Schmidt methods for combining correlation coefficients was tested for cases in which population effect sizes were both fixed and variable. After a brief tutorial on these meta-analytic methods, the author presents 2 Monte Carlo simulations that compare these methods for cases in which the number of studies in the meta-analysis and the average sample size of studies were varied. In the fixed case the methods produced comparable estimates of the average effect size; however, the Hunter-Schmidt method failed to control the Type I error rate for the associated significance tests. In the variable case, for both the Hedges and colleagues and Hunter-Schmidt methods, Type I error rates were not controlled for meta-analyses including 15 or fewer studies and the probability of detecting small effects was less than .3. Some practical recommendations are made about the use of meta-analysis. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Magnitude of sex differences in spatial abilities: A meta-analysis and consideration of critical variables.

In recent years, the magnitude, consistency, and stability across time of cognitive sex differences have been questioned. The present study examined these issues in the context of spatial abilities. A meta-analysis of 286 effect sizes from a variety of spatial ability measures was conducted. Effect sizes were partitioned by the specific test used and by a number of variables related to the experimental procedure in order to achieve homogeneity. Results showed that sex differences are significant in several tests but that some intertest differences exist. Partial support was found for the notion that the magnitude of sex differences has decreased in recent years. Finally, it was found that the age of emergence of sex differences depends on the test used. Results are discussed with regard to their implications for the study of sex differences in spatial abilities. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Monte Carlo simulations of protein folding. II. Application to protein A, ROP, and crambin

The hierarchy of lattice Monte Carlo models described in the accompanying paper (Kolinski, A., Skolnick, J. Monte Carlo simulations of protein folding. I. Lattice model and interaction scheme. Proteins 18:338-352, 1994) is applied to the simulation of protein folding and the prediction of 3-dimensional structure. Using sequence information alone, three proteins have been successfully folded: the B domain of staphylococcal protein A, a 120 residue, monomeric version of ROP dimer, and crambin. Starting from a random expanded conformation, the model proteins fold along relatively well-defined folding pathways. These involve a collection of early intermediates, which are followed by the final (and rate-determining) transition from compact intermediates closely resembling the molten globule state to the native-like state. The predicted structures are rather unique, with native-like packing of the side chains. The accuracy of the predicted native conformations is better than those obtained in previous folding simulations. The best (but by no means atypical) folds of protein A have a coordinate rms of 2.25 A from the native C alpha trace, and the best coordinate rms from crambin is 3.18 A. For ROP monomer, the lowest coordinate rms from equivalent C alpha s of ROP dimer is 3.65 A. Thus, for two simple helical proteins and a small alpha/beta protein, the ability to predict protein structure from sequence has been demonstrated.