Commentary on Meehl and Waller's (2002) Path Analysis and Verisimilitude.

P. E. Meehl and N. G. Waller's (see record 2002-18342-001) proposed method may not yield unique solutions for model parameters nor unique solutions for model lack of fit. The author argues from a naturalistic-cognitive philosophy of science that science seeks objective knowledge and that hypothesis testing is central to achieving that goal. It is also argued that P. E. Meehl and N. G. Waller's proposal blurs the distinction between hypothesis testing and explorations of the data seeking an optimal model to serve as a prospective inductive generalization. But it is noted that inductive generalizations are never unique and must be tested to eliminate those that reflect subjective aspects of the researcher's methods and points of view. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

A note on validity generalization procedures.

Reviews validity generalization (VG) procedures and finds them to be subject to the logical fallacy of "affirming the consequent." It is argued that alternative models may explain variation in validity coefficients as well as the cross-situational consistency model espoused by many users of the VG approach. Moreover, some of the assumptions that form the statistical foundation of VG work are open to question. It is suggested that Fisher z transformations, which remove most of these problems and usually produce more conservative estimates of the degree to which sampling error may account for variability in correlations, be used. It is also recommended that a more stringent criterion than the 75% decision rule for rejecting the situational specificity hypothesis be used. (64 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Validity generalization: Rejoinder to Schmidt, Hunter, and Raju (1988).

A rejoinder to the Schmidt, Hunter, and Raju (1988) critique of James, Demaree, and Mulaik (1986) is presented. It is proposed that the recommendations offered by James et al (1986) remain intact. These recommendations were that greater attention be given to alternative explanations of the results of validity generalization (VG) analyses, that use of the 75% rule be discontinued, and that VG analyses be predicated on Fisher z coefficients. In addition, the implication of low statistical power for interpreting results of a VG analysis is considered and the potential need for new mathematical models for VG is reviewed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Errata to Larzelere and Mulaik.

The article, "Single-Sample Tests for Many Correlations," by Robert E. Larzelere and Stanley A. Mulaik (Psychological Bulletin, 1977, Vol. 84, No. 3, pp. 557-569) contains the following errors: On page 558, the sentence that includes Equation 1 should read: "Assuming that the X and Y variables have a joint multivariate normal distribution, one then regards the multiple correlation as significantly different from zero if: (1)F=R2(N-p-1)/[(1-R2)p] is greater than Fα, the corresponding critical value of the F distribution with p and N-p-1 degrees of freedom at the 100(1-α) percentile level, with N the sample size." On page 559, the sentence that includes Equation 2 should read: "The population correlation p(W, Y) between W and Y is then regarded as significantly different from zero if: (2) F=rwy2(N-p-1)/[1-rwy2]p) is greater than the critical value of Fα of the F distribution with p and N-p-1 degrees of freedom at the 100(1-α) percentile level." On page 559, the sentence that reads, "If p(W, Y) is regarded as equal to zero, then any variable Xι with a nonzero weight wι in the linear combination W is also considered to have a zero correlation with Y," should be deleted. Thanks are due to Paul A. Games. (The following abstract originally appeared in record 1978-00149-001) When more than one correlation coefficient is tested for significance in a study, the probability of making at least one Type I error rises rapidly as the number of tests increases, and the probability of making a Type I error after a Type I error on a previous test is usually greater than the nominal significance level used in each test. To avoid excessive Type I errors with multiple tests of correlations, it is noted that researchers should use procedures that answer research questions with a single statistical test and/or should use special multiple-test procedures. A review of simultaneous-test and multiple-test procedures for correlations (e.g. Bartlett and Rajalakshman's test, multistage Bonferroni procedure, union-intersection tests, and the rank adjusted method) is presented, and several new procedures are described. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Inferring the communality of a variable in a universe of variables.

Given a sample of variables from a universe of variables measuring an unlimited number of different properties, and assuming that the most likely additional variables to be found in the universe are those differing from the sample variables by arbitrarily small amounts, it is argued that the most likely inference about the communality of a sample variable in the context of the universe is the reliability of the variable. All true variance in a sample of variables should thus be used in inferring common variance in the universe. (15 ref.) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Single-sample tests for many correlations.

When more than one correlation coefficient is tested for significance in a study, the probability of making at least one Type I error rises rapidly as the number of tests increases, and the probability of making a Type I error after a Type I error on a previous test is usually greater than the nominal significance level used in each test. To avoid excessive Type I errors with multiple tests of correlations, it is noted that researchers should use procedures that answer research questions with a single statistical test and/or should use special multiple-test procedures. A review of simultaneous-test and multiple-test procedures for correlations (e.g. Bartlett and Rajalakshman's test, multistage Bonferroni procedure, union-intersection tests, and the rank adjusted method) is presented, and several new procedures are described. (40 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Are personality factors raters' conceptual factors?

3 separate factor-analytic studies were conducted to compare personality factors with semantic factors using 76 trait-rating scales. In 1 study raters rated real persons using the scales, in another stereotype persons. In the third study raters rated the "meaning" of select trait words on the trait-rating scales. A comparison of factors found in the 3 studies using an index of factor similarity showed that 5 of 11 factors found for ratings of real persons were congruent with 5 of 10 factors found for the ratings of stereotype persons, while the same 5 in these 2 studies were congruent with 4 of 9 factors found for ratings of the meaning of select trait words. These results suggested that "personality factors" based upon trait ratings of persons can be interpreted as distinct concepts implied by trait words rather than internal structural features of persons. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Evaluation of goodness-of-fit indices for structural equation models.

Discusses how current goodness-of-fit indices fail to assess parsimony and hence disconfirmability of a model and are insensitive to misspecifications of causal relations (a) among latent variables when measurement model with many indicators is correct and (b) when causal relations corresponding to free parameters expected to be nonzero turn out to be zero or near zero. A discussion of philosophy of parsimony elucidates relations of parsimony to parameter estimation, disconfirmability, and goodness of fit. AGFI in {lisrel} is rejected. A method of adjusting goodness-of-fit indices by a parsimony ratio is described. Also discusses less biased estimates of goodness of fit and a relative normed-fit index for testing fit of structural model exclusive of the measurement model. (PsycINFO Database Record (c) 2011 APA, all rights reserved)     

Validity generalization in the context of situational models.

A primary objective of validity generalization (VG) analysis is to decompose the between-situation variance in validities into (1) variance attributable to between-situation differences in statistical artifacts and (2) variance attributable to between-situation differences in (unidentified) situational moderators. This process is based on the assumption that the effects of statistical artifacts on validities are independent of the effects of situational moderators on validities. The present article seeks to question the independence assumption by theoretically integrating situational variables into the VG estimation process. It is shown that the independence assumption may be untenable because at least 1 artifact (criterion reliability) is a function of, rather than independent of, the situational variables that moderate validities. An alternative approach to VG analysis is recommended. This approach rests heavily on proactive research designs in which potential situational moderators are included in the generalizability analysis. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Realism, pragmatism, and the implications of the new philosophy of science for psychology.

Commends P. T. Manicas and P. F. Secord (see record 1984-00037-001) for alerting psychologists to the changes that have occurred in the philosophy of science since 1970. However, the present author takes issue with Manicas and Secord for appearing to suggest that the revolution in the philosophy of science has been resolved in favor of ontological realism. It is suggested that what they should have said was that the ontological status of hypotheses has changed. (12 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)