Power of univariate pairwise multiple comparison procedures.

Used Monte Carlo simulation to evaluate the power of various procedures under the constraint of experimentwise Type I error control. Both the configuration of population means and the method of assessing power influenced the relative powers of the procedures. The Shaffer-Welsch or one of several Q-test procedures had the greatest any-pair power or power to detect at least true difference. Either Peritz's F-test procedure or a model testing procedure had the greatest all-pairs power or power to detect all true differences. (29 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Testing for adverse impact when sample size is small.

Adverse impact evaluations often call for evidence that the disparity between groups in selection rates is statistically significant, and practitioners must choose which test statistic to apply in this situation. To identify the most effective testing procedure, the authors compared several alternate test statistics in terms of Type I error rates and power, focusing on situations with small samples. Significance testing was found to be of limited value because of low power for all tests. Among the alternate test statistics, the widely-used Z-test on the difference between two proportions performed reasonably well, except when sample size was extremely small. A test suggested by G. J. G. Upton (1982) provided slightly better control of Type I error under some conditions but generally produced results similar to the Z-test. Use of the Fisher Exact Test and Yates's continuity-corrected chi-square test are not recommended because of overly conservative Type I error rates and substantially lower power than the Z-test. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Testing multiple outcomes in repeated measures designs.

This study investigates procedures for controlling the familywise error rate (FWR) when testing hypotheses about multiple, correlated outcome variables in repeated measures (RM) designs. A content analysis of RM research articles published in 4 psychology journals revealed that 3 quarters of studies tested hypotheses about 2 or more outcome variables. Several procedures originally proposed for testing multiple outcomes in 2-group designs are extended to 2-group RM designs. The investigated procedures include 2 modified Bonferroni procedures that adjust the level of significance, α, for the effective number of outcomes and a permutation step-down (PSD) procedure. The FWR, any-variable power, and all-variable power are investigated in a Monte Carlo study. One modified Bonferroni procedure frequently resulted in inflated FWRs, whereas the PSD procedure controlled the FWR. The PSD procedure could be substantially more powerful than the conventional Bonferroni procedure, which does not account for dependencies among the outcome variables. However, the difference in power between the PSD procedure, which does account for these dependencies, and Hochberg's step-up procedure, which does not, were negligible. A numeric example illustrates implementation of these multiple-testing procedures. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Improved Bonferroni-type multiple testing procedures.

[Correction Notice: An erratum for this article was reported in Vol 104(2) of Psychological Bulletin (see record 2008-06007-001). An error was made in the author note on page 145. Correspondence should be addressed to Burt S. Holland, Department of Statistics, Temple University, Speakman Hall (006-00), Philadelphia, Pennsylvania 19122. Margaret DiPonzio Copenhaver is now at Merck Sharp & Dohme Research Laboratories, West Point, Pennsylvania.] The Bonferroni multiple comparisons procedure is customarily used when doing several simultaneous tests of significance in relatively nonstandard situations in which other methods do not apply. We review some new and improved competitors to the Bonferroni procedure, that although constraining generalized Type I error probability to be at most α, afford increased power in exchange for increased complexity in implementation. An improvement to the weighted form of the Bonferroni procedure is also presented. Several data sets are reanalyzed with the new methods. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Controversy and consensus regarding the use of cognitive ability testing in organizations.

Seven hundred three members of the Society for Industrial and Organizational Psychology indicated agreement or disagreement with 49 propositions regarding cognitive ability tests in organizations. There was consensus that cognitive ability tests are valid and fair, that they provide good but incomplete measures, that different abilities are necessary for different jobs, and that diversity is valuable. Items dealing with the unique status of cognitive ability were most likely to generate polarized opinions. A 2-factor model, classifying items as those reflecting societal concerns over the consequences of ability testing and those reflecting an emphasis on the unique status of "g," fit the data well, and these factors proved especially important for predicting responses to the more controversial items. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Are power analyses of published literature valuable?

Spence (2001), in a response to the author's power analysis of three health psychology-related journals (Maddock & Rossi, see record 2000-14051-009), noted that these types of analyses are not particularly informative because 90% of published studies report statistically significant findings (Sterling, Rosenbaum, & Weinkam, 1995). Spence also made the assumption that most of the effects in the published studies were likely to be medium or large, so the lack of power for small effects is not a problem. J. Maddock disagrees with this assumption, and presents several explanations in defense of his argument. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

A testing effect with multimedia learning.

A testing effect occurs when a learner performs better on a retention test after studying the material and taking a practice-retention test than after studying the material twice. In the present study, 282 participants watched a narrated animation about lightning formation and then watched the presentation again (restudy), took a practice-retention test (practice-retention), or took a practice-transfer test (practice-transfer). First, the testing effect was replicated with multimedia material, such that the practice-retention group outperformed the restudy group on a delayed retention test. Second, a testing effect was found for taking a practice-transfer test, such that the practice-transfer group outperformed the restudy group on a delayed transfer test. Third, the results supported a transfer-appropriate processing explanation of the testing effect, in which the practice-retention group outperformed the practice-transfer group on delayed retention but the opposite pattern was obtained on delayed transfer. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Hypothesis testing and theory evaluation at the boundaries: Surprising insights from Bayes's theorem.

Because the probability of obtaining an experimental finding given that the null hypothesis is true [p(H?/F)] is not the same as the probability that the null hypothesis is true given a finding [p(H?/F)], calculating the former probability does not justify conclusions about the latter one. As the standard null-hypothesis significance-testing procedure does just that, it is logically invalid (J. Cohen, 1994). Theoretically, Bayes's theorem yields [p(H?/F)], but in practice, researchers rarely know the correct values for 2 of the variables in the theorem. Nevertheless, by considering a wide range of possible values for the unknown variables, it is possible to calculate a range of theoretical values for [p(H?/F)] and to draw conclusions about both hypothesis testing and theory evaluation. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

便携式伺服阀测试仪的研制

为了现场测试电液伺服阀状态专门研制了便携式伺服阀测试仪,该测试仪能对射流管伺服阀、喷档伺服阀及比例阀进行高精度控制调节,能迅速判断阀控故障。电路设计遵循EMC电磁干扰协议,采用进口低漂移、高稳定器件,输出电流可达微安级控制,抗干扰不抖动,数字量电流显示准确直观。并运用先进电源管理方案,自身功耗低,适合长时间稳定工作。     

Evaluation of a new Army visual testing instrument.

A new device (Instrument B—an optical device to simulate the 20-foot distance of wall alleys) for testing photopic visual acuity was evaluated by comparison of test results with those obtained from the Standard Wall Chart Visual Acuity Examination (WC). Instrument B scores and WC scores correlated in the .90's, and test-retest reliabilities were also in the .90's. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Power and satisfaction in marriage: A review and critique.

Examines empirical studies of the relation between husband–wife power interactions and marital satisfaction. Both self-report and observational measures of marital power interactions have tended, in general, to suffer from similar difficulties, including overly simplistic conceptualizations of power processes and limitations in the measurement of marital power and marital satisfaction. Despite these weaknesses, the present group of studies has been fairly consistent in indicating that marriages in which the wife appears to be dominant are the most likely to be unhappy, whereas highest levels of satisfaction are more often found among egalitarian couples. Another group of studies focusing on the bases of marital power suggests that coercive control techniques may be related to marital dissatisfaction. A discussion of the kind of expanded framework that studies will need in order to make additional theoretical and substantive contributions to the understanding of power and satisfaction in marriage is presented. (90 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Genetic testing: Psychological aspects and implications.

As the number of genes associated with inherited disease continues to grow, researchers and practitioners in behavioral medicine will encounter complex psychological issues faced by individuals at risk for these diseases. A review of the literature concerning prenatal, carrier, and predictive genetic testing suggests that the severity of psychological risks posed by research-based genetic testing is not great. However, subgroups of individuals with particular psychological traits may be more vulnerable to adverse effects. Available data do not provide evidence that genetic testing promotes changes in health-related behaviors. Thus, although there may be less of a role for mental health professionals in the psychological counseling of genetic testing participants, there is a need for research and practice to facilitate health protective behaviors in response to genetic risk information. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Middle class bias in personality testing.

"The higher neurotic tendency scores obtained by the working class on the Bernreuter were found to be accounted for by a third of the inventory items. These items were found to be biased in one of the following ways: (a) they reflected middle-class values, mostly middle-class masculine values… ; (b) their connotations were sufficiently obvious to allow the middle class to benefit from their greater test motivation. We conclude that working-class scores on the inventory are spuriously high." (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The effects of testing accommodations on students' performances and reactions to testing.

This study examined the effects of testing accommodations on students' test performances and reactions to the use of testing accommodations. Participants (N = 170) were fourth- and eighth-grade students, with and without disabilities. All students were administered, with and without accommodations, equivalent forms of widely used math and reading tests. Students completed a questionnaire to summarize their reactions to the use of accommodations. The findings indicated testing accommodations overall had a positive impact on students' individual reading and math scores. Furthermore, testing accommodations had a differential positive effect on reading scores for students with disabilities compared to students without disabilities. The relationship between students' perceptions of testing accommodations and the effects of testing accommodations on their test performances was not significant, although most students had positive perceptions of testing accommodations. Students perceived the provision of accommodations as fair for students without disabilities and more fair for students with disabilities. These findings are interpreted within a validity framework and contribute to a greater understanding of testing accommodations by integrating information concerning effects on scores and consequential effects on test-takers. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The testing effect in recognition memory: A dual process account.

The testing effect, or the finding that taking an initial test improves subsequent memory performance, is a robust and reliable phenomenon--as long as the final test involves recall. Few studies have examined the effects of taking an initial recall test on final recognition performance, and results from these studies are equivocal. In 3 experiments, we attempt to demonstrate that initial testing can change the ways in which later recognition decisions are executed even when no difference can be detected in the recognition hit rates. Specifically, initial testing was shown to enhance later recollection but leave familiarity unchanged. This conclusion emerged from three dependent measures: source memory, exclusion performance, and remember/know judgments. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Responses to issues raised about validity, bias, and fairness in high-stakes testing.

We are pleased that our article (see record 2008-05553-001) prompted this series of four commentaries and that we have this opportunity to respond. We address each in turn. Duckworth (see record 2009-06923-012) and Kaufman and Agars (see record 2009-06923-013) discussed, respectively, two broad issues concerning the validity of selection systems, namely, the expansion of the predictor domain to include noncognitive predictors of performance and the expansion of the criterion domain to include additional criteria (e.g., creativity). We agree with these arguments, noting that they expand on points made in our original article. Wicherts and Millsap (see record 2009-06923-014) rightly noted the distinction between measurement bias and predictive bias and the fact that a finding of no predictive bias does not rule out the possibility that measurement bias still exists. They took issue with a statement we cited from Cullen, Hardison, and Sackett (2004) that if motivational mechanisms, such as stereotype threat, result in minority group members obtaining lower observed scores than true scores (i.e., a form of measurement bias), then the performance of minority group members should be under predicted. Our characterization of Cullen et al.’s (2004) statement was too cryptic; what was intended was a statement to the effect that if the regression lines for majority and minority groups are identical at the level of true predictor scores, then a biasing factor resulting in lower observed scores than true scores for minority group members would shift the minority group regression line to result in under prediction for that group. We do agree with Helms’s (see record 2009-06923-015) call for studying the reasons why racial- group differences are found and encourage this line of research; however, we view the study of racial-group differences and the study of determinants of those differences as complementary. We thank the authors for contributing these commentaries and for stimulating this discussion. Duckworth (2009) and Kaufman and Agars (2009) discussed important issues regarding expanding the predictor and criterion domains. Wicherts and Millsap (2009) correctly noted distinctions between predictive and measurement bias and used stereotype threat as a mechanism to discuss these issues. Helms (2009) raised several issues regarding the validity and fairness of standardized tests. In all cases, we welcomed the opportunity to discuss these topics and provide more detail on issues relating to high-stakes standardized testing. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Effects of directionality of significance tests on the bias of accessible effect sizes.

The proportion of studies that use one-tailed statistical significance tests (π) in a population of studies targeted by a meta-analysis can affect the bias of the sample effect sizes (sample ESs, or ds) that are accessible to the meta-analyst. H. C. Kraemer, C. Gardner, J. O. Brooks, and J. A. Yesavage (1998) found that, assuming π?=?1.0, for small studies (small Ns) the overestimation bias was large for small population ESs (δ?=?0.2) and reached a maximum for the smallest population ES (viz., δ?=?0). The present article shows (with a minor modification of H. C. Kraemer et al.'s model) that when π?=?0, the small-N bias of accessible sample ESs is relatively small for δ?≤?0.2, and a minimum (in fact, nonexistent) for δ?=?0. Implications are discussed for interpretations of meta-analyses of (a) therapy efficacy and therapy effectiveness studies, (b) comparative outcome studies, and (c) studies targeting small but important population ESs. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Repeated testing: A technique for assessing the roles of elaborative and organizational processing in the representation of social knowledge.

A new technique for evaluating the roles of elaborative and organizational processing in the representation of social information is described. Although these concepts have found increasing application in theories of social knowledge, most investigations of elaboration and organization have relied on measures that can lead to interpretive ambiguities. Two studies show how analysis of the item-gain and item-loss components of multitrial free recall provides a means for assessing the contributions of elaborative and organizational processing while avoiding many of the problems found with existing measures. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The practice of psychological testing in clinical services in the United States.

Questionnaire responses were obtained from 185 (61% of 304 agencies including VA stations, state mental hospitals, institutions for mental defectives, outpatient clinics, and counseling centers. The median number of different tests was used 26; the range was from 5 to 8. The most frequently used tests are the Rorschach, Draw-A-Person (Machover), Thematic Apperception Test, Visual Motor Gestalt Test (Bender), Stanford-Binet, WAIS, and MMPI in that order. Comparisons are made of the 20 most used psychological tests in each of 3 decades. From Psyc Abstracts 36:02:2HA79S. (PsycINFO Database Record (c) 2010 APA, all rights reserved)