The equivalence of measures and the correction for attenuation.

In literature reviews and critiques, measures are often evaluated in regard to their conceptual equivalence. In making this evaluation, the empirical correlation between the 2 measures being compared should be corrected for attenuation at least approximately. Unless this correction is applied, alternative measures may be presumed to be importantly different when giving due weight to the unreliabilities present, it may be seen that the identical underlying dimension is being reflected. In psychology, where enough discrepancies already characterize our findings, this interpretive error should not deny us the occasional equivalences that come along. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Conceptual and procedural knowledge of mathematics: Does one lead to the other?

This study examined relations between children's conceptual understanding of mathematical equivalence and their procedures for solving equivalence problems (e.g., 3?+?4?+?5?=?3?+?9). Students in 4th and 5th grades completed assessments of their conceptual and procedural knowledge of equivalence, both before and after a brief lesson. The instruction focused either on the concept of equivalence or on a correct procedure for solving equivalence problems. Conceptual instruction led to increased conceptual understanding and to generation and transfer of a correct procedure. Procedural instruction led to increased conceptual understanding and to adoption, but only limited transfer, of the instructed procedure. These findings highlight the causal relations between conceptual and procedural knowledge and suggest that conceptual knowledge may have a greater influence on procedural knowledge than the reverse. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

An inferential confidence interval method of establishing statistical equivalence that corrects Tryon's (2001) reduction factor.

Evidence of group matching frequently takes the form of a nonsignificant test of statistical difference. Theoretical hypotheses of no difference are also tested in this way. These practices are flawed in that null hypothesis statistical testing provides evidence against the null hypothesis and failing to reject H? is not evidence supportive of it. Tests of statistical equivalence are needed. This article corrects the inferential confidence interval (ICI) reduction factor introduced by W. W. Tryon (2001) and uses it to extend his discussion of statistical equivalence. This method is shown to be algebraically equivalent with D. J. Schuirmann's (1987) use of 2 one-sided t tests, a highly regarded and accepted method of testing for statistical equivalence. The ICI method provides an intuitive graphic method for inferring statistical difference as well as equivalence. Trivial difference occurs when a test of difference and a test of equivalence are both passed. Statistical indeterminacy results when both tests are failed. Hybrid confidence intervals are introduced that impose ICI limits on standard confidence intervals. These intervals are recommended as replacements for error bars because they facilitate inferences. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Cross-language measurement equivalence of parenting measures for use with Mexican American populations.

Increasing diversity among families in the United States often necessitates the translation of common measures into various languages. However, even when great care is taken during translations, empirical evaluations of measurement equivalence are necessary. The current study demonstrates the analytic techniques researchers should use to evaluate the measurement equivalence of translated measures. To this end we investigated the cross-language measurement equivalence of several common parenting measures in a sample of 749 Mexican American families. The item invariance results indicated similarity of factor structures across language groups for each of the parenting measures for both mothers and children. Construct validity tests indicated similar slope relations between each of the 4 parenting measures and the outcomes across the 2 language groups for both mothers and children. Equivalence in intercepts, however, was only achieved for some outcomes. These findings indicate that the use of these measures in both within-group and between-group analyses based on correlation/covariance structure is defensible, but researchers are cautioned against interpretations of mean level differences across these language groups. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

What would be if what is wasn't? Rejoinder to Judiesch, Schmidt, and Hunter (1993).

In their critique of the N. S. Raju et al (1990) utility model, M. K. Judiesch et al (see record 1994-19598-001) claimed that Raju et al simply shifted the problem from one of estimating the standard deviation of the dollar-valued performance (Y) to that of estimating the coefficient of variation (?–iY–n/μ–iY–n). In showing the inaccuracy of this claim, the authors demonstrate that the Judiesch et al misunderstandings of the underlying assumptions of the Raju et al model have led to their misinterpretations. Furthermore, the authors refute the contentions of Judiesch et al that (1) the A parameter in the Raju et al model is equal to average employee value and (2) it is appropriate to correct the validity coefficient for criterion unreliability when conducting a utility analysis. A more detailed discussion of the issues relating to transforming the original performance scale is offered. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The relative susceptibility of two rating scales to disturbances resulting from shifts in stimulus context.

A simple and a complex rating scale were compared with respect to their resistance to distorting effects produced by limited and shifting stimulus contexts. These 9-point scales assessed "shifting in the value of common stimuli as a function of context" and "loss of refinement or correlational accuracy." Results favored the detailed rating scale with judgmental equivalence and accuracy. Utility of creating experimental stress tests for rating scale evaluation is implied. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Distractibility in the conceptual performance of schizophrenics.

This study explores the role of interfering factors in the conceptual performance of schizophrenics. 40 schizophrenic patients at the Downey Hospital and 25 normals were used as Ss. They were required to sort cards containing 4 figures with 3 four-figured standard cards. One figure on each standard card was arbitrarily designated as correct for sorting. More of the schizophrenics than the normals used the incorrect "distracter" as a basis for sorting, even when conceptual sorting was required. The results support the hypothesis that schizophrenics fail conceptual tests because response to distracters prevents the demonstration of his concept-forming ability. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Differential effects of neonatal handling on anxiety, corticosterone response to stress, and hippocampal glucocorticoid and serotonin (5-HT)2A receptors in Lewis rats

The health-related quality of life (HRQoL) literature presents a confused picture of what 'equivalence' in the cross-cultural use of HRQoL questionnaires means and how it can be assessed. Much of this confusion can be attributed to the 'absolutist' approach to the cross-cultural adaptation of HRQoL questionnaires. The purpose of this paper is to provide a model of equivalence from a universalist perspective and to link this to the translation and adaptation of HRQoL questionnaires. The model evolved from reviews of the HRQoL and other literatures, interviews and discussions with researchers working in HRQoL and related areas and practical experience in the adaptation and development of HRQoL instruments. The model incorporates six key types of equivalence. For each type of equivalence the paper provides a definition, proposes various strategies for examining whether and how types of equivalence can be achieved, illustrates the relationships between them and suggests the order in which they should be tested. The principal conclusions are: (1) that a universalist approach to the cross-cultural adaptation of HRQoL instruments requires that six types of equivalence be taken into account; (2) that these are sufficient to describe and explain the nature of the cross-cultural adaptation process; (3) that this approach requires careful qualitative research in target cultures, particularly in the assessment of conceptual equivalence; and (4) that this qualitative work will provide information which will be fundamental in deciding whether to adapt an existing instrument and which instrument to adapt. It should also result in a more sensitive adaptation of existing instruments and provide valuable information for interpreting the results obtained using HRQoL instruments in the target culture.     

Failure to find the picture superiority effect in implicit conceptual memory tests.

According to the transfer appropriate processing framework (H. L. Roediger, M. S. Weldon, & B. A. Challis, 1989), if pictures engage more conceptual processing than words, then they should produce more priming on implicit conceptual tests. Experiments 1 and 2 did not find any significant advantage of pictures on the implicit category production or word association tests. When these tests were given as explicit cued-recall tests in Experiment 3, pictures were recalled better than words, producing a dissociation and indicating that the materials were sensitive to differences in picture and word processing. In Experiments 4 and 5, the implicit tests showed level-of-processing effects indicating that they were sensitive to differences in conceptual processing. Therefore, it is hypothesized that (a) conceptual processing plays a minor role, if any, in superior picture recall and that visual distinctiveness is a more important factor; and (b) distinctiveness is more important in intentional than incidental retrieval. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Validity of the Graduate Record Examination: Estimated or tacitly known?

Comments on the article by R. J. Sternberg and W. M. Williams (see record 1997-04591-002) regarding the empirical validity of the Graduate Record Examination in predicting graduate student performance in psychology. It is argued that although Sternberg and Williams addressed an important topic, they purposely did not use what are now widely accepted procedures for estimating predictive validities. Their failure to use appropriate parameter estimation techniques produced inaccurate results and misleading conclusions. Key errors made included (1) the failure to take range restriction into account, (2) failure to take criterion unreliability into account, and (3) reanalysis. It is concluded that these methodological flaws should not be duplicated by others, as it would represent a step backward. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Beyond alpha: An empirical examination of the effects of different sources of measurement error on reliability estimates for measures of individual-differences constructs.

On the basis of an empirical study of measures of constructs from the cognitive domain, the personality domain, and the domain of affective traits, the authors of this study examine the implications of transient measurement error for the measurement of frequently studied individual-differences variables. The authors clarify relevant reliability concepts as they relate to transient error and present a procedure for estimating the coefficient of equivalence and stability (L. J. Cronbach, 1947), the only classical reliability coefficient that assesses a 3 major sources of measurement error (random response, transient, and specific factor errors). The authors conclude that transient error exists in all 3 trait domains and is especially large in the domain of affective traits. Their findings indicate that the nearly universal use of the coefficient of equivalence (Cronbach's alpha; L. J. Cronbach, 1951), which fails to assess transient error, leads to overestimates of reliability and undercorrections for biases due to measurement error. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Recognizing attenuation effects in the strategy of research.

In the preceding paper, Stephens and Crowne argue that the investigator should not recognize the influence of unreliability in determining the obtained correlation between nominally related measures. The present note reiterates our earlier suggestion that a balanced, sophisticated view of a research area should pay due attention, among other considerations, to the way the attenuation effect may be determining obtained correlations. Proper application of the correction for attenuation has the heuristic effect of informing the researcher whether a raw correlation can be ascribed to the poor but improvable quality of his measures, or whether definite conceptual differences between the measures being related are likly to be involved. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Assessment of the Rescorla-Wagner model.

The Rescorla-Wagner model has been the most influential theory of associative learning to emerge from the study of animal behavior over the last 25 years. Recently, equivalence to this model has become a benchmark in assessing connectionist models, with such equivalence often achieved by incorporating the Widrow-Hoff delta rule. This article presents the Rescorla-Wagner model's basic assumptions, reviews some of the model's predictive successes and failures, relates the failures to the model's assumptions, and discusses the model's heuristic value. It is concluded that the model has had a positive influence on the study of simple associative learning by stimulating research and contributing to new model development. However, this benefit should neither lead to the model being regarded as inherently "correct" nor imply that its predictions can be profitably used to assess other models. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Some perspectives on "the attenuation paradox in test theory."

Clarification is given of the "paradox" posed by Loevinger (see 29: 3299), i.e., "validity" (the product-moment r of test score with the common factor of test items) decreases upon the increase beyond a certain point of item intercorrelations in a test whose items are of equal difficulty. Four points are made: (1) the "region of paradox" is reduced using a curvilinear r rather than the usual product-moment "validity"; (2) the paradox's incidence in aptitude and achievement tests is not as great as is frequently believed; (3) over-all "validity" for a group is inferior to the test's discriminating power for a specified decision problem for a specific examinee; and (4) the conception that "… for optimum results the percentage of correct answers to each item should equal the percentage of examinees to be selected" is correct only if the latter percentage is 50. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The partial testing design: a less costly way to test equivalence for sensitivity and specificity

We propose a new, less costly, design to test the equivalence of digital versus analogue mammography in terms of sensitivity and specificity. Because breast cancer is a rare event among asymptomatic women, the sample size for testing equivalence of sensitivity is larger than that for testing equivalence of specificity. Hence calculations of sample size are based on sensitivity. With the proposed design it is possible to achieve the same power as a completely paired design by increasing the number of less costly analogue mammograms and not giving the more expensive digital mammograms to some randomly selected subjects who are negative on the analogue mammogram. The key idea is that subjects who are negative on the analogue mammogram are unlikely to have cancer and hence contribute less information for estimating sensitivity than subjects who are positive on the analogue mammogram. To ascertain disease state among subjects not biopsied, we propose another analogue mammogram at a later time determined by a natural history model. The design differs from a double sampling design because it compares two imperfect tests instead of combining information from a perfect and imperfect test.     

Superordinate categorization via learned stimulus equivalence: Quantity of reinforcement, hedonic value, and the nature of the mediator.

Three experiments examined superordinate categorization via stimulus equivalence training in pigeons. Experiment 1 established superordinate categories by association with a common number of food pellet reinforcers, plus it established generalization to novel photographic stimuli. Experiment 2 documented generalization of choice responding from stimuli signaling different numbers of food pellets to stimuli signaling different delays to food reinforcement. Experiment 3 indicated that different numbers of food pellets did not substitute as discriminative stimuli for the photographic stimuli with which the food pellets had been paired. The collective results suggest that the effective mediator of superordinate categories that are established via learned stimulus equivalence is not likely to be an accurate representation of the reinforcer, neither is it likely to be a distinctive response that is made to the discriminative stimulus. Motivational or emotional mediation is a more likely account. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Problems and pitfalls in using capital budgeting and financial accounting techniques in assessing the utility of personnel programs.

In recent years, it has been advocated that capital budgeting and financial accounting techniques be used in evaluating the utility of human resources programs such as selection, training, and performance appraisal (Cronshaw & Alexander, 1985; Boudreau, 1983a, 1983b). We have demonstrated that many of these methods are often conceptually and logically inappropriate. We also showed that even in cases in which these techniques are logically applicable, their use may have unintended negative consequences. Finally, we discussed the question of the appropriate conceptual definition of utility. We conclude that different conceptual definitions of utility are useful under different circumstances; there is no single "correct" definition of utility. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Measurement invariance of neuropsychological tests in diverse older persons.

Objective: Comparability of meaning of neuropsychological test results across ethnic, linguistic, and cultural groups is important for clinicians challenged with assessing increasing numbers of older ethnic minorities. We examined the dimensional structure of a neuropsychological test battery in linguistically and demographically diverse older adults. Method: The Spanish and English Neuropsychological Assessment Scales (SENAS), developed to provide psychometrically sound measures of cognition for multiethnic and multilingual applications, was administered to a community dwelling sample of 760 Whites, 443 African Americans, 451 English-speaking Hispanics, and 882 Spanish-speaking Hispanics. Cognitive function spanned a broad range from normal to mildly impaired to demented. Multiple group confirmatory factor analysis was used to examine equivalence of the dimensional structure for the SENAS across the groups defined by language and ethnicity. Results: Covariance among 16 SENAS tests was best explained by five cognitive dimensions corresponding to episodic memory, semantic memory/language, spatial ability, attention/working memory, and verbal fluency. Multiple Group confirmatory factor analysis supported a common dimensional structure in the diverse groups. Measures of episodic memory showed the most compelling evidence of measurement equivalence across groups. Measurement equivalence was observed for most but not all measures of semantic memory/language and spatial ability. Measures of attention/working memory defined a common dimension in the different groups, but results suggest that scores are not strictly comparable across groups. Conclusions: These results support the applicability of the SENAS for use with multiethnic and bilingual older adults, and more broadly, provide evidence of similar dimensions of cognition in the groups represented in the study. (PsycINFO Database Record (c) 2011 APA, all rights reserved)     

Relationship between job knowledge and the reliability of conceptual similarity schemata.

Several investigators have argued that raters often rely on the conceptual similarity among performance dimension labels to guide the pattern of their performance ratings. Recent studies have used individual-level conceptual similarity (COS) judgments to investigate this systematic distortion hypothesis and related performance rating issues. In this article the results from 4 studies are reported in which 171 subjects completed COS judgments on 2 occasions. In 3 separate studies the reliability of COS schemata was found to be positively related to the rater's relevant job knowledge. In a 4th study it was found that changes in COS schemata over a 9-week interval may result from COS unreliability as much as from any meaningful reconceptualization of COS structure. Implications for performance rating research are reviewed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Extracting functional equivalence from reversing contingencies.

In 2 experiments assessing acquired equivalence, human participants were initially presented with 4 cues, 2 of which were paired with 1 outcome and 2 of which were paired with a 2nd outcome. These contingencies were then reversed across several training blocks such that, although each cue was paired equally often with each of the two outcomes across blocks, cues A and B always signaled the same outcome within blocks (as did cues C and D). In both experiments, performance on a subsequent transfer discrimination was enhanced when participants were required to generalize between stimuli that had been paired with the same outcome within each block of training. Additional tests did not yield evidence of a bias toward a specific set of cue–outcome contingencies in either experiment. Moreover, interviews conducted at the end of Experiment 2 revealed that performance on the transfer discrimination was enhanced only in participants who discovered the equivalence relationships during initial training. The results challenge simple associative, and attentional, accounts of acquired equivalence and favor the view that this effect is mediated by comparisons of the similarity of adjacent cue–outcome structures. (PsycINFO Database Record (c) 2010 APA, all rights reserved)