"Asymptotically distribution-free (ADF) interval estimation of coefficient alpha": Correction.

Reports an error in "Asymptotically distribution-free (ADF) interval estimation of coefficient alpha" by Alberto Maydeu-Olivares, Donna L. Coffman and Wolfgang M. Hartmann (Psychological Methods, 2007[Jun], Vol 12[2], 157-176). The sentence describing Equation 1 is incorrect. The corrected sentence is presented in the erratum. (The following abstract of the original article appeared in record 2007-07830-003.) The point estimate of sample coefficient alpha may provide a misleading impression of the reliability of the test score. Because sample coefficient alpha is consistently biased downward, it is more likely to yield a misleading impression of poor reliability. The magnitude of the bias is greatest precisely when the variability of sample alpha is greatest (small population reliability and small sample size). Taking into account the variability of sample alpha with an interval estimator may lead to retaining reliable tests that would be otherwise rejected. Here, the authors performed simulation studies to investigate the behavior of asymptotically distribution-free (ADF) versus normal-theory interval estimators of coefficient alpha under varied conditions. Normal-theory intervals were found to be less accurate when item skewness >1 or excess kurtosis >1. For sample sizes over 100 observations, ADF intervals are preferable, regardless of item skewness and kurtosis. A formula for computing ADF confidence intervals for coefficient alpha for tests of any size is provided, along with its implementation as an SAS macro. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

A score method of constructing asymmetric confidence intervals for the mean of a rating scale item.

This article presents a generalization of the Score method of constructing confidence intervals for the population proportion (E. B. Wilson, 1927) to the case of the population mean of a rating scale item. A simulation study was conducted to assess the properties of the Score confidence interval in relation to the traditional Wald (A. Wald, 1943) confidence interval under a variety of conditions, including sample size, number of response options, extremeness of the population mean, and kurtosis of the response distribution. The results of the simulation study indicated that the Score interval usually outperformed the Wald interval, suggesting that the Score interval is a viable method of constructing confidence intervals for the population mean of a rating scale item. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

What is coefficient alpha? An examination of theory and applications.

Psychological research involving scale construction has been hindered considerably by a widespread lack of understanding of coefficient alpha and reliability theory in general. A discussion of the assumptions and meaning of coefficient alpha is presented. This discussion is followed by a demonstration of the effects of test length and dimensionality on alpha by calculating the statistic for hypothetical tests with varying numbers of items, numbers of orthogonal dimensions, and average item intercorrelations. Recommendations for the proper use of coefficient alpha are offered. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

A taxonomy of effect size measures for the differential functioning of items and scales.

[Correction Notice: An erratum for this article was reported in Vol 95(5) of Journal of Applied Psychology (see record 2010-18410-004). There was an error in Formula 6 on page 731 for the pooled standard deviation of the ESSD index. The correct formula is given in the erratum. Related to this, in Table 8 on page 739, the ETSSD statistic should have been .094 for the cross cultural comparison and .001 for the Administration Format example.] Much progress has been made in the past 2 decades with respect to methods of identifying measurement invariance or a lack thereof. Until now, the focus of these efforts has been to establish criteria for statistical significance in items and scales that function differently across samples. The power associated with tests of differential functioning, as with all significance tests, is affected by sample size and other considerations. Additionally, statistical significance need not imply practical importance. There is a strong need as such for meaningful effect size indicators to describe the extent to which items and scales function differently. Recently developed effect size measures show promise for providing a metric to describe the amount of differential functioning present between groups. Expanding upon recent developments, this article presents a taxonomy of potential differential functioning effect sizes; several new indices of item and scale differential functioning effect size are proposed and illustrated with 2 data samples. Software created for computing these indices and graphing item- and scale-level differential functioning is described. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Varying coefficient meta-analytic methods for alpha reliability.

The conventional fixed-effects (FE) and random-effects (RE) confidence intervals that are used to assess the average alpha reliability across multiple studies have serious limitations. The FE method, which is based on a constant coefficient model, assumes equal reliability coefficients across studies and breaks down under minor violations of this assumption. The RE method, which is based on a random coefficient model, assumes that the selected studies are a random sample from a normally distributed superpopulation. The RE method performs poorly in typical meta-analytic applications where the studies have not been randomly sampled from a normally distributed superpopulation or have been randomly sampled from a nonnormal superpopulation. A new confidence interval for the average reliability coefficient of a specific measurement scale is based on a varying coefficient statistical model and is shown to perform well under realistic conditions of reliability heterogeneity and nonrandom sampling of studies. New methods are proposed for assessing reliability moderator effects. The proposed methods are especially useful in meta-analyses that involve a small number of carefully selected studies for the purpose of obtaining a more accurate reliability estimate or to detect factors that moderate the reliability of a scale. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

A coefficient alpha for test-retest data.

Transient errors are caused by variations in feelings, moods, and mental states over time. If these errors are present, coefficient alpha is an inflated estimate of reliability. A true-score model is presented that incorporates transient errors for test-retest data, and a reliability estimate is derived. This estimate, referred to as the test-retest alpha, is less than coefficient alpha if transient error is present and is less susceptible to effects due to item recall than a test-retest correlation. An assumption underlying the test-retest alpha is essential tau equivalency of items. A test-retest split-half coefficient is presented as an alternative to the test-retest alpha when this assumption is violated. The test-retest alpha is the mean of all possible test-retest split-half coefficients. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Brain attack. Acute therapeutic interventions. Free radical scavengers and antioxidants

This study used tests of content memory (item recognition of words and abstract designs), context memory (order recognition of verbal and nonverbal items), and working memory (recognition at a short retention interval) to examine patterns of performance in 27 schizophrenic patients, 52 chronic alcoholic patients, and 66 healthy control participants. When performance was age- and IQ-adjusted the schizophrenia group was significantly impaired in item and order recognition of verbal and nonverbal material; the alcoholic group was impaired only in order recognition for both material types. Item- and order-recognition deficits in the schizophrenia group were greatest at the shortest retention intervals, a pattern previously observed in patients with Parkinson's disease, suggesting a prominence of a working memory deficit in schizophrenia.     

Patterns of content, contextual, and working memory impairments in schizophrenia and nonamnesic alcoholism.

This study used tests of content memory (item recognition of words and abstract designs), context memory (order recognition of verbal and nonverbal items), and working memory (recognition at a short retention interval) to examine patterns of performance in 27 schizophrenic patients, 52 chronic alcoholic patients, and 66 healthy control participants. When performance was age- and IQ-adjusted the schizophrenia group was significantly impaired in item and order recognition of verbal and nonverbal material; the alcoholic group was impaired only in order recognition for both material types. Item- and order-recognition deficits in the schizophrenia group were greatest at the shortest retention intervals, a pattern previously observed in patients with Parkinson's disease, suggesting a prominence of a working memory deficit in schizophrenia. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

A note on using stratified alpha to estimate the composite reliability of a test composed of interrelated nonhomogeneous items.

The relationship between stratified alpha (alphas) and the reliability of a test composed of interrelated nonhomogeneous items is examined. It is mathematically demonstrated that when there is congeneric equivalence within the strata or subtests, the difference between the coefficients is a function of the variances of the loadings within the strata. When the items within each stratum are essentially tau equivalent, these variances are 0, and alphas and true reliability are equal, provided errors of measurement are uncorrelated. If errors of measurement are positively correlated and there is essential tau equivalence within strata, stratified alpha will overestimate reliability. These findings indicate that recent studies involving stratified alpha (A. Kamata, A. Turhan, & E. Darandari, 2003; H. G. Osburn, 2000) need to be interpreted with some degree of caution. Nevertheless, the hypothetical population data presented in this article suggest that under certain circumstances, stratified alpha can be considerably greater than alpha and closer to the true reliability. Because the former is easily computed, it is recommended that with stratified tests, practicing researchers should routinely calculate both alpha and stratified alpha coefficients. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Mind over milkshakes: Mindsets, not just nutrients, determine ghrelin response.

[Correction Notice: An erratum for this article was reported in Vol 30(4) of Health Psychology (see record 2011-13978-002). In the second paragraph on the first page, the Allison & Uhl 1964 citation is incorrect. The corrected sentence and full citation is in the erratum. All versions of this article have been corrected.] Objective: To test whether physiological satiation as measured by the gut peptide ghrelin may vary depending on the mindset in which one approaches consumption of food. Methods: On 2 separate occasions, participants (n = 46) consumed a 380-calorie milkshake under the pretense that it was either a 620-calorie “indulgent” shake or a 140-calorie “sensible” shake. Ghrelin was measured via intravenous blood samples at 3 time points: baseline (20 min), anticipatory (60 min), and postconsumption (90 min). During the first interval (between 20 and 60 min) participants were asked to view and rate the (misleading) label of the shake. During the second interval (between 60 and 90 min) participants were asked to drink and rate the milkshake. Results: The mindset of indulgence produced a dramatically steeper decline in ghrelin after consuming the shake, whereas the mindset of sensibility produced a relatively flat ghrelin response. Participants' satiety was consistent with what they believed they were consuming rather than the actual nutritional value of what they consumed. Conclusions: The effect of food consumption on ghrelin may be psychologically mediated, and mindset meaningfully affects physiological responses to food. (PsycINFO Database Record (c) 2011 APA, all rights reserved)     

Development of the Social Interest Index.

Developed the 32-item Social Interest Index to measure the level of social interest an individual has attained. Social interest was viewed as the willingness to contribute and cooperate within the areas of 4 life tasks: works, friendship, love, and self-significance. Item selection was made on the basis of agreement among 3 prominent Adlerians, item correlation with the total instrument score, and noncorrelation with the Marlowe-Crowne Social Desirability scale. The test-retest reliability coefficient, with 83 undergraduates and a 14-day interval, was .79. The Cronbach alpha coefficient of reliability was .81. Findings relating the level of social interest to the variables of sex, socioeconomic status, age, academic achievement, and the 18 scales of the California Psychological Inventory are presented and discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Combining dispositions and evaluations of vocation and job to account for counterproductive work behavior in adolescent job apprentices.

[Correction Notice: An erratum for this article was reported in Vol 12(3) of Journal of Occupational Health Psychology (see record 2007-10372-007). The following errors were committed: (1) In Table 1, item 7 in the variable column is mislabeled. (2) On page 171, in the paragraph before the Discussion section, the sentence beginning on line 14 reads incorrectly. (3) The caption for Figure 1 reads incorrectly. (4) On page 166, left column, 4th line from the bottom, the sentence reads incorrectly. The corrected information is presented in the erratum.] In the present research, we investigated the joint impact of selected antecedents of counterproductive work behavior (CWB). A sample of German apprentices reported on their CWB and completed measures of situational evaluations (vocational preference, level and constructiveness of job satisfaction) believed to trigger CWB and of dispositional motivators (measured by integrity test subscales) and controls (self-control and another subset of integrity scales) of CWB. All predictors investigated showed the expected bivariate relationships with CWB. Multivariate analyses revealed that the triggering effect of an unfavorable vocational choice on CWB was fully mediated by job satisfaction. When predictors were aggregated, a composite of dispositional control variables had the largest effect on CWB and moderated the effects of motivational dispositions and situational evaluations. These results extend the knowledge on antecedents of CWB by investigating previously overlooked variables and samples and partially replicate recent findings on the joint impact of dispositions and work-related evaluations on CWB. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Individual differences in intraperson variability in mood.

The experience sampling method and palm-top computers were used to obtain 75–200 randomly timed in situ assessments of 11 mood-related items from 54 Ss over 12–24 days. The variability in the distribution of an S's responses to each item was used as an estimate of intrasubject mood variability. Mood variability was stable across time and across situations. The intercorrelations among the individual item variabilities were also substantial; when the items were combined into a mood variability scale, the coefficient alpha was .84. The stability and internal consistency of mood variability could not be reasonably attributed to similarity in item valences, differences among the Ss in the situations they encountered, response biases, or response errors. It was concluded that mood variability is a stable personal characteristic, but additional analyses suggested that it may be independent from other kinds of intraperson variability. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Gastro-oesophageal reflux disease

Using emerging international guidelines, stringent procedures were used to develop and evaluate Canadian-French, German and UK translations/adaptions of the 50 item, parent-completed Child Health Questionnaire (CHQ-PF50). Multitrait analysis was used to evaluate the convergent and discriminant validity of the hypothesized item sets across countries relative to the results obtained for a representative sample of children in the US. Cronbach's alpha coefficient was used to estimate the internal consistency reliability for each of the health scales. Floor and ceiling effects were also examined. Seventy-nine percent of all the item-scale correlations achieved acceptable internal consistency (0.40 or higher). The tests of the item convergent and discriminant validity were successful at least 87% of the time across all scales and countries. Equal item variance was observed 90% of the time across all countries. The reliability coefficients ranged from a low of 0.43 (parental time impact, Canadian English) to a high of 0.97 (physical functioning index, Canadian French) across all scales (median 0.80). Negligible floor effects were observed across countries. Noteworthy ceiling effects were observed, as expected, for the hypothesized physical scales (mean effect 73%). Conversely, fewer ceiling effects were observed for the psychosocial scales (range 3-17% behaviour-parental emotional impact). The item-scaling results obtained in these pilot studies support the psychometric properties of the American-English CHQ-PF50 and its respective translations.     

"A taxonomy of measurement invariance effect size indices": Correction to Meade (2010).

Reports an error in "A taxonomy of effect size measures for the differential functioning of items and scales" by Adam W. Meade (Journal of Applied Psychology, 2010[Jul], Vol 95[4], 728-743). There was an error in Formula 6 on page 731 for the pooled standard deviation of the ESSD index. The correct formula is given in the erratum. Related to this, in Table 8 on page 739, the ETSSD statistic should have been .094 for the cross cultural comparison and .001 for the Administration Format example. (The following abstract of the original article appeared in record 2010-13313-009.) Much progress has been made in the past 2 decades with respect to methods of identifying measurement invariance or a lack thereof. Until now, the focus of these efforts has been to establish criteria for statistical significance in items and scales that function differently across samples. The power associated with tests of differential functioning, as with all significance tests, is affected by sample size and other considerations. Additionally, statistical significance need not imply practical importance. There is a strong need as such for meaningful effect size indicators to describe the extent to which items and scales function differently. Recently developed effect size measures show promise for providing a metric to describe the amount of differential functioning present between groups. Expanding upon recent developments, this article presents a taxonomy of potential differential functioning effect sizes; several new indices of item and scale differential functioning effect size are proposed and illustrated with 2 data samples. Software created for computing these indices and graphing item- and scale-level differential functioning is described. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

How automatic are social judgments?

[Correction Notice: An erratum for this article was reported in Vol 50(2) of Journal of Personality and Social Psychology (see record 2008-10961-001). There are errors in the labeling of the ordinates of the figures. The correct labeling is provided in the erratum.] Adapted the encoding-specificity paradigm developed by D. Thomson and E. Tulving (see record 1971-03487-001) to test 3 operational indicants of automatism (absence of intention, of interference from other mental activity, and of awareness). Recruited for a digit-recall study, 95 undergraduate students read sentences describing actions during the retention interval of either an easy or a difficult digit-recall task. Later, sentence recall was cued by (a) disposition cues, (b) strong semantic associates to the sentence actor, or (c) words representing the gist of the sentence, or (d) sentence recall was not cued. Awareness was measured immediately after the last sentence was read. Disposition-cued recall was higher than (b) or (d) and was unaffected by digit recall difficulty. Awareness of making dispositional inferences was only weakly correlated with disposition-cued recall. Results suggest that disposition inferences occurred at encoding, without intention, without interference by differential drain on processing capacity, and with little awareness. Thus, making dispositional inferences seems to be largely, but not entirely, automatic. (50 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Temporal predictability facilitates causal learning.

[Correction Notice: An erratum for this article was reported in Vol 140(3) of Journal of Experimental Psychology: General (see record 2011-16270-001). Figure 2 (p. 759) contained an error. The corrected figure appears in the correction.] Temporal predictability refers to the regularity or consistency of the time interval separating events. When encountering repeated instances of causes and effects, we also experience multiple cause–effect temporal intervals. Where this interval is constant it becomes possible to predict when the effect will follow from the cause. In contrast, interval variability entails unpredictability. Three experiments investigated the extent to which temporal predictability contributes to the inductive processes of human causal learning. The authors demonstrated that (a) causal relations with fixed temporal intervals are consistently judged as stronger than those with variable temporal intervals, (b) that causal judgments decline as a function of temporal uncertainty, and (c) that this effect remains undiminished with increased learning time. The results therefore clearly indicate that temporal predictability facilitates causal discovery. The authors considered the implications of their findings for various theoretical perspectives, including associative learning theory, the attribution shift hypothesis, and causal structure models. (PsycINFO Database Record (c) 2011 APA, all rights reserved)     

Evaluation of medical students in radiology. Written testing using uncued multiple-choice questions

RATIONALE AND OBJECTIVES: Evaluation of uncued multiple-choice questions (UMCQ) was compared with traditional multiple-choice questions (MCQ) for assessing medical student performance during radiology electives. Methods for analyzing and improving the quality of UMCQ examinations are described. METHODS: The authors compared the performance of radiology medical students on similarly constructed MCQ and UMCQ tests. For the UMCQ examination, the reliability (coefficient alpha), standard error of measurement, item difficulty index, and corrected item-to-total test coefficient (point biserial correlation) were analyzed. RESULTS: Students' level of performance was lower on UMCQs (mean percent correct score = 68.9 +/- 10.2 standard deviation [SD]) than on MCQs (mean percent correct score = 75.6 +/- 12.4 SD). Coefficient alpha for the UMCQ format was .7690 (standard error of measurement mean = 4.89%). Analysis of the item difficulty index and point biserial correlation for each test item provided information for improving the quality of the UMCQ examination. CONCLUSIONS: Because the UMCQ measures students' abilities to recall critical information without providing cues, this format can be used to overcome some of the limitations of conventional MCQs. With simple computerization, analysis of UMCQ testing instruments provides important feedback to both the examinees and the examiner.     

A statistical lower bound to population reliability.

Discusses the concept of a lower bound sample estimator of population reliability. Although it is known that coefficient α is a lower bound to reliability under very general conditions, it is noted that this property applies only in the population. Based on earlier work dealing with the sampling distribution of α coefficients and with interval estimation, a coefficient ρ({l}) is presented that is a lower bound to population reliability in both the population and the sample. Several examples are presented to illustrate that when the sample lower bound coefficient is adequately high, useful inferences can be made about the population reliability, even in relatively small samples. (16 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Are individuals' reputations related to their history of behavior?

[Correction Notice: An erratum for this article was reported in Vol 98(1) of Journal of Personality and Social Psychology (see record 2009-24670-012). Coefficient alpha reliabilities for the social connectedness variable were reported as .97 and .78 in Studies 2 and 3, respectively. Instead, they should have been reported as intraclass correlations (ICC) of .87 and .74, respectively. In the original reliability analyses, the authors included self-ratings of social connectedness and thus omitted participants from the analysis who did not provide a self-rating. Similarly, the authors included self-ratings when assessing the reliability of social connectedness in an unreported classroom sample (n=36) that was collected at the same time as the data reported in Study 2. In that unreported sample, they originally obtained a coefficient that was far below satisfactory levels, leading them to exclude that data set. However, after taking out the self-ratings in that unreported sample, the ICC was .82. The erratum summarizes the correlations reported in Study 2 for the unreported and included data sets combined.] Functional theories of reputation imply that individuals' reputations are tied to their history of behavior. However, indirect evidence suggests that the relation between reputation and behavior might be tenuous at best. In 3 studies, the authors tracked the development of reputations among individuals who engaged in multiple negotiation tasks across several weeks. The authors found that on average, individuals' reputations were only mildly related to their history of behavior. However, the link between reputation and behavior was stronger for some individuals than others--specifically, for individuals who were more well-known and received more social attention in the community. In contrast, for less well-known individuals, their behavior had little impact on their reputation. The findings have implications for psychologists' understanding of reputations, person perceptions in larger groups, and the costs and benefits of social visibility. (PsycINFO Database Record (c) 2010 APA, all rights reserved)