Reply to Noller and Shum: Couple version of FACES III: Validity and reliability.

Replies to the article by Noller and Shum (1990; see record 1990-27249-001), which focused on the couple version of FACES III. I want to commend Noller and Shum for their well-done validity and reliability analysis which provided norms and replication of the couple version of FACES III. Since FACES III was primarily developed using family data, Noller and Shum provide a useful service by using the couple version of FACES and providing norms based on Australian couples. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Validity versus reliability.

After reviewing recent studies involving the selection of items for interest scales, in which scales with higher validity (and fewer items) generally had lower reliability, the author presents the original odd-even reliabilities and recently-collected test-retest reliabilities (over an average 18-year interval) for 15 scales of the Strong VIB. The test-retest reliabilities were all lower than the odd-even reliabilities, and the shrinkage was greatest for those scales with the lowest original reliabilities. It is concluded that, for prediction in the distant future, scale reliability is important. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The Social Phobia Inventory: Validity and reliability in an adolescent community sample.

Social anxiety disorder and social anxiety are highly prevalent psychological conditions in youth. Because of the known risks associated with the spectrum of social anxiety in youth, early detection with valid and reliable assessment measures is imperative. Self-report measures have become the most widely used method of assessment for child and adolescent social anxiety. Because research to date on self-report measures of social anxiety disorder in adolescents is limited, the primary objective of the present study was to contribute to the literature by examining the validity and reliability of a new self-report measure, the Social Phobia Inventory (SPIN), in a community sample of adolescents. The SPIN is a 17-item measure purported to assess the full spectrum of social anxiety disorder symptomatology. Previously, psychometric research on the SPIN in adult populations has demonstrated its validity and reliability. In the current psychometric examination, results revealed strong support for the temporal stability, internal consistency, and construct validity of the SPIN, suggesting it is an appropriate screening measure for the assessment of social anxiety disorder in adolescents. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Validity generalization and hypothetical reliability distributions: A test of the Schmidt-Hunter procedure.

A Monte Carlo study was conducted to determine how variations in true reliability distributions affect validity generalization estimates that are based on hypothetical reliability distributions proposed by Schmidt and Hunter. Both interactive and noninteractive validity generalization equations were examined. True reliability distributions and sample sizes were systematically varied. Depending on the sample size and nature of the true distributions, artifactual variance was either overestimated or underestimated by both equations. In some cases, extremely large errors were observed. Even when the data were generated with true reliability distributions identical to the hypothetical distributions, both equations systematically overestimated artifactual variance. The severity of estimation errors increased proportionally as sample size per validity coefficient increased. Implications of using assumed distributions and recommendations for validity generalization researchers are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Validity and reliability of the MTI photoscreener

OBJECTIVES: The prevalence and necessity for early detection of vision problems illustrate the need for improved methods of vision screening in preschool children. This study assessed the validity and reliability of a new device, the MTI photoscreener in a cross-sectional field study. METHODS: An appropriate sample size (> 140) was calculated and recruited for the study. All children (N = 161) in a migrant workers summer education program were screened with the MTI Photoscreener. Simultaneously and in a masked design, disease status was determined by the Modified Clinic Technique, a well established method for diagnosing the conditions which the MTI screener was designed to detect. RESULTS: Validity measures revealed a sensitivity of 54%, specificity of 87%, phi coefficient of 0.40, and positive predictive value of 52%. Repeatability was assessed by the kappa coefficient, by a test for effect modification by examiner, and by comparison of sensitivity and specificity across 12 masked examiners. The kappa coefficient was 0.38. A test for effect modification suggested that differences existed among the examiners. Variability of sensitivity was high, but variability of specificity was low. CONCLUSIONS: Methods for vision screening in preschool children are limited. The MTI Photoscreener is an easy and efficient method, but the validity and reliability is a concern. Comparison of our results with other studies suggests future potential for this instrument provided protocols are refined and further field studies reveal efficacy.     

Validity data on a modified version of Pearson's Novelty Experiencing Scale.

Describes the validation of a modified version of Pearson's (1970) Novelty Experiencing Scale, using 232 female and 198 male 12th graders. Measures of external sensation-seeking, internal cognitive experience-seeking, internal sensation-seeking, and external cognitive experience-seeking all proved reliable, and factorially homogeneous. Intercorrelations among the measures, and correlations against Pearson's Desire-for-Novelty Scale resembled previous findings by P. H. Pearson (1970 and 1971). 3 predicted sex differences occurred (plus one about which no prediction was made). (French summary) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The validity and reliability of the Violence Risk Scale-Sexual Offender version: Assessing sex offender risk and evaluating therapeutic change.

The Violence Risk Scale-Sexual Offender version (VRS-SO) is a rating scale designed to assess risk and predict sexual recidivism, to measure and link treatment changes to sexual recidivism, and to inform the delivery of sexual offender treatment. The VRS-SO comprises 7 static and 17 dynamic items empirically or conceptually linked to sexual recidivism. Dynamic items with higher ratings identify treatment targets linked to sexual offending. A modified stages of change model assesses the offender's treatment readiness and change. File-based VRS-SO ratings were completed on 321 sex offenders followed up an average of 10 years postrelease. VRS-SO scores predicted sexual and nonsexual violent recidivism postrelease and demonstrated acceptable interrater reliability and concurrent validity. A factor analysis of the dynamic items generated 3 factors labeled Sexual Deviance, Criminality, and Treatment Responsivity, all of which predicted sexual recidivism and were differentially associated with different sex offender types. The dynamic items together made incremental contributions to sexual recidivism prediction after static risk was controlled for. Positive changes in the dynamic items, measured at pre- and posttreatment, were significantly related to reductions in sexual recidivism after risk and follow-up time were controlled for, suggesting that dynamic items are indeed dynamic or changeable in nature. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Validity and reliability studies of a computer-based scoring system for inkblot responses.

A computer scoring system has been validated for scoring 17 Holtzman Inkblot Technique Variables: Location, Rejection, Form Definiteness, Color, Shading, Movement, Integration, Human, Animal, Anatomy, Sex, Abstract, Anxiety, Hostility, Barrier, Penetration, and Popular. The basic sample consisted of 145 college students to whom the HIT was group administered. An expert scorer's values were the criteria for validating computer scores. Validity of computer scoring was attested by comparability of means and standard deviations, by acceptable correlations between the 2 methods, and by identical factor structure among 8 rotated factors. The correlation of the computer with the average of 3 hand scorers equaled or approached the interscorer reliability of the scorers. Cross-validation studies demonstrated that equally satisfactory results were obtained for both Forms A and B. Finally, the computer was able to achieve scores from group records which were essentially equal to scores from records individually administered 1 wk. earlier and hand scored. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

"Validity generalization and hypothetical reliability distributions: A test of the Schmidt-Hunter Procedure": Correction.

Reports an error in the original article by P. W. Paese and F. S. Switzer (Journal of Applied Psychology, 1988[May], Vol 73[2], 267–274). Observed artifactual variance (ARV) estimates were traced to an error in the Monte Carlo computer program. Results of the replication indicated that ARV was overestimated by the noninteractive equation. The interactive equation provided unbiased estimates of ARV. (The following abstract of this article originally appeared in record 1988-25308-001.) A Monte Carlo study was conducted to determine how variations in true reliability distributions affect validity generalization estimates that are based on hypothetical reliability distributions proposed by Schmidt and Hunter. Both interactive and noninteractive validity generalization equations were examined. True reliability distributions and sample sizes were systematically varied. Depending on the sample size and nature of the true distributions, ARV was either overestimated or underestimated by both equations. In some cases, extremely large errors were observed. Even when the data were generated with true reliability distributions identical to the hypothetical distributions, both equations systematically overestimated ARV… (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Validity and reliability of scaffolded peer assessment of writing from instructor and student perspectives.

Although peer reviewing of writing is a way to create more writing opportunities in college and university settings, the validity and reliability of peer-generated grades are a major concern. This study investigated the validity and reliability of peer-generated writing grades of 708 students across 16 different courses from 4 universities in a particular scaffolded reviewing context: Students were given guidance on peer assessment, used carefully constructed rubrics, and were provided clear incentives to take the assessment task seriously. Distinguishing between instructor and student perspectives of reliability and validity, the analyses suggest that the aggregate ratings of at least 4 peers on a piece of writing are both highly reliable and as valid as instructor ratings while (paradoxically) producing very low estimates of reliability and validity from the student perspective. The results suggest that instructor concerns about peer evaluation reliability and validity should not be a barrier to implementing peer evaluations, at least with appropriate scaffolds. Future research needs to investigate how to address student concerns about reliability and validity and to identify scaffolds that may ensure high levels of reliability and validity. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The reliability of relationship satisfaction: A reliability generalization meta-analysis.

We conducted a reliability-generalization meta-analysis of 7 of the most frequently used measures of relationship satisfaction: the Locke–Wallace Marital Adjustment Test (LWMAT), the Kansas Marital Satisfaction Scale (KMS), the Quality of Marriage Index, the Relationship Assessment Scale, the Marital Opinion Questionnaire, Karney and Bradbury's (1997) semantic differential scale, and the Couples Satisfaction Index. Six hundred thirty-nine reliability coefficients from 398 articles and 636,806 individuals provided internal consistency reliability estimates for this meta-analysis. We present the average score reliabilities for each measure, characterize the variance in score reliabilities across studies, and consider sample and study characteristics that are predictive of score reliability. Overall, the KMS and the LWMAT appear to be the strongest and weakest measures, respectively, from a reliability perspective. We discuss the importance of considering reliability invariance when making cross-group comparisons and provide recommendations for researchers when electing a measure of relationship satisfaction. (PsycINFO Database Record (c) 2011 APA, all rights reserved)     

Validity and reliability of the Lesbian, Gay, Bisexual Working Alliance Self-Efficacy Scales.

In this paper, the authors report on the development and initial psychometric evaluation of the Lesbian, Gay, and Bisexual Working Alliance Self-Efficacy Scales (LGB-WASES) with data collected from two studies and 534 counseling trainees. Exploratory factor analysis results yielded a 32-item scale with a three-factor model (a) Emotional Bond, (b) Establishing Tasks, and (c) Setting Goals. LGB-WASES scores were internally consistent and remained stable over a 3-week period. Construct validity evidence suggests the LGB-WASES scores were (a) positively related to general perceptions of counseling self-efficacy and multicultural counseling competency, (b) negatively related to attitudes toward lesbians and gay men, and (d) unrelated to social desirability. Recommendations for future research are also discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The Concept of Validity.

This article advances a simple conception of test validity: A test is valid for measuring an attribute if (a) the attribute exists and (b) variations in the attribute causally produce variation in the measurement outcomes. This conception is shown to diverge from current validity theory in several respects. In particular, the emphasis in the proposed conception is on ontology, reference, and causality, whereas current validity theory focuses on epistemology, meaning, and correlation. It is argued that the proposed conception is not only simpler but also theoretically superior to the position taken in the existing literature. Further, it has clear theoretical and practical implications for validation research. Most important, validation research must not be directed at the relation between the measured attribute and other attributes but at the processes that convey the effect of the measured attribute on the test scores. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The Facial Expression Coding System (FACES): Development, validation, and utility.

This article presents information on the development and validation of the Facial Expression Coding System (FACES; A. M. Kring & D. Sloan, 1991). Grounded in a dimensional model of emotion, FACES provides information on the valence (positive, negative) of facial expressive behavior. In 5 studies, reliability and validity data from 13 diverse samples, including students, psychiatric patients, and community adults, are presented, and results indicate that raters can reliably agree on instances of positive and negative expressive behavior. Validity studies indicate that FACES ratings are related in predictable ways to another observational coding system, facial muscle activity, individual-difference measures of expressiveness and personality, skin conductance, heart rate, and reports of experienced emotion. FACES can be a useful tool for assessing expressive behavior in a variety of contexts. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Therapists' perspectives of couple problems and treatment issues in couple therapy.

Randomly selected samples of practicing couple therapists who were members of the American Psychological Association's Division 43 or the Association for Marriage and Family Therapy completed a survey of couple problem areas and therapeutic issues encountered in couple therapy. Therapists rated problem areas in terms of occurrence, treatment difficulty, and damaging impact. A composite of these 3 dimensions suggested that the most important problems were lack of loving feelings, power struggles, communication, extramarital affairs, and unrealistic expectations. Comparison of the findings with therapist ratings obtained by S. K Geiss and K. D. O'Leary (1981) suggests considerable stability in presenting problems in couple therapy over the past 15 years. Therapist-generated characteristics associated with negative outcome were also identified, the most common being partners' inability or unwillingness to change and lack of commitment. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Validity and Meaning: Comment.

Comments on an article by Robert L. Ebel (see record 1962-05654-001). This article discusses basic problems in psychological testing and measurement, and as such the author touches upon basic problems of scientific psychology as well. According to the commentator the problems the author raises are very well taken. For a while it seems that he will come up with what points to a solution but some reflection shows that although he approached the "truth," he failed to reach it. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The outcomes of couple and family therapy: Findings, conclusions, and recommendations.

Building on a special issue of the Journal of Marital and Family Therapy that was devoted exclusively to assessing the state of empirical knowledge about the efficacy and effectiveness of Couple and Family Therapy (CFT), this article presents a set of conclusions on the state of scientific knowledge about the outcomes of CFT. The primary conclusion is that there is substantial evidence to suggest that CFT is effective in general, and that it may even be superior to alternative treatments for certain major problems and disorders. Certain methodological and conceptual problems with this body of research are addressed and recommendations for improving CFT research are presented. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The Obsessive-Compulsive Inventory: Development and validation of a short version.

This article reports on the development of a revised version of the Obsessive-Compulsive Inventory (OCI; E. B. Foa, M. J. Kozak, P. Salkovskis, M. E. Coles, & N. Amir, 1998), a psychometrically sound, theoretically driven, self-report measure. The revised OCI (OCI-R) improves on the parent version in 3 ways: It eliminates the redundant frequency scale, simplifies the scoring of the subscales, and reduces overlap across subscales. The reliability and validity of the OCI-R were examined in 215 patients with obsessive-compulsive disorder (OCD), 243 patients with other anxiety disorders, and 677 nonanxious individuals. The OCI-R, which contains 18 items and 6 subscales, has retained excellent psychometric properties. The OCI-R and its subscales differentiated well between individuals with and without OCD. Receiver operating characteristic (ROC) analyses demonstrated the usefulness of the OCI-R as a diagnostic tool for screening patients with OCD, utilizing empirically derived cutscores. (PsycINFO Database Record (c) 2011 APA, all rights reserved)     

Integrative behavioral couple therapy: An acceptance-based, promising new treatment for couple discord.

Although traditional behavioral couple therapy (TBCT) has garnered the most empirical support of any marital treatment, concerns have been raised about both its durability and clinical significance. Integrative behavioral couple therapy (IBCT) was designed to address some of these limitations by combining strategies for fostering emotional acceptance with the change-oriented strategies of TBCT. Results of a preliminary clinical trial, in which 21 couples were randomly assigned to TBCT or IBCT, indicated that therapists could keep the 2 treatments distinct, that both husbands and wives receiving IBCT evidenced greater increases in marital satisfaction than couples receiving TBCT, and that IBCT resulted in a greater percentage of couples who either improved or recovered on the basis of clinical significance data. Although preliminary, these findings suggest that IBCT is a promising new treatment for couple discord. (PsycINFO Database Record (c) 2010 APA, all rights reserved)