Construct revalidation of a forced-choice rating form.

A forced-choice rating form was revalidated by using a type of construct validation based on the hypothesis that a manager's effectiveness is reflected in the performance level of his subordinates. 11 plant managers were ranked on overall effectiveness by 3 independent judges, and the relationship between these rankings and the average performance report scores of 142 first-line supervisors in the respective plants was determined by analysis of variance and correlation techniques. Results showed a significant overall relationship between plant-manager rankings and production-supervisor scores on the forced-choice form (p = .005) and significant correlations on 2 of the 6 subscales, with the highest relationship apparent in the Human Relations area (p = .025). The findings support the hypothesis of a relationship between management effectiveness and subordinate performance, and provide evidence to indicate continued validity of the rating instrument. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Positive and negative faking on a forced-choice authoritarian scale.

"The effects of faking both favorably and unfavorably upon a forced-choice scale of authoritarianism… were studied… . There were large distribution changes resulting from a set to 'look bad' compared to the conventional set, but only small changes emerged from faking favorably. Furthermore, the scores obtained by the subjects under the faking conditions, especially with the negative set, were opposite in direction from the intentions of the testees and the expectations of the experimeters… . The utility of this nonpolitical derivative of the California F Scale was recommended for leadership research in industrial, as well as military, settings." 19 references. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Matching indices for use in forced-choice scale construction.

"… an attempt to integrate the results of previous studies by analyzing a comprehensive list of appearance scales derived from… the literature and from insights of people making decisions in a forced-choice test situation." 100 items were taken randomly from a larger pool, but later randomly reduced to 40. To 12 appearance scales was added a discrimination index. A correlation matrix, generated from mean item ratings, was factor analyzed to yield 5 factors. "The finding of a general factor, supported by previous studies, brings an element of economy to forced-choice scale construction tending to support the pairing of items on only one appearance index." (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Development of a criterion of research productivity.

"A forced-choice rating scale designed to determine the extent of a person's productive research behavior was developed at a Midwestern physical science foundation. Of the two experimental scales developed the better form showed an interrator reliability coefficient of .62 and a validity of .60. When the ratings of two raters were averaged the validity of the scale increased to .74." (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Effects of variations in rating scale format on judgment.

The purpose of this study was to determine whether mean occupation evaluation ratings would differ as a function of 7 variations in rating-scale format. 60 basic airmen rated 15 occupations on 9 occupation-requirement factors for each format. A 3-way analysis of variance (occupations, factors, scale format) resulted in statistically significant terms for each of the main effects and for all 4 interaction terms. It was concluded that rating-scale format was a determiner of the judgment of raters in this sample and that selection of an optimal format should be based upon capability to predict a criterion. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Development of the forced-choice rating scale technique.

Forced-choice (FC) rating scales came about because of dissatisfaction with conventional scales. Reliabilities and validities of FC methods compare favorably with other methods. Studies on the FC method show this scale is more resistant than other scales to effects of bias. Formats using 4 favorable items, from which the rater chooses the items most characteristic of the person rated, prove superior to other formats. This superiority appears in validities, reliabilities, and preferences of raters using the form. Conditions under which statement indices are obtained should be as similar as possible to conditions under which the final scale will be administered. Combining FC scores with other scale scores yields better results than using either instrument alone. Ample evidence exists that more research can fruitfully be done in the area. (54 ref.) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The general applicability of a scale for rating research productivity.

"The RBD III, a forced-choice rating form to provide scores indicative of a person's productive research behavior in physical science research settings, was administered in a setting other than the one in which it was developed." 50 Ss were selected at random from 168 research engineers. Supervisory judgments of a person's creativity activity indicated its validity. The "RBD III can be used to provide criterion scores for research productivity in other physical science research settings." (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Dimensions of dreaming: A factored scale for rating dream reports.

127 dream reports of 24 Ss were assessed on 20 psychological characteristics. Nearly all characteristics were assessed by 2 raters, and some by S as well, creating a total of 43 variables. These were subjected to principal component analysis and analytic orthogonal rotation. About 63% of the total variance is accounted for by 8 dimensions: vivid fantasy, active control, pleasantness, verbal aggression, physical aggression, heterosexuality, perception (vs. conception), and reference to past experience. In a resulting condensed scale, each dimension is indexed by a single characteristic. These 8 characteristics are essentially uncorrelated. The last 2 are assessed by S alone; rater agreement in assessing the 1st 6 is .63, .71, .62, .74, .44, and .66. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Development and evaluation of a scale for measuring social acquiescence.

"A 56-item scale of social acquiescence has been developed by internal consistency item analysis. It has a reliability above .90 and correlates .95 with the tendency to accept or reject 300 proverbs, aphorisms, and similar statements. According to population differences and correlations between scales and various personality and intelligence measures, one who earns a high Social Acquiescence score resembles Sinclair Lewis' 'Babbitt'—an unquestioning conformer to social demands." (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The relationship between word frequency and perceptibility with a forced-choice technique.

"A number of studies have suggested that word thresholds obtained from the ascending method of limits are particularly susceptible to linguistic biases and hence may indicate little about perception. In the present investigation a forced-choice technique was employed in which S was presented with several words and required to identify the spatial location of a given word. With this technique, which reduces or eliminates response bias, the same positive relationship between frequency of prior usage and performance as had been found with the ascending method of limits was obtained. The implications of a perceptual interpretation of these findings were discussed." From Psyc Abstracts 36:04:4GG91T. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

A score method of constructing asymmetric confidence intervals for the mean of a rating scale item.

This article presents a generalization of the Score method of constructing confidence intervals for the population proportion (E. B. Wilson, 1927) to the case of the population mean of a rating scale item. A simulation study was conducted to assess the properties of the Score confidence interval in relation to the traditional Wald (A. Wald, 1943) confidence interval under a variety of conditions, including sample size, number of response options, extremeness of the population mean, and kurtosis of the response distribution. The results of the simulation study indicated that the Score interval usually outperformed the Wald interval, suggesting that the Score interval is a viable method of constructing confidence intervals for the population mean of a rating scale item. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Construct validity of an attention rating scale for traumatic brain injury.

Attention deficits are nearly ubiquitous after traumatic brain injury (TBI). In the subacute phase of moderate to severe TBI, these deficits may be difficult to measure with the precision needed to predict outcomes, assess degree of recovery, and monitor treatment response. This article reports the findings of four studies, three observational and one a randomized, controlled treatment trial of methylphenidate (MP), designed to provide construct validation of the Moss Attention Rating Scale (MARS), an observational measure of attention dysfunction following TBI. One hundred seven participants with moderate to severe TBI were enrolled during treatment on an inpatient rehabilitation unit. MARS scores were provided independently by four rehabilitation disciplines (Physical, Occupational and Speech Therapies and Nursing). Results indicated that the MARS: (1) is more strongly related to concurrent measures of cognitive versus physical disability, supporting its validity as a measure of cognition, (2) is more strongly related to concurrent psychometric measures of attention versus measures thought to rely less on attention, supporting its validity as a measure of attention; and (3) predicts 1-year outcomes of TBI better than psychometric measures of attention. However, the MARS (4) was not differentially affected by MP versus placebo treatment. Results support the construct validity and utility of the MARS, with further research needed to clarify its role in treatment outcome assessment. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Influence of performance evaluation rating segmentation on motivation and fairness perceptions.

This study investigated the impact of rating segmentation (i.e., the number of alternative appraisal categories available for rating employee performance) on motivation and perceptions of fairness. Participants were 305 student volunteers who performed a catalog search task, A 2 X 4 factorial design with 2 performance rating systems--low segmentation (3 categories) and moderate segmentation (5 categories)--and 4 performance levels was used. Overall, the results indicated that the 5-category system resulted in higher self-efficacy regarding participants' ability to reach the next higher rating category, higher goals for rating improvement, and higher rating improvements than the 3-category system. The effects of rating system and performance rating on rating improvement were partially mediated by self-efficacy and personal goals. The rating system and the performance rating affected perceptions of distributive and procedural justice. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Prediction of improvement in treated and untreated patients using the Rorschach prognostic rating scale.

An attempt was made to evaluate the predictive validity for improvement of the Rorschach Prognostic Rating Scale (RPRS) and the MMPI. A group of untreated psychiatric outpatients (N = 40) and a group of outpatients treated with psychotherapy (N = 21) were given the Rorschach and MMPI before their assignment to the treatment or waiting list groups. After a waiting or treatment period of approximately 6 mo., each patient was clinically rated as improved or unimproved. The RPRS was significantly (p     

Analysis of a job evaluation system.

When supervisors or craftsmen attempt to rate specialists, should they combine their judgments? What is the interrelationship between the parts and what is the reliability of the Air Force's job evaluation system? The latter was tried out on a sample of jobs with 50 Air Force Specialists as Ss. Consensus ratings did not seem necessary. Using too few factors seemed undesirable. A descrepancy was found between the results of 2 measures of reliability, although one, at least, provided fairly high values. (PsycINFO Database Record (c) 2011 APA, all rights reserved)     

Chance guessing in a forced-choice recognition task and the detection of malingering.

Guessing occurs on forced-choice (FC) tests for which responses cannot be based on relevant knowledge. Its importance is in inverse relation to the level of knowledge being measured, so that it becomes an increasing component of test scores as the level of knowledge decreases. It is also used as a benchmark to detect simulated impairment. This investigation examined the role of guessing in a 2-alternative FC face recognition test. Chance groups shown only the test items were asked to guess which were the targets, to measure the variation in scores likely to be found with pure guessing. Controls performed normally, and two simulation groups tried to fake amnesia. Results suggested that simple guessing in the chance group produced variable scores that overlapped both low genuine and "malingering" performance. Low control scores were hidden by the guessing "chance bonus," which ameliorated the apparent level of decline in memory. Simulators told of their role at the outset (before presentation of the target items) produced more convincing "amnesic" scores than those told only just before the test, who produced the expected below-chance level of score. It is suggested that guessing variability should be taken into account in interpreting FC scores. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Computer-administered interviews and rating scales.

To evaluate the value of computer-administered interviews and rating scales, the following topics are reviewed in the present article: (a) strengths and weaknesses of structured and unstructured assessment instruments, (b) advantages and disadvantages of computer administration, and (c) the validity and utility of computer-administered interviews and rating scales. Computer-administered evaluations are more comprehensive and reliable and less biased than evaluations routinely conducted in clinical practice. Also, the use of continuous monitoring systems, which increasingly entail the use of computer administration, has been related to improved treatment outcome. However, the use of computer-administered interviews and rating scales will sometimes lead to false positive diagnoses, and for this reason, it is recommended that computer assessment be combined with clinical judgment. (PsycINFO Database Record (c) 2011 APA, all rights reserved)     

Fakability of a forced-choice personality test under realistic high school employment conditions.

The Gordon Personal Profile was administered to junior and senior high school students for vocational guidance purposes. Three months later it was readministered as an employment test to students applying for jobs. Those not seeking jobs took the test again as a guidance test… .Individuals did not change their profile patterns substantially from a guidance situation to an employment situation, and mean increases for the group were found to be moderate." (PsycINFO Database Record (c) 2010 APA, all rights reserved)