The effect of appraisal instrument on managerial perceptions of fairness and satisfaction with appraisals from their peers.

Examined whether the type of appraisal instrument (behavioral observation scales [BOS], behaviorally anchored rating scales [BARS], trait scales, or using no formal appraisal instrument) affected satisfaction with a peer appraisal and perceptions of fairness. 91 managers, while working in teams on a simulated task, provided one another with feedback. Satisfaction with peer appraisals was higher when BOS, BARS, or no formal instrument (control) was used to give feedback than was the case with a trait scale. Procedural justice was perceived as higher when either BOS or no instrument was used to give feedback than when the feedback was based on a trait scale. Trait scales were not perceived to be an acceptable instrument by peers for assessing their performance. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Multiple uses of performance appraisal: Prevalence and correlates.

Performance appraisal is used in organizations for a variety of purposes. However, little empirical research has been conducted to determine (a) the extent to which performance appraisal is used for each of several purposes in industry, (b) the extent to which appraisal data may be used for multiple and possibly conflicting uses within the same organization, and (c) organizational correlates of these uses. A survey questionnaire designed to answer these questions was mailed to 243 members of Division 14 of the American Psychological Association who were employed in industry. A factor analysis of the 106 completed questionnaires indicated four general uses of information from performance appraisals. The use of performance appraisal to simultaneously make distinctions between and within individuals is common. Canonical correlation analyses indicated that organizational characteristics were significantly related to uses of performance appraisal. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Task-based performance appraisal for blue-collar jobs: Evaluation of race and sex effects.

Court standards for job analysis were used to develop a task-based performance appraisal instrument for use with multiple blue-collar jobs in a small plastics plant. The instrument consisted of 10 performance categories (i.e., general reasoning, written communications) that were rated in terms of importance to the job and performance. Race and sex effects in performance appraisal scores and overall performance rankings of 150 employees (of whom 62 were Black and 88 were female) in 3 jobs were analyzed with separate factorial analyses of covariance. Neither race nor sex effects were significant with the performance appraisal data. There was a significant race difference with the overall ranking data, although it accounted for only 2% of the variance. In a 2nd study, a similar instrument was used to analyze race and sex differences in performance appraisal for 233 employees (of whom 31 were female, 50 were Black, and 30 were Hispanic) in 3 job groups of a large petroleum refinery. Neither race nor sex effects were significant. Findings support the use of the task-based approach to performance appraisal. (14 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

A study of guidelines for the conduct and analysis of simultaneous clinical replications.

Examined the development, implementation, and evaluation of guidelines for the simultaneous application of clinical replications. The 15 authors served as Ss. Success of the guidelines was evaluated by their ability to yield (1) information concerning the probability of the treatment's success, and (2) hypotheses concerning uncontrolled variables that may limit generalization of the treatment. Results support the validity of the guidelines in accomplishing these purposes. The use of such guidelines may serve as a vehicle for the integration of the roles of researcher and practitioner. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Radiology faculty evaluation: bi-institutional implementation of an experimental appraisal instrument

RATIONALE AND OBJECTIVES: The authors evaluated the reliability and validity of an experimental radiology faculty appraisal instrument. MATERIALS AND METHODS: Residents from the University of Wisconsin (UW) and Indiana University (IU) were asked to use a previously developed behaviorally based, 53-item experimental faculty appraisal instrument. Twenty UW residents evaluated 29 UW faculty members and 37 IU residents evaluated 31 IU faculty members by using the experimental instrument. Residents also evaluated faculty by using their institution's existing appraisal instrument. RESULTS: Correlations between existing and experimental forms were .69 and .87 for UW and IU, respectively. Existing form reliabilities were .89 and .94 and experimental form reliabilities were .98 and .98 for UW and IU, respectively. Experimental form length was reduced to 30 items by eliminating the questions that correlated the least with section scores. Reliabilities of scores on the shortened form were .97 and .98 and correlated .65 and .88 with scores on the longer form for UW and IU, respectively. CONCLUSION: Ratings obtained with the existing forms correlated substantially with the experimental form, attesting to the experimental form's validity. Shortening the experimental form had a minimal effect on the reliability and validity of the data obtained. The behavior-based form was used to rate behaviors that residents believed discriminated between effective and ineffective instructors, enabling an objective and relevant assessment to be made.     

A reader's guide to the evaluation of prognostic studies

In this article, we review guidelines which may be used to evaluate studies documenting prognosis. We describe a clinical problem involving the prognosis of a patient in an intensive care unit. An approach to the literature search is then outlined. The results of the literature search are described and criteria for the appraisal of articles describing prognosis and prognostic factors are discussed using one article as an example.     

Testing the validity of an instrument for assessing family of origin history

Family of origin history information is often gathered for research and clinical purposes in psychology, but there has been relatively little examination of the validity of the data obtained. This report describes several tests of the validity of the recently developed Family Background Questionnaire, a relatively behaviorally specific instrument designed to comprehensively assess characteristics of one's family of origin. Using a sample of 678 nonclinical and clinical participants, it was found that a history of incest, parental chemical dependency, clinical status, socioeconomic status, and birth order all predicted Family Background Questionnaire scores as hypothesized and that 38% of the variance in Total Scale scores was accounted for by these variables.     

Clinical practice as natural laboratory for psychotherapy research: A guide to case-based time-series analysis.

Both researchers and practitioners need to know more about how laboratory treatment protocols translate to real-world practice settings and how clinical innovations can be systematically tested and communicated to a skeptical scientific community. The single-case time-series study is well suited to opening a productive discourse between practice and laboratory. The appeal of case-based time-series studies, with multiple observations both before and after treatment, is that they enrich our design palette by providing the discipline another way to expand its empirical reach to practice settings and its subject matter to the contingencies of individual change. This article is a user's guide to conducting empirically respectable case-based time-series studies in a clinical practice or laboratory setting. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Estimating the satisfaction of information users

Several factors influence the responses of users of information retrieval systems who are asked to evaluate the service provided by a system. To realistically estimate user satisfaction, different aspects of the search outcome should be explored. This paper analyzes the data obtained in a follow-up questionnaire sent to a sample of MEDLINE users who had requested searches at seven information centers within a two-month period. The questionnaire requested the user's appraisal of the value of the search and of the usefulness of the retrieved references. A comparison of different items of the questionnaire revealed some contradictions between an overall appraisal of the service and more specific responses on the outcome of the search. Caution is advised in inferring the satisfaction of information needs from the users' evaluation of an information retrieval system.     

Are performance appraisal ratings from different rating sources comparable?

The purpose of this study was to test whether a multisource performance appraisal instrument exhibited measurement invariance across different groups of raters. Multiple-groups confirmatory factor analysis as well as item response theory (IRT) techniques were used to test for invariance of the rating instrument across self, peer, supervisor, and subordinate raters. The results of the confirmatory factor analysis indicated that the rating instrument was invariant across these rater groups. The IRT analysis yielded some evidence of differential item and test functioning, but it was limited to the effects of just 3 items and was trivial in magnitude. Taken together, the results suggest that the rating instrument could be regarded as invariant across the rater groups, thus supporting the practice of directly comparing their ratings. Implications for research and practice are discussed, as well as for understanding the meaning of between-source rating discrepancies. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Medical practice guidelines in malpractice litigation: an early retrospective

The role of practice guidelines in malpractice litigation has been discussed in some theoretical detail. Little information is available, however, on how courts use guidelines or on the effort of state legislatures to explicitly link guideline compliance with malpractice defenses. We review all relevant case law and legislative enactments to shed light on the influence of medical malpractice on guidelines. We also use data from a nationwide survey of malpractice attorneys to supplement our legal analysis. Although guidelines are being used for both inculpatory and exculpatory purposes in common-law litigation (a two-way street), legislatures are interested in applying them only for exculpatory purposes (a one-way street).     

On the AHCPR Depression in Primary Care guidelines: Further considerations for practitioners.

The majority of cases of clinical depression go unrecognized and untreated, despite the fact that depression is an eminently treatable disorder. The Agency for Health Care Policy and Research (AHCPR) recently published a set of clinical practice guidelines focused on depression in primary care settings. The review of the literature on which the guidelines are based is thorough and appropriate and should enhance the detection of depression and the quality of pharmacotherapy for depression. However, the guidelines encourage primary care physicians to provide pharmacotherapy to their depressed patients as the 1st line of treatment. The wisdom of this recommendation is questioned and revisions to the guidelines are suggested. Specifically, patients should be informed of the broad array of treatment options available and provided with a more balanced presentation of the potential benefits of psychotherapy for depression. Patients should decide which treatment alternative they wish to undergo. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Practice guidelines and the practice of medicine: is it the end of clinical judgment and expertise?

Clinical practice guidelines are an increasing part of efforts to improve the quality and reduce the cost of health care. They are recommendations for the evidence-based care of average patients, not rules for all patients. At best they are developed by panels representing a wide array of expertise and experience related to the clinical question, are based on comprehensive, critical review of scientific evidence, make clear how value judgments affect recommendations, and take into account all of the issues bearing on clinical decisions, such as effectiveness, risk, convenience, cost, cost-effectiveness, and the resources needed to carry out the recommendations. Physicians have a mixed opinion of guidelines, believing they are both useful summaries to improve the quality of care and potential tools to judge and control them. Although guidelines may point out the best research evidence to guide the care of average patients, they are not a substitute for clinical judgment, which should be applied to each individual patient.     

Toward an integrated program design: Evaluating the status of diversity training in a graduate school curriculum.

Domain D of the American Psychological Association's (APA's) accreditation guidelines (APA, 1995) requires doctoral training programs in psychology to address diversity. Measurement of diversity integration is difficult and often addresses only faculty and student composition (E. M. Altmaier; see record 1993-31088-001). This study had 3 purposes: (1) the development of an instrument for rating diversity inclusion in courses, the Diversity Rating Form (DRF; S. A. Kuba, 1995); (2) the comparison of faculty diversity inclusion according to demographic characteristics; and (3) the application of the DRF to a graduate school program. The DRF was completed by 613 students and 37 faculty members. It demonstrated high reliability (alpha?=?.95)and construct validity (r?=?.88, p?p?p?     

The Index of Drug Involvement: a partial validation

Social workers often need a wider range of assessment scales that can be used in education, research, and practice. The Index of Drug Involvement (IDI) is a short-form assessment scale that may be of use to social work educators, researchers, and practitioners who work with clients with drug abuse or chemical dependency problems. This article describes the IDI; provides information about its administration, scoring, and interpretation; and describes the initial research conducted to validate the instrument. This article provides information about the reliability of the IDI; reports the standard error of measurement; and presents findings concerning the content, construct, and criterion validity of the instrument. Also presented is initial information about the development and use of a clinical cutting score that will help practitioners evaluate the clinical significance of a drug abuse problem and that can be a guide for establishing initial and final treatment goals.     

Multirater feedback and conditions for change.

Discusses the use of multirater feedback (feedback from peers and subordinates) rather than manager feedback for appraisal purposes in the workplace. The author argues that using multirater feedback for administrative purposes ignores basic principles from the counseling and stress literature about how people change, invites rater inflation and poor data, and is naive to organizational issues of hierarchy, status, and power. The trend toward using multirater feedback is discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Guidelines for clinical practice

Clinical practice guidelines are systematically developed statements that are intended to support medical decision making in well-defined clinical situations. Essentially, their object is to reduce the variability in medical practice, to improve quality, and to make appropriated control of the financial resources possible. Internationally, ever more organisations, associations, and institutions are concerned with the development of guidelines in many different areas of care. Making implicit knowledge explicit is one of the associated advantages of guidelines: they have a potential utility in training, in process evaluation, and in the reevaluation of outcome studies. In liability issues, their existence has a double effect: they can be used to justify medical behaviour, and they constitute a generally accepted reference point. A derivative problem is the legal liability of the compilers of the guidelines. The principle of the guideline approach can be challenged academically: science cannot give a definition of optimal care with absolute certainty. What is called objectivity often rests on methodologically disputable analyses; also the opinion of opinion leaders is not always a guarantee for scientific soundness. Moreover, patients are not all identical: biological variability, situational factors, patient expectations, and other elements play a role in this differentiation. Clinicians are often hesitant with respect to clinical guidelines: they are afraid of cookbook medicine and curtailment of their professional autonomy. Patients fear reduction of individualization of care and the use of guidelines as a rationing instrument. The effects of the introduction of clinical practice guidelines on medical practice, on the results and on the cost of care vary but are generally considered to be favourable. The choice of appropriate strategies in development, dissemination, and implementation turns out to be of critical importance. The article ends with concrete suggestions for the various steps in the development of guidelines and their actual compilation.     

Spreadsheet evaluation of computerized medical records: the impact on quality, time, and money

Information and communication technologies are presumed to play a critical role in improving effectiveness and efficiency of clinical care. Although the most promising directions of technological development are microcomputer-generated computerized medical record systems, documenting their value has been a major challenge for health care providers. This paper proposes a 15-item spreadsheet instrument for evaluating computerized medical records, and demonstrates how it was experimentally applied to a 6-year long experience at three sites. In conclusion, preliminary implications and guidelines are drawn with regard to practice and research in this area.     

Parents' perceptions of caregiving: development and validation of a measure of processes

The authors describe the development of a 56-item questionnaire, the Measure of Processes of Care (MPOC-56), designed to find out what parents of a child with a chronic health problem think of the services they and their child receive and of how those services affect psychosocial outcome. The instrument is internally consistent and is reliable on retests. Its validity was shown by positive correlations of its five scales with parents' satisfaction, and negative correlations with parents' stress in relation to services received. The MPOC-56 is a generic measure which can be used for clinical, quality assurance and research purposes.