Expert judgment: Some necessary conditions and an example.

Studied characteristics of expertise in situations where judges deal with multidimensional information. Psychometric criteria were advocated as being indicative of expert judgment: (a) Experts should tend to cluster variables in the same way when identifying and organizing cues; (b) expert judgment should be highly reliable (intrajudge reliability), show both convergent and discriminant validity, and be relatively free of judgmental bias when measuring cues; and (c) experts should weight and combine information in similar ways. These criteria were applied to data obtained when 3 medical pathologists rated the amount of histological signs in biopsy slides. Results show that these experts generally met the criteria, although they did not seem to weigh information similarly. (20 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Pesticide exposures and fetal death: a review of the epidemiologic literature

Despite considerable concern regarding the effects on reproductive outcome of exposures to pesticides, convincing evidence for the developmental toxicity of occupational and environmental pesticide exposure in humans is lacking. In this comprehensive review of the English language epidemiologic literature, we summarize studies that have examined potential associations between fetal deaths (both spontaneous abortions and stillbirths) and specific pesticides, as well as maternal and paternal employment in occupations with potential for exposure. While many of the epidemiologic studies to date suffer from methodologic problems, the data are suggestive of increased risks of fetal deaths associated with pesticides in general and maternal employment in the agricultural industry. There is a clear need for epidemiologic research that focuses on specific pesticide products or chemical families, with improved exposure assessment. The potential role of solvents in developmental toxicity associated with pesticide use by both males and females should also be considered.     

Effect of heavy metals and storage time on two types of forest litter: basal respiration rate and exchangeable metals

BACKGROUND: Techniques are needed to assess anesthesiologists' performance when responding to critical events. Patient simulators allow presentation of similar crisis situations to different clinicians. This study evaluated ratings of performance, and the interrater variability of the ratings, made by multiple independent observers viewing videotapes of simulated crises. METHODS: Raters scored the videotapes of 14 different teams that were managing two scenarios: malignant hyperthermia (MH) and cardiac arrest. Technical performance and crisis management behaviors were rated. Technical ratings could range from 0.0 to 1.0 based on scenario-specific checklists of appropriate actions. Ratings of 12 crisis management behaviors were made using a five-point ordinal scale. Several statistical assessments of interrater variability were applied. RESULTS: Technical ratings were high for most teams in both scenarios (0.78 +/- 0.08 for MH, 0.83 +/- 0.06 for cardiac arrest). Ratings of crisis management behavior varied, with some teams rated as minimally acceptable or poor (28% for MH, 14% for cardiac arrest). The agreement between raters was fair to excellent, depending on the item rated and the statistical test used. CONCLUSIONS: Both technical and behavioral performance can be assessed from videotapes of simulations. The behavioral rating system can be improved; one particular difficulty was aggregating a single rating for a behavior that fluctuated over time. These performance assessment tools might be useful for educational research or for tracking a resident's progress. The rating system needs more refinement before it can be used to assess clinical competence for residency graduation or board certification.     

Toward the development of structured criteria for interpretation of functional analysis data

Using functional analysis results to prescribe treatments is the preferred method for developing behavioral interventions. Little is known, however, about the reliability and validity of visual inspection for the interpretation of functional analysis data. The purpose of this investigation was to develop a set of structured criteria for visual inspection of multielement functional analyses that, when applied correctly, would increase interrater agreement and agreement with interpretations reached by expert consensus. In Study 1, 3 predoctoral interns interpreted functional analysis graphs, and interrater agreement was low (M = .46). In Study 2, 64 functional analysis graphs were interpreted by a panel of experts, and then a set of structured criteria were developed that yielded interpretive results similar to those of the panel (exact agreement = .94). In Study 3, the 3 predoctoral interns from Study 1 were trained to use the structured criteria, and the mean interrater agreement coefficient increased to .81. The results suggest that (a) the interpretation of functional analysis data may be less reliable than is generally assumed, (b) decision-making rules used by experts in the interpretation of functional analysis data can be operationalized, and (c) individuals can be trained to apply these rules accurately to increase interrater agreement. Potential uses of the criteria are discussed.     

Acute work-related poisoning by pesticides in The Netherlands; a one year follow-up study

The National Poisons Control Centre of the National Institute of Public Health and the Environment in the Netherlands conducted a prospective study on acute poisoning arising from exposure to pesticides in agricultural workers. The study was performed to determine the extent and severity of acute pesticide poisoning in the Netherlands and the working conditions that lead to these poisonings. All cases of potential acute occupational intoxication by pesticides in which the Poisons Control Centre was consulted in 1991 were thoroughly studied by an occupational hygienist and a specialist in internal medicine. With the consent of the patients and their physicians, the patients' medical condition and the working conditions leading to exposure were investigated on the spot. After the exclusion of 73 patients (27 non-occupational exposures, 7 occupational exposures in non-agricultural workers, 1 accident occurred abroad, 32 patients with illnesses unrelated to pesticides and 6 who could not be traced for follow-up), 54 cases of possible acute work-related pesticide poisoning remained for study. In 37 of the 54 events there was a direct relation between exposure to pesticides and acute health problems. In one patient doubt remained about the origin of the complaints and in 16 of the 54 cases pesticide poisoning was highly unlikely and the complaints could be attributed to other diseases. In the 37 remaining cases symptoms consisted of skin and/or eye lesions (23 cases) and systemic health effects (14 cases). Exposure to the soil disinfectant 1,3-dichloropropene resulted in severe skin damage. Direct contact of pesticides with the eyes invariability resulted in local irritation. Severe systemic poisonings occurred after exposure to organophosphate and carbamate insecticides and the soil disinfectant methyl bromide. Investigations at the site of the exposure revealed 43 cases of clear exposure to pesticides, in which, except for two cases, 1 worker per incident was involved. In 67% of the cases exposures took not place during pesticide dissemination, but during preparatory activities (35%), repair of application equipment (14%) and during re-entry (14%). In 79% of the cases splashing of pesticides or spray drift led to the exposure. In most accidents (74%) imperfect technical design or technical defects were important risk factors for exposure. Although most workers were aware of the risk of using pesticides, they were still careless in taking adequate protective measures. Especially during preparatory and reparations activities the wearing of protective clothing has to be emphasized.     

Effects on mock jurors of experts favorable and unfavorable toward hypnotically elicited eyewitness testimony.

Mock jurors were told that a rape victim's initial identification of the defendant was made either during a police interrogation or a hypnotic interrogation. Jurors given the hypnotic interrogation case then saw a videotape of an expert who was favorable toward hypnotically elicited testimony, an expert who was unfavorable, or both experts. Jurors' private beliefs of the defendant's guilt (assessed before exposure to the experts) was not influenced by knowledge that the witness had been hypnotized. However, jurors' guilt ratings were significantly influenced by both the favorable and the unfavorable experts and by the process of deliberation. Juries in all treatments strongly favored acquittal, and jury verdicts were not significantly influenced by expert testimony. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Interrater reliability and agreement of subjective judgments.

Indexes of interrater reliability and agreement are reviewed, and suggestions are made regarding their use in counseling psychology research. The distinction between agreement and reliability is clarified, and the relationships between these indexes and the level of measurement and type of replication are discussed. Indexes of interrater reliability appropriate for use with ordinal and interval scales are considered. The intraclass correlation as a measure of interrater reliability is discussed in terms of the treatment of between-raters variance and the appropriateness of reliability estimates based on composite or individual ratings. The advisability of optimal weighting schemes for calculating composite ratings is also considered. Measures of interrater agreement for ordinal and interval scales are described, as are measures of interrater agreement for data at the nominal level of measurement. (54 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The kinetics of cytokine gene expression in the thyroids of mice developing granulomatous experimental autoimmune thyroiditis

It was hypothesized that occupational exposure to pesticides during a spraying season causes changes in semen quality that might be detected in a longitudinal study. We analyzed the within-person changes in semen quality and reproductive hormones across a spraying season in groups of farmers using and not using pesticides. A total of 248 men collected two semen samples (participation rate: 32%). The median sperm concentration declined significantly from the first to the second sample in both groups, but there was no statistical difference in the decline between the two groups, unadjusted or adjusted. Only minor changes were found in sperm morphology, vitality, motility, sperm chromatin denaturation (SCSA), and reproductive hormones, and the differences in changes between the two groups were nonsignificant, or, in the opposite direction to the expected. There was no relation between the changes in sperm parameters in relation to pesticide exposure variables. In conclusion, use of pesticides by Danish farmers is not a likely cause of short-term effects on semen quality and reproductive hormones.     

Childhood leukaemia and exposure to pesticides: results of a case-control study in northern Germany

The association between childhood leukaemia and exposure to pesticides was examined in a population-based case-control study conducted in Lower Saxony, Northern Germany. Between July 1988 and June 1992, 219 newly diagnosed cases were identified, of whom 173 participated in the study. Two sex- and age-matched control groups were recruited: local controls from the same communities as the newly diagnosed cases of leukaemia and state controls from other randomly selected communities in Lower Saxony. An additional study group consisted of 175 cases of solid tumours. When the leukaemia cases were compared with the local controls, positive associations with parental occupational exposure, particularly agriculture-related exposure, were observed, which were statistically non-significant. A significant association was found for pesticide use in gardens (odds ratio = 2.52, 95% confidence interval: 1.0-6.1). No positive associations were seen when the leukaemia cases were compared to the state controls, but this finding could be explained by a higher proportion of state controls living in rural areas. In communities with a significantly elevated standardised incidence ratio of childhood leukaemia over the last decade (1984-1993), the prevalence of pesticide use in the garden was 21%, compared with the 10% in other communities. None of the examined risk factors were more common among cases of solid tumours. Our findings add some evidence to the hypothesis that pesticides are a risk factor for childhood leukaemia, and there are good reasons to consider abundant pesticide use in rural areas as a possible cause for clustering of childhood leukaemia.     

Primary fallopian tube cancer: a review of the literature

BACKGROUND: We conducted a population-based case-control study to describe the relationship between occupational exposure to estrogenic chemicals and the occurrence of breast cancer in Cape Cod, Massachusetts. METHODS: Incident cases of breast cancer (n = 261) diagnosed from 1983 through 1986 and controls (n = 753) were interviewed to gather information on breast cancer risk factors and all full-time jobs held since age 18. Blinded exposure assessments were employed using the data from the NIOSH National Occupational Exposure Survey, chemical production and usage information, and the expert judgement of a certified industrial hygienist. RESULTS: Overall, 29.5% of cases and 32.5% of controls had probable occupational exposure to one or more xenoestrogens. Probable exposure to nonylphenol (21.5% of cases, 21.4% of controls), butyl benzyl phthalate (10.0% of cases, 13.2% of controls), BHA (7.3% of cases, 9.6% of controls), bisphenol A (9.6% of cases, 11.6% of controls), and 4-tert-butylphenol (2.7% of cases and 5.3% of controls) were relatively commons, while probable exposure to the other xenestrogens was rare. Only PCBs and 4-octylphenol were associated with moderate increase in the odds of breast cancer (PCBs: 5 exposed cases and 6 exposed controls, adjust odds ratio: 3.2, 95% CI = 0.8-12.2, and 4-octylphenol: 6 exposed cases and 5 exposed controls, adjusted odds ratio: 2.9, 95% CI = 10.8).     

Possible association of a cholecystokinin promotor polymorphism (CCK-36CT) with panic disorder

Classification of Crohn's disease (CD) by disease behavior--either inflammatory (INF), fibrostenotic (FS), or fistulizing/perforating (FP)--has been proposed as a means of assisting management decisions and predicting outcomes for subgroup analysis in clinical trials and for making phenotype/genotype associations in molecular genetic studies. Accurate and reproducible classification of CD patient subgroups is of paramount importance in such studies but to be useful, the classification scheme must have good interrater agreement. We sought to assess the interrater agreement associated with the disease-behavior classification scheme of CD. Twelve patients with CD were randomly selected from a database of 964 patients with CD undergoing medical or surgical treatment or both. Clinical details of the 12 cases, along with their radiographs and surgical and pathological reports, were presented to a panel of 20 experts who were asked to classify each case based on the patient's overall disease course (scenario A) and as if the patient were being entered into a clinical trial on that day (scenario B). Calculations of strength of interrater agreement were made and were expressed as the kappa statistic (kappa), with kappa < 0.2 = poor strength of agreement; kappa 0.21 - 0.4 = fair; kappa 0.41 - 0.6 = moderate; kappa 0.61 - 0.8 = good; and kappa 0.81 - 1.0 = very good. Five panel participants did not complete the study, and three clinical vignettes were excluded because of incomplete scoring, leaving a total of 15 panel experts assessing nine cases. Overall interrater agreement was only fair with kappa = 0.353 for scenario A and kappa = 0.291 for scenario B. Interrater agreement was less when only the most straightforward case in each disease category was evaluated. Classification of CD by pattern of disease behavior yields only fair interrater agreement. This raises concerns regarding its applicability, particularly in ongoing studies of genotype/phenotype associations. Further refinement of disease subtypes and clear operational definitions are required.     

Intraocular lens materials and styles: a review

OBJECTIVE: To compare criteria for coronary revascularization developed by the expert panel process and by decision analysis. METHOD: We reviewed the medical records of 3080 chronic stable angina patients who either underwent coronary artery bypass graft surgery (CABG) or percutaneous transluminal coronary angioplasty (PTCA) and determined the agreement between appropriateness ratings made by two expert physician panels, one from the United States and the second from The Netherlands. We also evaluated the agreement between these panels' appropriateness ratings and a decision analytic model's effectiveness categories. RESULTS: There was poor agreement between U.S. and Dutch panel appropriateness ratings for PTCA (kappa = 0.03) and slight agreement for bypass surgery (kappa = 0.18). Dutch ratings had substantial agreement with the decision analytic models effectiveness categories for both PTCA and CABG (kappa = 0.83 and 0.79, respectively) whereas there was no systematic agreement between U.S. ratings and the decision analytic model for PTCA and poor agreement for CABG (kappa = 0.00 and 0.18, respectively). CONCLUSIONS: Although the level of agreement between expert panels and decision analysis on when a procedure is appropriate or effective may vary by procedure and the strength of the scientific evidence, we found that Dutch physicians agree much more strongly with decision analysis than U.S. physicians.     

Investigation into the repeated recovery of coagulase-negative staphylococci from blood taken at the end of cardiopulmonary by-pass

The assessment of needs has been put forth as an important dimension in both the planning, development and evaluation of psychiatric services. A needs assessment tool for routine use should ideally be brief, not require additional staff, be valid and reliable with respect to setting, gender and culture, and also be sensitive to changes in needs and differences in needs between subgroups of patients. In this study, the interrater reliability of the Swedish version of the Camberwell Assessment of Needs instruments (CAN), measuring needs in 22 areas, was investigated in a cross-sectional sample consisting of 119 psychiatric inpatients and outpatients. The prevalence and severity of needs, prevalence of unmet needs, as well as current help received was also investigated. The interrater reliability of the instrument was found to be good both measured as total percentages of agreement in ratings, where an agreement of 80% or above was found in 90% of the comparisons, and as measured by Cohen's kappa. Predominant needs in the sample were in addition to clinical needs concerning psychological distress and psychotic symptoms, social needs such as the need for company, daytime activities and information concerning condition and treatment. The CAN seems to be a instrument with a high interrater reliability also when used under routine conditions as in this study. It also seems to have a high degree of feasibility with regard to different types of patients and settings and a sensitivity concerning differences in needs in subgroups of patients.     

Development of categorization and reasoning in the natural world: Novices to experts, naive similarity to ecological knowledge.

Two experiments investigate the role of similarity and causal-ecological knowledge in expert and novice categorization and reasoning. In Experiment 1, university undergraduates and commercial fishermen sorted marine creatures into groups; although there was substantial agreement, novices sorted largely on the basis of appearance, whereas experts often cited commercial, ecological, or behavioral factors, and systematically subdivided fish on the basis of ecological niche. In Experiment 2, experts and novices were asked to generalize a blank property or novel disease from a pair of marine creatures. Novices relied on similarity to guide generalizations. Experts used similarity to reason about blank properties but ecological relations to reason about diseases. Expertise appears to involve knowledge of multiple relations among entities and context-sensitive application of those relations. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

On the "general acceptance" of eyewitness testimony research: A new survey of the experts.

In light of recent advances, this study updated a prior survey of eyewitness experts (S. M. Kassin, P. C. Ellsworth, & V. L. Smith, 1989). Sixty-four psychologists were asked about their courtroom experiences and opinions on 30 eyewitness phenomena. By an agreement rate of at least 80%, there was a strong consensus that the following phenomena are sufficiently reliable to present in court: the wording of questions, lineup instructions, confidence malleability, mug-shot-induced bias, postevent information, child witness suggestibility, attitudes and expectations, hypnotic suggestibility, alcoholic intoxication, the cross-race bias, weapon focus, the accuracy–confidence correlation, the forgetting curve, exposure time, presentation format, and unconscious transference. Results also indicate that these experts set high standards before agreeing to testify. Despite limitations, these results should help to shape expert testimony so that it more accurately represents opinions in the scientific community. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

An investigation of the validity of expert true score estimates in appraisal research.

Researchers of appraisal rating accuracy have defined ratee true scores as the mean ratings given by experts provided with enhanced opportunities to observe performance. The external validity of accuracy research depends on the relevance of these expert estimates to true performance. In our study, we provided expert raters with enhanced opportunities to view videotapes of five ratees under conditions of high true dimension intercorrelation or low true intercorrelation. The accuracy of expert ratings was compared with the accuracy of nonexpert ratings of the same tapes viewed under more typical rating conditions. Subjects' ratings were compared with ratee true scores defined in terms of objective worker output. Results indicated that experts were more accurate than nonexperts, regardless of the true dimension intercorrelations. Accuracy indices computed by using objective true scores were highly correlated with indices computed by using the mean expert ratings as true score estimates. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Pesticide poisoning: a major health problem in Sri Lanka

Acute pesticide poisoning is a major public health problem in Sri Lanka. In several agricultural districts, it precedes all other causes of death in government hospitals. Most of the acute poisoning cases are intentional (suicide) and occur among young adults, mainly males. Poisoning due to occupational exposure is also common, but less well documented. In an irrigation area in Sri Lanka a very high incidence of serious pesticide poisoning was observed, with 68% due to intentional ingestion of liquid pesticides. It is argued that the easy availability and widespread use of highly hazardous pesticides is the most important reason for this high number of poisoning cases. The frequent application of highly hazardous pesticides in high concentrations was often irrational and posed serious health and financial risks to the farmers. Sales promotion activities and credit facilities promoted this excessive pesticide use, which was not counteracted by an agricultural extension service. Hazardous practices when spraying pesticides were due to the impossibility of applying recommended protective measures under the local conditions, rather than to lack of knowledge. Current emphasis on programs that promote the safe use of pesticides through education and training of farmers will be ineffective in Sri Lanka because knowledge is already high and most poisoning cases are intentional. Instead, enforcement of legislation to restrict availability of the most hazardous pesticides would result in an immediate health benefit. Improved agricultural extension services to promote alternative non-chemical methods of pest control is the most important strategy, in the long term, to prevent acute pesticide poisoning.     

The risk of Parkinson's disease with exposure to pesticides, farming, well water, and rural living

We assessed exposure to pesticides, farming, well water use, and rural living as risk factors for Parkinson's disease (PD) in a population-based case-control study consisting of men and women > or = 50 years of age who had primary medical care at Henry Ford Health System in metropolitan Detroit. Enrolled PD patients (n = 144) and control subjects (n = 464) were frequency-matched for age, race, and sex. When adjusted for these variables and smoking status, there was a significant association of occupational exposure to herbicides (odds ratio [OR], 4.10; 95% CI, 1.37, 12.24) and insecticides (OR, 3.55; 95% CI, 1.75, 7.18) with PD, but no relation was found with fungicide exposure. Farming as an occupation was significantly associated with PD (OR, 2.79; 95% CI, 1.03, 7.55), but there was no increased risk of the disease with rural or farm residence or well water use. The association of occupational exposure to herbicides or insecticides with PD remained after adjustment for farming. The association of farming with PD was maintained after adjustment for occupational herbicide exposure and was of borderline significance after adjustment for occupational insecticide exposure. These results suggest that PD is associated with occupational exposure to herbicides and insecticides and to farming and that the risk of farming cannot be accounted for by pesticide exposure alone.