Methods for evaluating the provision of well child care.

BACKGROUND: Well child visits are important for reducing the incidence of avoidable illness and disease. The Omnibus Reconciliation Act of 1989 (OBRA '89) set goals for well child or Early and Periodic Screening, Diagnosis, and Treatment (EPSDT) visits. Many health plans are evaluated in terms of the number of well child visits provided, yet the method used for collecting the data influences the indicator results and conclusions drawn from them. METHODS: In a retrospective cohort design, medical record review and administrative data were compared as methods for ascertaining the number of well child visits received by Iowa Medicaid-eligible children for the period from July 1, 1997 through December 31, 1998. Compliance with the American Academy of Pediatrics' "Recommendations for Preventive Pediatric Health Care" periodicity guidelines was assessed. RESULTS: Using administrative data, 29.6% (n = 1,489) of children received a well child visit. If medical record review was used, 39.6% (n = 1,003) of children had a visit. The concordance between the rates was quite low (kappa = 0.30). Medical record review supported that an EPSDT visit was provided for only 68% of the children who had a claim or encounter billed as providing well child care (n = 441). DISCUSSION: Administrative data may underestimate the performance of EPSDT visits in comparison to medical record review. In addition, having a claim for an EPSDT visit did not necessarily mean the child received the basic components of a well child exam. The methodology for performance indicators used to evaluate health plans should be carefully validated.     

Assessing consumer perceptions of inpatient psychiatric treatment: the perceptions of care survey

BACKGROUND: Consumer perceptions of behavioral health care are widely recognized as important quality indicators. This article reports the development and use of the Perceptions of Care (PoC) survey, a standardized public domain measure of consumer perceptions of the quality of inpatient mental health or substance abuse care. The goals were to develop a low-cost, low-burden survey that would address important quality domains, allow for interprogram comparisons and national benchmarks, be useful for quality improvement purposes, and meet accreditation and payer requirements. METHODS: The sample was composed of 6,972 patients treated in 14 inpatient behavioral health or substance abuse treatment programs. The PoC survey was given to patients by program staff in the 24-hour period before discharge. RESULTS: Aggregate reports and ratings of care identified areas that are highly evaluated by consumers, as well as areas that provide opportunities for quality improvement. Factor analysis identified four domains of care, and a 100-point score was developed for each domain. Regression analyses identified significant predictors of perceptions of care for use in computing risk-adjusted scores. Unadjusted and adjusted scores were presented to demonstrate the impact of risk adjustment on quality of care scores and relative ranking of programs. Examples were given of how programs used survey results to improve the quality of care. DISCUSSION: Results demonstrated that the PoC survey is sensitive to detecting differences among inpatient behavioral health programs and can be useful in guiding quality improvement efforts. However, risk adjustment is important for appropriate interpretation of results.     

Sustainability of incentives for excellent research — The German case

The state authorities in Germany used to fund public sector research without controlling the performance of the research units. This has changed during past decade, where the dominant mechanism by which formerly unconditional state funds are allocated nowadays is indicator-based performance measurement. The indicator sets used to measure the research-related performance in the German public science sector are usually very narrow, often consisting exclusively of finished doctoral theses and third-party funds. Using a unique dataset of 473 German research units from astrophysics, nanotechnology, economics and biotechnology, this paper outlines principles for the construction of sensible indicator sets for the performance measurement of scientific research groups. It is argued that scientific production is multidimensional. Thus one-sided indicator sets that fail to cover the relevant output dimensions give rise to incentives that will ultimately lower the performance of the science sector in total. Indicator sets should strive for sustainable incentives, which can be guaranteed if the sets are broad enough. As a starting point it is shown that the very common performance indicator ‘acquired third-party funds’ may affect research efficiency negatively, especially if the level of third-party funds is already very high. Therefore, we conclude that third-party funds should be used with great care, if at all.     

土结相互作用对框架结构倒塌的影响研究

在结构的强震倒塌研究中, 结构底部土体的变形会影响上部结构的非线性性能和倒塌机制。该文以一10层钢筋混凝土框架结构为例, 利用纤维梁模型模拟梁、柱的非线性反应, 并根据ATC-40的土体弹簧模型分别考虑软弱土、中软土、中硬土的影响。分析表明, 考虑土结相互作用后, 结构顶点位移变大, 由此产生的结构侧移会加大结构的整体二阶效应, 同时结构的变形明显集中于首层, 对结构反应不利;从结构的破坏形态来看, 不考虑土结相互作用, 结构的破坏始于首层柱底, 而考虑土结相互作用后, 首层柱顶首先屈服破坏, 在上部结构塑性性能未充分发挥前, 结构底层发生倒塌破坏, 结构的抗倒塌能力降低。     

Evaluating journal performance metrics

Journals have been ranked on the basis of impact factors for a long time. This is a quality indicator, and often favours review journals with few articles. Integrated impact indicators try to factor in size (quantity) as well, and are correlated with total number of citations. The total number of papers in a portfolio can be considered a zeroth order performance indicator and the total number of citations a first order performance indicator. Indicators like the h-Index and the g-Index are actually performance indicators in that they integrate both quality and quantity assessment into a single number. The p-Index is another variant of this class of performance indicators and is based on the cubic root of a second order performance indicator called the exergy indicator. The Eigenfactor score and article influence are respectively first order quantity and quality indicators. In this paper, we confirm the above relationships.     

Developing performance measures for alcohol and other drug services in managed care plans. Washington Circle Group

BACKGROUND: Monitoring the quality and availability of alcohol and other drug (AOD) services must be a central tenet of any health-related performance measurement system. The Washington Circle Group (WCG), which was convened by the Center for Substance Abuse Treatment Office of Managed Care in March 1998, has developed a core set of performance measures for AOD services for public- and private-sector health plans. It is also collaborating with a broad range of stakeholders to ensure widespread adoption of these performance measures by health plans, private employers, public payers, and accrediting organizations. CORE PERFORMANCE MEASURES: Four domains were identified, with specific measures developed for each domain: (1) prevention/education, (2) recognition, (3) treatment (including initiation of alcohol and other plan services, linkage of detoxification and AOD plan services, treatment engagement, and interventions for family members/significant others), and (4) maintenance of treatment effects. CONTINUING EFFORTS: Four measures that are based on administrative information from health plans and two measures that require a consumer survey of behavioral health care are undergoing extensive pilot testing. The WCG has reached out to a broad range of stakeholders in performance measurement and managed care to acquaint them with the measures and to promote their investigation and adoption. As results of pilot testing become available, these outreach efforts will continue. CONCLUSIONS: Performance measures for AOD services need to become an integral part of a comprehensive set of behavioral and physical health performance measures for managed care plans.     

A recursive field-normalized bibliometric performance indicator: an application to the field of library and information science

Two commonly used ideas in the development of citation-based research performance indicators are the idea of normalizing citation counts based on a field classification scheme and the idea of recursive citation weighing (like in PageRank-inspired indicators). We combine these two ideas in a single indicator, referred to as the recursive mean normalized citation score indicator, and we study the validity of this indicator. Our empirical analysis shows that the proposed indicator is highly sensitive to the field classification scheme that is used. The indicator also has a strong tendency to reinforce biases caused by the classification scheme. Based on these observations, we advise against the use of indicators in which the idea of normalization based on a field classification scheme and the idea of recursive citation weighing are combined.     

Towards a new crown indicator: an empirical analysis

We present an empirical comparison between two normalization mechanisms for citation-based indicators of research performance. These mechanisms aim to normalize citation counts for the field and the year in which a publication was published. One mechanism is applied in the current so-called crown indicator of our institute. The other mechanism is applied in the new crown indicator that our institute is currently exploring. We find that at high aggregation levels, such as at the level of large research institutions or at the level of countries, the differences between the two mechanisms are very small. At lower aggregation levels, such as at the level of research groups or at the level of journals, the differences between the two mechanisms are somewhat larger. We pay special attention to the way in which recent publications are handled. These publications typically have very low citation counts and should therefore be handled with special care.     

International variation in clinical injury incidence: exploring the performance of indicators based on health care, anatomical and outcome criteria

OBJECTIVE: To analyse international variation in clinical injury incidence, and explore the performance of different injury indicators in cross-country comparisons. METHODS: Hospital discharge data of seven European countries (Austria, Denmark, Ireland, Netherlands, Norway, England and Wales) were analysed. We tested existing and newly developed indicators based on (a) health care use, (b) anatomical criteria, or (c) expected health outcome: admissions excluding day-cases (a), hospital stay 4+ (a) and 7+ days (a), (serious) long-bone fractures (b), selected radiological verifiable fractures 'SRVFs' (b), and indicators based on international (Global Burden of Disease) and Dutch disability weights). Assessment criteria were reduction in incidence variation and length of stay in hospital, and the association between incidence and mortality rates. RESULTS: Indicators based on health care use led to increased variation in incidence rates. Long bone fractures and SRVFs, and both indicators based on injuries with moderate to high disability showed similar variation in clinical incidence compared to the crude rates, smaller variation in median length of stay in hospital and a good association with mortality rates. CONCLUSION: No perfect or near perfect indicators of clinical injury incidence exist. For international comparisons, indicators based on disability weights, SRVFs and long bone fractures may be sensible indicators to use, in the absence of a direct measure of anatomical severity.     

Applying quality improvement principles to achieve healthy work organizations

BACKGROUND: Health care has used total quality management (TQM)/quality improvement (QI) methods to improve quality of care and patient safety. Research on healthy work organizations (HWOs) shows that some of the same work organization factors that affect employee outcomes such as quality of life and safety can also affect organizational outcomes such as profits and performance. An HWO is an organization that has both financial success and a healthy workforce. For a health care organization to have financial success it must provide high-quality care with efficient use of scarce resources. To have a healthy workforce, the workplace must be safe, provide good ergonomic design, and provide working conditions that help to mitigate the stress of health care work. INTEGRATING TQM/QI INTO THE HWO PARADIGM: If properly implemented and institutionalized, TQM/QI can serve as the mechanism by which to transform a health care organization into an HWO. To guide future research, a framework is proposed that links research on QI with research on HWOs in the belief that QI methods and interventions might be an effective means by which to create an HWO. Specific areas of research should focus on identifying the work organization, cultural, technological, and environmental factors that affect care processes; affect patient health, safety, and satisfaction; and indirectly affect patient health, safety, and satisfaction through their effects on staff and care process variables. SUMMARY: Integrating QI techniques within the paradigm of the HWO paradigm will make it possible to achieve greater improvements in the health of health care organizations and the populations they serve.     

Determinants of success for biomedical researchers: a perception-based study in a health science research environment

New institutions are coming to the fore as stakeholders in research, particularly hospitals and clinical departments involved in providing health care. As a result, new environments for research are gaining importance. This study aims to investigate how different individual characteristics, together with collective and contextual factors, affect the activity and performance of researchers in the particular setting of hospitals and research centres affiliated with the Spanish National Health System (NHS). We used a combination of quantitative science indicators and perception-based data obtained through a survey of researchers working at NHS hospitals and research centres. Inbreeding and involvement in clinical research is the combination of factors with the greatest influence on scientific productivity, because these factors are associated with increased scientific output both overall as well as in high-impact journals. Ultimately, however, satisfaction with human resources in research group combined with gender (linked in turn to leadership) is the combination of factors associated most clearly with the most relevant indicator of productivity success, i.e. the number of articles in high-impact journals as principal author. Researchers’ competitiveness in obtaining research funding as principal investigator is associated with a combination of satisfaction with research autonomy and involvement in clinical research. Researchers’ success is not significantly related with their age, seniority and international experience. The way health care institutions manage and combine the factors likely to influence research may be critical for the development and maintenance of research-conducive environments, and ultimately for the success of research carried out in hospitals and other settings within the national public health system.     

The combined effect of public profiling and quality improvement efforts on heart failure management

BACKGROUND: A before-and-after study was conducted to examine the combined effect of public profiling and quality improvement activities on management of heart failure (HF) in the hospital setting. METHODS: Thirty-one hospitals in southeastern Michigan participated in this profiling and quality improvement study. One hospital closed after the baseline measurement. Two quality indicators were developed to evaluate the key processes of HF care, and one profiling indicator was designed for public profiling. The baseline results of the profiling indicator were publicly released. The individual hospitals were identified in the profiling report by name as "having statistically higher (or lower) rates than average." Remeasurement results were compared to the baseline results by using t-tests for the individual hospitals and all 30 hospitals as an aggregate. RESULTS: Two-thirds of the hospitals improved ejection fraction documentation; the aggregate result improved 5.4 percentage points (p < 0.05). No change was observed in the aggregate measure of prescribing angiotensin-converting enzyme inhibitors (ACEIs) to eligible HF patients at discharge. Hospitals with low baseline rates made improvement in ACEI use at discharge, but those with good baseline performance tended to decline in performance. There was a 2.2 percentage point increase (p < 0.05) in the profiling indicator. SUMMARY AND CONCLUSIONS: There seemed to be differential impacts of interventions across indicators and hospitals. Public profiling may have the most positive impact on hospitals with low performance at baseline. Maintaining the baseline good practice was a struggle for hospitals with relatively high baseline rates.     

Applying the domain management model in treating patients with chronic diseases

BACKGROUND: In the health care system in the United States, the management of chronic health conditions and their functional consequences challenge and frustrate patients, caregivers/families, health care providers, and physicians. Contributing factors include a lack of physician and health care provider training and a health system that emphasizes diagnosis and management of acute illnesses. A broader patient care model is required for patients with chronic disease(s). USING THE DOMAIN MANAGEMENT MODEL (DMM) TO CLASSIFY PATIENTS' CLINICAL PROBLEMS: The DMM is a synthesis of approaches used in internal medicine, geriatric medicine, and physical medicine and rehabilitation. All clinical problems, their treatments, and their outcomes can be classified and followed over time in a multiaxial model with four domains-medical/surgical issues, mental status/emotions/coping, physical function, and living environment. APPLICATIONS OF THE DMM IN MEDICAL RECORD TEMPLATES: Use of the four domain headings in standard templates can lead to an improved awareness of all the relevant issues in the management of chronic illnesses. This awareness precedes a physician's implementation of better care processes. Also, good patient care decisions require good information. MANAGEMENT OF FUNCTIONAL PROBLEMS: The DMM can be used to educate care providers and organize care in terms of important and common functional problem (for example, trouble walking, which lacks a standard approach in health care). CONCLUSION: This common framework for the organization, documentation, and communication of patients' care over time will help teach systematic mangement of chronic health conditions and help with future research on complex patient management.     

“Come and share a story with me”: Promoting engagement between Ethiopian and Non-Ethiopian Israelis via joint digital narratives

Does building a collaborative digital story lead to reduction of intergroup tension? Studies show that joint storytelling can help reach deeper levels of understanding through exposure to another group's narrative. We hypothesized that engagement in a collaborative digital storytelling task would lead to reductions of negative emotions and attitudes. In our study, dyads of Ethiopian and non-Ethiopian Israelis interacted via a unique digital platform that supports storytelling in a comics-like format aiming to reduce intergroup tension. Their responses to this task were measured before and after the joint activity. Findings showed a decrease in negative emotional states in intergroup contexts (STAI) between pre-and post-intervention for members of both groups, and a slight decrease in negative attitudes toward the out-group. In other words, joint-digital storytelling sessions lead groups in complex relations to be more likely to support engagement. This study has implications for the effect of contact on intergroup tension.     

Conference overview: through the patient's eyes--improvement strategies that work.

BACKGROUND: To promote a continuing dialogue among innovators in patient-centered health care quality assessment and improvement, The Picker Institute (Boston) sponsored conferences in 1995 and 1996, and in July 1997 launched the first of its summer symposia designed primarily for its clients. This overview summarizes the work and ideas presented at The Picker Institute's second summer symposium, "Through the Patient's Eyes: Improvement Strategies That Work," held in Cambridge, Massachusetts, on July 9-10, 1998. ISSUES AND PARTICIPANTS: Plenary session speaker David H. Gustafson, PhD, emphasized four key themes in his discussion of breakthrough improvement and service-focus on customers, innovation through information technology, an empirical methodology for predicting success, and leadership. Donald M. Berwick, MD, argued that it is better to treat the consumer not as an inspector but as an integral element in the total system of health care. In the closing plenary session, John Stone, MD, stated that although physicians like to think of themselves as teachers, it is the patients who teach them with the stories they bring. "Listen to the patient," he concluded. "The patient is telling you the diagnosis." Health care researchers and professionals, representing a wide variety of settings and patient conditions, reported in breakout sessions on their practical experiences using patient-generated data on quality and strategies for improving care. Several presenters described their practical experience using patient-centered measures as part of a coordinated approach to systemwide improvement. In other sessions, presenters offered advice about how to present patient survey data to colleagues and encourage their participation in prioritizing and acting on improvement opportunities.     

Designing an intervention to improve the management of Helicobacter pylori infection

BACKGROUND: An intervention to improve the testing and treatment of Helicobacter pylori (HP) in patients receiving chronic acid suppression (AS) therapy was developed at Harvard Pilgrim Health Care (HPHC), a mixed-model not-for-profit health maintenance organization. METHODS: Ten full-time primary care physicians (4 staff model and 6 group practice) were interviewed in 1999 about their knowledge, attitudes, and practice regarding dyspepsia, the use of chronic AS drugs, and approaches to HP infection, as well as about the feasibility and acceptability of various potential interventions that might be used in a quality improvement program. RESULTS: Self-reported practice regarding dyspepsia and HP infection were relatively uniform, and physicians were generally aware of current recommendations. Three common misperceptions acted as barriers to optimal HP management: Untreated HP was not considered an important problem; patients who used drugs for chronic AS rarely had HP infection; and chronic use of AS drugs was considered effective and without adverse consequence. All physicians wanted brief educational materials with explicit guidelines, preferably locally adapted and endorsed by local experts. All informants agreed that the main barrier to successful QI interventions was the requirement for any extra time or effort, particularly when directed at populations of patients who do not have symptomatic complaints. DISCUSSION: The interviews revealed the many barriers to improving the management of HP infection and to targeting educational messages and tailoring different methods for facilitating practice change across different managed care settings. Evidence-based components of the intervention program include physician education, a notification/reminder system, and practice-based tools to facilitate change and minimize workload.     

Illness as a social construct: understanding what asthma means to the patient to better treat the disease

BACKGROUND: Improved medical outcomes and patient satisfaction with health care may not always result from effective interventions on disease. Asthma is increasing in prevalence, morbidity, and mortality, despite a revolution in medical technology. Patient-created illness narratives were used to move beyond the biomedical model of disease to understand the social forces at work on the patient's experience of illness and his or her responses to it. METHODS: Children and adolescents with moderate to severe asthma used Video Intervention/Prevention Assessment (VIA) to create visual narratives of their experiences living with and managing asthma. The illness narratives included documentation of the patients' day-to-day lives and medical management, interviews of family and friends, and personal monologues of their thoughts and feelings. Narratives were coded for illness-related attitudes and behaviors. Emerging themes were identified using grounded theory and analyzed through the theoretical frameworks of medicine, psychology, clinical social work, and anthropology. RESULTS: Visual narratives were created by 20 participants from 8 to 25 years of age and of diverse racial and ethnic backgrounds. The patients' illness responses varied, presenting in one or more psychological categories-disability, denial, self-comforting, and "specialness"--and in three distinct social domains-family/home, friends/school or work, and patient-clinician encounters. CONCLUSIONS: Participants' visual narratives yielded a unique understanding of illness as a social construct. Patients with comparable disease states had different illness experiences, sometimes motivating health-related behaviors that appeared inconsistent with the management needs of the disease. Improvement in outcomes and patient satisfaction may be achieved through a broader understanding of illness in the social context of patients' lives.     

不同认知难度影响因素下数据信息搜索的视觉生理反应规律

目的 数字化产线数据监控界面中的数据图表呈现方式对监测员认知绩效的影响十分关键.以数据形式、任务种类及颜色和呈现方式为变量,依次展开行为反应与眼动追踪实验.方法 以产线生产制造过程中的损失工时数据为实验材料设计实验,通过记录行为反应数据中的反应时间和正确率,以及眼动追踪数据中的凝视时间和瞳孔直径大小指标,分析并讨论在不同影响因素下具有认知难度差异的数据图表认读实验中,反应时间与正确率、凝视时间与瞳孔直径大小间的反应变化一致性规律.结果 数据形式、任务种类对行为反应指标,以及呈现方式、颜色对视觉反应指标均具有显著性影响.结论 随着数据图表不同影响因素下的认知难度差异变化,行为反应体现为正确率与反应时间呈一定程度负相关;视觉反应指标体现为凝视时间与瞳孔直径变化的一致性,并且瞳孔直径变化对颜色因素具有更高敏感度.该结论能在数据图表认知绩效的评价与设计优化中起到一定参考作用.     

Use of consumer ratings for quality improvement in behavioral health insurance plans

BACKGROUND: The Consumer Assessment of Behavioral Healthcare Services (CABHS) survey collects consumers' reports about their health care plans and treatment. The use of the CABHS to identify opportunities for improvement, with specific attention to how organizations have used the survey information for quality improvement, is described. METHODS: In 1998 and 1999, data were collected from five groups of adult patients in commercial health plans and five groups of adult patients in public assistance health plans with services received through four organizations (one of three managed behavioral health care organizations or a health system). Patients who received behavioral health care services during the previous year were mailed the CABHS survey. Non-respondents were contacted by telephone to complete the survey. RESULTS: Response rates ranged from 49% to 65% for commercial patient groups and from 36% to 51% for public assistance patients. Promptly getting treatment from clinicians and aspects of care most influenced by health plan policies and operations, such as access to treatment and plan administrative services, received the least positive responses, whereas questions about communication received the most positive responses. In addition, questions about access- and plan-related aspects of quality showed the most interplan variability. Three of the organizations in this study focused quality improvement efforts on access to treatment. DISCUSSION: Surveys such as the CABHS can identify aspects of the plan and treatment that are improvement priorities. Use of these data is likely to extend beyond the behavioral health plan to consumers, purchasers, regulators, and policymakers, particularly because the National Committee for Quality Assurance is encouraging behavioral health plans to use a similar survey for accreditation purposes.