Applied cognitive task analysis (ACTA): a practitioner's toolkit for understanding cognitive task demands

Cognitive task analysis (CTA) is a set of methods for identifying cognitive skills, or mental demands, needed to perform a task proficiently. The product of the task analysis can be used to inform the design of interfaces and training systems. However, CTA is resource intensive and has previously been of limited use to design practitioners. A streamlined method of CTA, Applied Cognitive Task Analysis (ACTA), is presented in this paper. ACTA consists of three interview methods that help the practitioner to extract information about the cognitive demands and skills required for a task. ACTA also allows the practitioner to represent this information in a format that will translate more directly into applied products, such as improved training scenarios or interface recommendations. This paper will describe the three methods, an evaluation study conducted to assess the usability and usefulness of the methods, and some directions for future research for making cognitive task analysis accessible to practitioners. ACTA techniques were found to be easy to use, flexible, and to provide clear output. The information and training materials developed based on ACTA interviews were found to be accurate and important for training purposes.     

Task analysis for the investigation of human error in safety-critical software design: a convergent methods approach

An investigation was conducted into sources of error within a safety-critical software design task. A number of convergent methods of task- and error-analysis were systematically applied: hierarchical task analysis (HTA), error log audit, error observation, work sample and laboratory experiment. HTA, which provided the framework for the deployment of subsequent methods, revealed possible weaknesses in the areas of task automation and job organization. Application of other methods within this more circumscribed context focused on the impact of task and job design issues. The use of a convergent methods approach draws attention to the benefits and shortcomings of individual analysis methods, and illustrates the advantages of combining techniques to analyse complex problems. The features that these techniques should possess are highlighted.     

Reporting results of latent growth modeling and multilevel modeling analyses: Some recommendations for rehabilitation psychology.

Objective: There has been a general increase in interest and use of modeling techniques that treat data as nested, whether it is people nested within larger units, such as families or treatment centers, or observations nested under people. The popularity can be witnessed by noting the number of new textbooks and articles related to latent growth curve modeling and multilevel modeling. This paper discusses both of these techniques in the context of longitudinal research designs, with the main purposes of highlighting some benefits and issues related to the use of these models and outlining guidelines for reporting results from studies using multilevel modeling or latent growth modeling. Implications: These longitudinal analytic techniques can be greatly beneficial to researchers conducting rehabilitation studies, but there are several issues related to their use and reporting that need to be taken into consideration. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Recommended guidelines for uniform reporting of pediatric advanced life support: the Pediatric Utstein Style. A statement for healthcare professionals from a task force of the American Academy of Pediatrics, the American Heart Association, and the European Resuscitation Council

This consensus document is an attempt to provide an organized method of reporting pediatric ALS data in out-of-hospital, emergency department, and in-hospital settings. For this methodology to gain wide acceptance, the task force encourages development of a common data set for both adult and pediatric ALS interventions. In addition, every effort should be made to ensure that consistent definitions are used in all age groups. As health care changes, we will all be challenged to document the effectiveness of what we currently do and show how new interventions or methods of treatment improve outcome and/or reduce cost. Only through collaborative research will we obtain the necessary data. For these reasons, and to improve the quality of care and patient outcomes, it is the hope of the task force that clinical researchers will follow the recommendations in this document. It is recognized that further refinements of this statement will be needed; these recommendations will improve only when researchers, clinicians, and EMS personnel use them, work with them, and modify them. Suggestions, recommendations, and other comments aimed at improving the reporting of pediatric resuscitation should be sent to Arno Zaritsky, MD, Eastern Virginia Medical School, Children's Hospital of The King's Daughter, Division of Critical Care Medicine, 601 Children's Lane, Norfolk, VA 23507.     

A computer-based reporting system in radiology of the chest

The aim of this study was to compare our computer-aided report writing system to standard techniques. A computer-aided reporting system for reporting on chest X-rays was developed and linked with a UNIX-based radiology information system. The reporting system consists of different text modules which can be composed to a complete radiological report. The quality of the report, the system's efficiency, flexibility and availability at the radiology information system (RIS), as well as its acceptance by the users, were criteria we took as a basis of evaluation. Acceptance and flexibility of the system were tested by assessing necessary changes and additions performed to text modules and by subjective evaluation. The time spent on working with the reporting system was evaluated by performing a time-analysis study comparing the new system to the two conventional methods, the tape system and handwriting. On average, the readers needed 22.6 text modules for each report. For the users the most important advantages of the new system are that primarily fewer changes to or additions to the text are necessary, and when they cannot be avoided they can be performed more easily. In comparison with the tape system and handwritten reports, it took on average two and a half to three times longer to report on a chest X-ray. The printed report had left the department within 6.4 h using the tape system within 0.4 h when handwritten and within 1.4 h when recorded by the reporting system. The completeness of the set of modules and the logical order were positively assessed by the users. The demand of time for reporting increases, but the final report is available sooner.     

Statistical methods in psychology journals: Guidelines and explanations.

In the light of continuing debate over the applications of significance testing in psychology journals and following the publication of J. Cohen's (1994) article, the Board of Scientific Affairs (BSA) of the American Psychological Association (APA) convened a committee called the Task Force on Statistical Interference (TFSI) whose charge was "to elucidate some of the controversial issues surrounding applications of statistics including significance testing and its alternatives; alternative underlying models and data transformation; and newer methods made possible by powerful computers" (BSA, personal communication, February 28, 1996). After extensive discussion, the BSA recommended that publishing an article in American Psychologist, as a way to initiate discussion in the field about changes in current practices of data analysis and reporting may be appropriate. This report follows that request. Following each guideline are comments, explanations, or elaborations assembled by L. Wilkinson for the task force and under its review. The report is concerned with the use of statistical methods only and is not meant as an assessment of research methods in general. The title and format of the report are adapted from an article by J. C. Bailar and F. Mosteller (1988). (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Assessment of digital EEG, quantitative EEG, and EEG brain mapping: report of the American Academy of Neurology and the American Clinical Neurophysiology Society

A. Digital EEG is an established substitute for recording, reviewing, and storing a paper EEG record. It is a clear technical advance over previous paper methods. It is highly recommended. (Class III evidence, Type C recommendation). B. EEG brain mapping and other advanced QEEG techniques should be used only by physicians highly skilled in clinical EEG, and only as an adjunct to and in conjunction with traditional EEG interpretation. These tests may be clinically useful only for patients who have been well selected on the basis of their clinical presentation. C. Certain quantitative EEG techniques are considered established as an addition to digital EEG in: C.1. Epilepsy: For screening for possible epileptic spikes or seizures in long-term EEG monitoring or ambulatory recording to facilitate subsequent expert visual EEG interpretation. (Class I and II evidence, Type A recommendation as a practice guideline). C.2. OR and ICU monitoring: For continuous EEG monitoring by frequency-trending to detect early, acute intracranial complications in the OR or ICU, and for screening for possible epileptic seizures in high-risk ICU patients. (Class II evidence, Type B recommendation as a practice option). D. Certain quantitative EEG techniques are considered possibly useful practice options as an addition to digital EEG in: D.1. Epilepsy: For topographic voltage and dipole analysis in presurgical evaluations. (Class II evidence, Type B recommendation). D.2. Cerebrovascular Disease: Based on Class II and III evidence, QEEG in expert hands may possibly be useful in evaluating certain patients with symptoms of cerebrovascular disease whose neuroimaging and routine EEG studies are not conclusive. (Type B recommendation). D.3. Dementia: Routine EEG has long been an established test used in evaluations of dementia and encephalopathy when the diagnosis remains unresolved after initial clinical evaluation. In occasional clinical evaluations, QEEG frequency analysis may be a useful adjunct to interpretation of the routine EEG when used in expert hands. (Class II and III evidence as a possibly useful test, Type B recommendation). E. On the basis of current clinical literature, opinions of most experts, and proposed rationales for their use, QEEG remains investigational for clinical use in postconcussion syndrome, mild or moderate head injury, learning disability, attention disorders, schizophrenia, depression, alcoholism, and drug abuse. (Class II and III evidence, Type D recommendation). F. On the basis of clinical and scientific evidence, opinions of most experts, and the technical and methodologic shortcomings, QEEG is not recommended for use in civil or criminal judicial proceedings. (Strong Class III evidence, Type E recommendation). G. Because of the very substantial risk of erroneous interpretations, it is unacceptable for any EEG brain mapping or other QEEG techniques to be used clinically by those who are not physicians highly skilled in clinical EEG interpretation. (Strong Class III evidence, Type E recommendation).     

Neuropathologic diagnosis of Alzheimer disease: interrater reliability in the assessment of senile plaques and neurofibrillary tangles

The potential advantages of using digital techniques instead of film-based radiography have been discussed extensively for the past 10 years. A major future application of digital techniques is computer-assisted diagnosis: the use of computer techniques to assist the radiologist in the diagnostic process. One aspect of this assistance is computer-assisted detection. The detection of small lung nodule has been recognized as a clinically difficult task for many years. Most of the literature has indicated that the rate for finding lung nodules (size range from 3 mm to 15 mm) is only approximately 65%, in those cases in which the undetected nodules could be found retrospectively. In recent published research, image processing techniques, such as thresholding and morphological analysis, have been used to enhance true-positive detection. However, these methods still produce many false-positive detections. We have been investigating the use of neural networks to distinguish true-positives nodule detections among those areas of interest that are generated from a signal enhanced image. The initial results show that the trained neural networks program can increase true-positive detections and moderately reduce the number of false-positive detections. The program reported here can perform three modes of lung nodule detection: thresholding, profile matching analysis, and neural network. This program is fully automatic and has been implemented in a DEC 5000/200 (Digital Equipment Corp, Maynard, MA) workstation. The total processing time for all three methods is less than 35 seconds. In this report, key image processing techniques and neural network for the lung nodule detection are described and the results of this initial study are reported.     

The science of choosing the right decision threshold in high-stakes diagnostics.

Many diagnostic tasks require that a threshold be set to convert evidence that is a matter of degree into a positive or negative decision. Although techniques of decision analysis used in psychology help one select the particular threshold that is appropriate to a given situation and purpose, just the concept of adjusting the threshold to the situation is not appreciated in many important practical arenas. Testing for the human immunodeficiency virus (HIV) and for dangerous flaws in aircraft structures are used here as illustrations. This article briefly reviews the relevant techniques and develops those 2 examples with data. It suggests that use of the decision techniques could substantially benefit individuals and society and asks how that use might be facilitated. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Reporting practices in confirmatory factor analysis: An overview and some recommendations.

Reporting practices in 194 confirmatory factor analysis studies (1,409 factor models) published in American Psychological Association journals from 1998 to 2006 were reviewed and compared with established reporting guidelines. Three research questions were addressed: (a) how do actual reporting practices compare with published guidelines? (b) how do researchers report model fit in light of divergent perspectives on the use of ancillary fit indices (e.g., L.-T. Hu & P. M. Bentler, 1999; H. W. Marsh, K.-T., Hau, & Z. Wen, 2004)? and (c) are fit measures that support hypothesized models reported more often than fit measures that are less favorable? Results indicate some positive findings with respect to reporting practices including proposing multiple models a priori and near universal reporting of the chi-square significance test. However, many deficiencies were found such as lack of information regarding missing data and assessment of normality. Additionally, the authors found increases in reported values of some incremental fit statistics and no statistically significant evidence that researchers selectively report measures of fit that support their preferred model. Recommendations for reporting are summarized and a checklist is provided to help editors, reviewers, and authors improve reporting practices. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Outpatient interventions for adolescent substance abuse: A quality of evidence review.

Previous reviews of outpatient interventions for adolescent substance abuse have been limited in the extent to which they considered the methodological quality of individual studies. The authors assessed 31 randomized trials of outpatient interventions for adolescent substance abuse on 14 attributes of trial quality. A quality of evidence score was calculated for each study and used to compare the evidence in support of different outpatient interventions. Across studies, frequently reported methodological attributes included presence of an active comparison condition, reporting of baseline data, use of treatment manuals, and verification of self-reported outcomes. Infrequently reported attributes included power and determination of sample size, techniques to randomize participants to condition, specification of hypotheses and primary outcomes, use of treatment adherence ratings, blind assessment, and inclusion of dropouts in the analysis. Treatment models with evidence of immediate superiority in 2 or more methodologically stronger studies included ecological family therapy, brief motivational interventions, and cognitive-behavioral therapy. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Using the 'new' statistics in nursing research

This paper argues that quantitative methods are under-used in nursing research. Although this is often because the qualitative approach is the most appropriate, it may also be because nurse researchers are not fully aware of modern, sophisticated data analysis techniques and have tended to use simple statistical techniques that often make the quantitative analysis of complex data very difficult and produce simplistic and unsatisfactory answers. The paper briefly discusses probability and survival modelling techniques suitable for use in complex nursing research situations and argues that these methods may help to bridge the qualitative-quantitative gap. Although these techniques are mathematically complex, they are easily applied in practice using dedicated computer programs. The paper describes their application using one such program, GLIM 4.     

Emerging standardization in pharmacoeconomics

In this study, we reviewed pharmacoeconomic guidelines from the United Kingdom, Spain, Italy, Australia, Canada, and the United States to determine areas of emerging standardization. We examined the published literature, publication guidelines of major health journals, and published and unpublished recommendations from various task forces and conferences on related topics. The review revealed several general principles for which there was consensus across guidelines. These common features included the importance of using and reporting transparent methods so that readers can easily understand what calculations are being performed on which data elements, minimizing bias, and providing justification for the methods and assumptions used. Differences were detected across guidelines on the following topics: type of pharmacoeconomic assessment, perspective, comparators and data sources, data analysis, cost analysis, future cost analysis, outcomes assessment, modeling, time horizon, discounting, disclosure, and generalizability. Evolution of economic guidelines hinges on whether the primary goal is to increase the consistency or increase the validity of economic assessments. Some balance between these two objectives is desirable.     

A systematic overview of the incidence of posterior capsule opacification

OBJECTIVE: Reported rates of posterior capsule opacification (PCO) vary widely and are based on various definitions of PCO, varying lengths and intervals of follow-up, and the use of different surgical techniques, intraocular lens (i.o.l.) designs, and methods of IOL implantation. This study was designed to obtain a more precise overall estimate of the incidence of PCO and to explore factors that might influence the rate of PCO development. DESIGN: A meta-analysis. METHODS: Published articles were selected for study based on a computerized MEDLINE search of the literature and a manual search of the bibliographies of relevant articles. Articles meeting selected inclusion criteria were reviewed systematically, and the reported data were abstracted and synthesized using the statistical techniques of meta-analysis. MAIN OUTCOME MEASURE: Pooled estimates of the proportion of eyes developing PCO at three postoperative timepoints--1 year, 3 years, and 5 years--were measured. RESULTS: There is significant heterogeneity among published rates of PCO. The overall pooled estimates (95% confidence limits) of the incidence of PCO were 11.8% (9.3%-14.3%) at 1 year, 20.7% (16.6%-24.9%) at 3 years, and 28.4% (18.4%-38.4%) at 5 years after surgery. There is no evidence of a significant decline in PCO incidence during the study period. CONCLUSIONS: Visually significant PCO develops in more than 25% of patients undergoing standard extracapsular cataract extraction or phacoemulsification with posterior chamber intraocular lens implantation over the first 5 years after surgery. Patient characteristics, surgical techniques, and differences in research design and reporting may account for some of the variability in reported rates. However, no specific factors were identified in the authors' analysis. More precise estimates of incidence and identification of risk factors for PCO will depend on the development of a standardized measurement of PCO and wider adoption of more rigorous study methodology.     

Reported physical symptoms elicited by unpredictable events and the Type A coronary-prone behavior pattern.

Hypothesized that unpredictable aversive events are causally linked to physical symptom reporting and that the Type A coronary-prone behavior pattern affects symptom reporting, such that Type A individuals fail to report symptoms when they expect to continue working on a task as compared to when they believe they have completed it. In the present study, 120 Type A and Type B female undergraduates (classified by scores on the Jenkins Activity Survey for Health Prediction) reported symptoms either at the end or in the middle of listening to unpredictable, predictable, or ambient noise in the laboratory. Unpredictable noise produced more symptom reporting than predictable noise, which in turn produced more symptom reporting than the ambient noise; Type A Ss reported fewer symptoms in the middle of the task than at the end, whereas Type Bs did not. Thus, both hypotheses were confirmed. (24 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Do procedures for verbal reporting of thinking have to be reactive? A meta-analysis and recommendations for best reporting methods.

Since its establishment, psychology has struggled to find valid methods for studying thoughts and subjective experiences. Thirty years ago, Ericsson and Simon (1980) proposed that participants can give concurrent verbal expression to their thoughts (think aloud) while completing tasks without changing objectively measurable performance (accuracy). In contrast, directed requests for concurrent verbal reports, such as explanations or directions to describe particular kinds of information, were predicted to change thought processes as a consequence of the need to generate this information, thus altering performance. By comparing performance of concurrent verbal reporting conditions with their matching silent control condition, Ericsson and Simon found several studies demonstrating that directed verbalization was associated with changes in performance. In contrast, the lack of effects of thinking aloud was merely suggested by a handful of experimental studies. In this article, Ericsson and Simon's model is tested by a meta-analysis of 94 studies comparing performance while giving concurrent verbalizations to a matching condition without verbalization. Findings based on nearly 3,500 participants show that the “think-aloud” effect size is indistinguishable from zero (r = –.03) and that this procedure remains nonreactive even after statistically controlling additional factors such as task type (primarily visual or nonvisual). In contrast, procedures that entail describing or explaining thoughts and actions are significantly reactive, leading to higher performance than silent control conditions. All verbal reporting procedures tend to increase times to complete tasks. These results suggest that think-aloud should be distinguished from other methods in future studies. Theoretical and practical implications are discussed. (PsycINFO Database Record (c) 2011 APA, all rights reserved)     

Delay Analysis within Construction Contracting Organizations

Delayed completion of a construction project is often caused by a complex interaction of a combination of events, some of which are the contractor’s risks and others are the project owner’s. The apportionment of the liability to give effect to the risk allocation has therefore been a matter of great controversy. Many delay analysis methodologies have been developed over the years for performing this task. This paper reports on an empirical study into the current practice in the use of these methodologies in the United Kingdom, as part of a wider study aimed at developing a framework for improving delay claims analysis. The part of the study reported here was based on a questionnaire survey of key informants. The issues investigated include the categories of staff within contracting organizations who contribute to delay claims analyses, the awareness, use and reliability of existing delay analysis methods and the obstacles to their use in practice. The main findings of the study are that: (1) the preparation of delay claims often requires input from commercial managers (quantity surveyors), schedulers, site managers, external claim consultants and estimators; (2) commercial managers have the greatest involvement; (3) claims analyzed using the as-built versus as-planned and the impacted as-planned techniques are often successful although there is considerable literature on the shortcomings of these techniques; and (4) the main obstacles to the use of the methods relates to deficiencies in project records and scheduling practice.     

Systematic review of maintenance of behavior change following physical activity and dietary interventions.

Objective: In the past decade, there has been no systematic review of the evidence for maintenance of physical activity and/or dietary behavior change following intervention (follow-up). This systematic review addressed three questions: 1) How frequently do trials report on maintenance of behavior change? 2) How frequently do interventions achieve maintenance of behavior change? 3) What sample, methodologic, or intervention characteristics are common to trials achieving maintenance? Design: Systematic review of trials that evaluated a physical activity and/or dietary behavior change intervention among adults, with measurement at preintervention, postintervention, and at least 3 months following intervention completion (follow-up). Main Outcome Measures: Maintenance of behavior change was defined as a significant between-groups difference at postintervention and at follow-up, for one or more physical activity and/or dietary outcome. Results: Maintenance outcomes were reported in 35% of the 157 intervention trials initially considered for review. Of the 29 trials that met all inclusion criteria, 21 (72%) achieved maintenance. Characteristics common to trials achieving maintenance included those related to sample characteristics (targeting women), study methods (higher attrition and pretrial behavioral screening), and intervention characteristics (longer duration [>24 weeks], face-to-face contact, use of more intervention strategies [>6], and use of follow-up prompts). Conclusions: Maintenance of physical activity and dietary behavior change is not often reported; when it is, it is often achieved. To advance the evidence, the field needs consensus on reporting of maintenance outcomes, controlled evaluations of intervention strategies to promote maintenance, and more detailed reporting of interventions. (PsycINFO Database Record (c) 2010 APA, all rights reserved)