Cognitive Analysis of Intraoperative Critical Events: A Problem-Driven Approach to Aiding Clinicians’ Performance*

Human error is responsible for a large proportion of the anaesthesia mishaps that occur annually in the United States. Ventilation-related events (VRE) constitute a significant number of anaesthesia critical incidents. Monitoring equipment and their displays extend anaesthesiologists’ resources during VRE but at the expense of additional cognitive demands. This project is a cognitive analysis of intraoperative (inside the operating room) VRE. Goal–means networks were utilised to build a problem-solving model of clinicians’ management of patients’ ventilation during anaesthesia and to map the demands of VRE. The model was also used to identify challenging VRE that were then simulated using a comprehensive anaesthesia simulator. The response of eight experienced clinicians was captured on videotape and analysed to investigate the effectiveness of medical equipment in supporting clinical decision making during VRE.     

Design of Information Content and Layout for Process Control Based on Goal–Means Domain Analysis

With regard to the design of information content in information display, it is often claimed that the abstraction hierarchy (AH) of the work domain should be considered as a basis for identifying and structuring the information content. The primary advantage of AH-based analysis and design is that functionally abstracted information can systematically be identified and provided to the operator, which has rarely been presented in traditional displays. This study evaluated the effectiveness of providing functional information, which was abstracted and represented based on goal–means analysis along the AH, to the operator in two task situations (fault diagnosis and operation). The results showed that the operator’s performance was improved with the high-level information, and the latter’s utility became greater when the goal–means relations between information at different abstraction levels were exhibited. From the results, three design principles for information display can be drawn. First, information should be identified and displayed at multiple abstraction levels. Second, the goal–means relations among the abstraction levels should be explicitly presented, especially for analytical cognitive tasks. Third, information layout should support information integration along decomposition structure within an abstraction level as well as along abstraction levels.     

Accessing Context in Wearable Computers

We present an easy interaction technique for accessing location-based contextual data shown on a head-worn wearable computer display. Our technique, called Context Compass, is based on a regular compass metaphor. Each object belonging to the user’s current context is visualised on a linear compass shown on the screen. The object directly in front of the user is shown in the middle of the compass and can be activated. Whenever the user turns his or her head, the objects on the screen move accordingly. Therefore, an object can be selected by simply turning one’s head towards it. Context Compass consumes a minimal amount of screen space, making it ideal for usage with see-through head-worn displays. An initial pilot study, applying a newly developed usability method customised especially for Context Compass, revealed that Context Compass can be learned virtually immediately. Further, the method itself proved to be successful in evaluating techniques such as Context Compass.     

Evaluation of ecological interface design for nuclear process control: situation awareness effects

OBJECTIVE: We determine whether an ecological interface display for nuclear power plant operations supports improved situation awareness over traditional and user-centered displays in a realistic environment. BACKGROUND: Ecological interface design (EID) has not yet been fully evaluated with real operators facing realistic scenarios. METHOD: Ecological displays were evaluated alongside traditional and user-centered "advanced" displays in a full-scope nuclear power plant simulation. Licensed plant operators used the displays in realistic scenarios that either had procedural support or did not have procedural support. All three displays were evaluated for their ability to support operator situation awareness. RESULTS: A significant three-way interaction effect was observed on two independent measures of situation awareness. For both measures, ecological displays improved situation awareness in scenarios that did not have procedural support, primarily in the detection phases of those scenarios. No other pronounced effects appeared across both measures. CONCLUSIONS: The observed improvement was sufficiently large to suggest that EID could improve situation awareness in situations where procedures are unavailable. However, the EID displays did not lead to improved situation awareness in the other conditions of the evaluation, and participants using these displays occasionally underperformed on single measures of situation awareness. This suggests that the approach requires further development, particularly in integrating EID with procedural support. APPLICATION: This research has important findings for the ongoing development of the EID approach, the design of industrial operator displays, and design to support situation awareness.     

Display Design of Process Systems Based on Functional Modelling

The prevalent way to present information in industrial computer displays is by using piping and instrumentation diagrams. Such interfaces have sometimes resulted in difficulties for operators because they are not sufficient to fulfil their needs. A systematic way that supports interface design therefore has to be considered. In the new design framework, two questions must be answered. Firstly, a modelling method is required to describe a process system. Such a modelling method can define the information content that must be displayed in interfaces. Secondly, how to communicate this information to operators efficiently must be considered. This will provide a basis for determining the visual forms that the information should take. This study discusses interface design of human–machine systems from these two points of view. Based on other scholars’ work, a comprehensive set of functional primitives is summarised as a basis to build a functional model of process systems. A library of geometrical presentations for these primitives is then developed. To support effective interface design, the concept of ‘functional macro’ is introduced and a way to map functional model to interface display is illustrated by applying several principles. To make our ideas clear, a central heating system is taken as an example and its functional model is constructed. Based on the functional model, the information to be displayed is determined. Several functional macros are then found in the model and their corresponding displays are constructed. Finally, by using the library of geometrical presentations for functional primitives and functional macros, the display hierarchy of the central heating system is developed. Reusability of functional primitives makes it possible to use the methodology to support interface design of different process systems.     

Decision Making and Management of Dynamic Risk

Managing dynamic environments often requires decision making under uncertainty and risk. Two types of uncertainty are involved: uncertainty about the state and the evolution of the situation, and ‘openness’ of the possible actions to face possible consequences. In an experimental study on risk management in dynamic situations, two contrasted ‘ecological’ scenarios – transposed from effective situations of emergency management – were compared in order to identify the impact of their ‘openness’ in the subjects’ strategies for decision making. The ‘Lost Child’ scenario presented qualitative and irreversible consequences (child’s death) and high uncertainty; it exerted high demands both in risk assessment (risk representation) and action elaboration and choice. A less open situation (‘Hydrocarbon Fire’) required a main choice between two contrasted actions, with quantitative computable consequences. The strategies of ‘experimental subjects’ (university students) and ‘operative subjects’ (professional fire-fighter officers) were compared in order to evaluate the ecological validity of experimental research in this field, from the point of view of the subjects themselves. The two scenarios appeared to be independent, so that quite different models of decision making have to be hypothesised, differing by the importance of assessing risk and defining possible actions on the one hand, and by the process of choice on the other. ‘Experimental’ subjects dramatically differed from ‘operative’ subjects when confronted with the same scenario, particularly for the less technical but more demanding scenario. It is hypothesised that three components might account for the effect of the situations and for the differences between and within groups of subjects: importance of situation assessment, spatial abilities, and global orientation of activity in managing dynamic risk.     

Interactive volumetric information visualization for document corpus management

This paper describes aminimally immersive three-dimensional volumetric interactive information visualization system for management and analysis of document corpora. The system, SFA, uses glyph-based volume rendering, enabling more complex data relationships and information attributes to be visualized than traditional 2D and surface-based visualization systems. Two-handed interaction using three-space magnetic trackers and stereoscopic viewing are combined to produce aminimally immersive interactive system that enhances the user’s three-dimensional perception of the information space. This new system capitalizes on the human visual system’s pre-attentive learning capabilities to quickly analyze the displayed information. SFA is integrated with adocument management and information retrieval engine named Telltale. Together, these systems integrate visualization and document analysis technologies to solve the problem of analyzing large document corpora. We describe the usefulness of this system for the analysis and visualization of document similarity within acorpus of textual documents, and present an example exploring authorship of ancient Biblical texts. Received: 15 December 1997 / Revised: June 1999     

Human Factors in the Design of Supervisory Control Systems and Human–Machine Interfaces for Highly Automated Complex Systems

The design of control systems and human–machine interfaces in the field of complex and safety-critical environments remains today an open issue, in spite of the high technological evolution of the last decades. The increasing use of automation has improved efficiency, safety and ease of operations but, at the same time, it has complicated operators’ situation awareness and has changed the nature of their possible errors. The research activity described in this paper is an attempt to develop a methodological framework to support designers of control systems and human–machine interfaces. In particular, it focuses on the need for a deeply recursive approach related to the implementation of the systemic and human aspects of the design process of a human–machine system, intended as a Joint Cognitive System. A validating case study has been performed, based on the full application of the framework on the control of the turbine/alternator system of a thermoelectric power plant in northern Italy. Correspondence and offprint requests to: M. Piccini, Politecnico di Torino Dipartimento di Energetica, C.so Duca degli Abruzzi 24, 10129 Turin, Italy. Tel.: +39 011 564 4413; Fax: +39 011 564 4499; Email: mipiccin@polito.it     

The Display of Electronic Commerce within Virtual Environments

In today’s competitive business environment, the majority of companies are expected to be represented on the Internet in the form of an electronic commerce site. In an effort to keep up with current business trends, certain aspects of interface design such as those related to navigation and perception may be overlooked. For instance, the manner in which a visitor to the site might perceive the information displayed or the ease with which they navigate through the site may not be taken into consideration. This paper reports on the evaluation of the electronic commerce sites of three different companies, focusing specifically on the human factors issues such as perception and navigation. Heuristic evaluation, the most popular method for investigating user interface design, is the technique employed to assess each of these sites. In light of the results from the analysis of the evaluation data, virtual environments are suggested as a way of improving the navigation and perception display constraints.     

Features of Decision-Making in Process Control Tasks: The Relevance of the Work Context

This paper is centred on evaluating some significant features of decision-making in process control tasks. The study was carried out in a petrol refinery, specifically, with distillation console operators. Operator verbalisations were recorded during the completion of two specific tasks and later categorised by raters using a list of cognitive categories. The inter-rater reliability was calculated together with qualitative evaluations of the main overlaps among the categories. From the raters’ evaluations, flow diagrams were drawn that represented the plans and tactics developed by operators and the implied cognitive processes (evaluation, prediction, action, etc.). We found that the operators began the tasks with a primary global situation assessment that determined the choice of whether to cope with the task ‘step by step’ or ‘globally’. The results showed two patterns of decision sequences made in normal adjustment performance or in problem situations. Other findings are related to the importance of characteristics such as prediction, anticipation, feedback and the role of the alarms selected by a situation assessment and individual characteristics.     

Aware Community Portals: Shared Information Appliances for Transitional Spaces

People wish to maintain a level of awareness of timely information, including presence of others in the workplace and other social settings. We believe this provides better exchange, coordination and contact within a community, especially as people work in asynchronous times and distributed locations. The challenge is to develop lightweight techniques for awareness, interaction and communication using shared information appliances. In this paper, we describe the design of an exploratory responsive display projected within a shared workspace at the MIT Media Lab. The system uses visual sensing to provide relevant information and constructs traces of people’s activity over time. Such aware portals may be deployed in casual workplace domains, distributed workgroups, and everyday public spaces.     

Distributed Cognition in an Emergency Co-ordination Center

*Formerly at the Department of Communication Studies, Linko¨ping University, Sweden. Most of this work was conducted during the author’s employment at the Department of Communication Studies. Recent research concerning the control of complex systems stresses the systemic character of the work of the controlling system, including the number of people and artefacts as well as the environment. This study adds to the growing body of knowledge by focusing on the internal working of such a system. Our vantage point is the theoretical framework of distributed cognition. Through a field study of an emergency co-ordination centre we try to demonstrate how the team’s cognitive tasks, to assess an event and to dispatch adequate resources, are achieved by mutual awareness, joint situation assessment, and the co-ordinated use of the technology and the physical arrangement of the co-ordination room.     

Negotiating Use: Making Sense of Mobile Technology

This paper is based on a study of the ways in which a group negotiated the use of a new mobile technology. The group was made up of ski instructors who, during a one-week ski trip, were equipped with a mobile awareness device called the Hummingbird. The group was studied using ethnomethodologically inspired qualitative methods, with the focus on the group members’ different views of the Hummingbird’s intended use. Negotiations of use occurred using two methods: talk and action. The users negotiated issues such as where and when to use the technology, and whether to consider the Hummingbird a work tool or a gadget for social events. Further, the empirical results clearly show how negotiations of new, mobile technology differ from stationary technology.     

Effects of Airborne Data Link Communication on Demands, Workload and Situation Awareness

An airborne air-to-ground data link communication interface was evaluated in a multi-sector-planning scenario using an Airbus A 340 full flight simulator. In a close-to-reality experimental setting, eight professional crews performed a flight mission in a mixed voice/data link environment. Experimental factors were the medium (voice vs. data link), workload (low vs. high) and the role in the cockpit (pilot flying vs. pilot non-flying). Data link communication and the usability of the newly developed communication interface were rated positively by the pilots, but there is a clear preference for using a data link only in the phase of cruise. Cognitive demands were determined for selected sections of en-route flight. Demands are affected mainly by increased communication needs. In the pilots’ view, although a data link has no effect on safety or the possibilities of intervention, it causes more problems. The subjective workload, as measured with the NASA Task Load Index, increased moderately under data link conditions. A data link has no general effect on pilots’ situation awareness although flight plan negotiations with a data link cause a distraction of attention from monitoring tasks. The use of a data link has an impact on air-to-ground as well as intra-crew communication. Under data link conditions the pilot non-flying plays a more active role in the cockpit. Before introducing data link communication, several aspects of crew resource management have to be reconsidered. Correspondence and offprint requests to: T. Müller, Technical University of Berlin, Institute of Psychology and Ergonomics, Department of Human–Machine Systems, Jebensstrasse 1, 10623 Berlin, Germany.     

A Representational Framework for Scenarios of System Use

Scenarios are becoming widely used in three areas of system development: software engineering, human–computer interaction (HCI), and organisational process design. There are many reasons to use scenarios during system design. The one usually advanced in support of the practice is to aid the processes of validating the developers’ understanding of the customers’ or users’ work practices, organisational goals and structures, and system requirements. All three areas identified above deal with these processes, and not surprisingly this has given rise to a profusion of scenario-based practices and representations. Yet there has been little analysis of why scenarios should be useful, let alone whether they are. Only by having such a framework for understanding what scenarios are, and what they are for, can we begin to evaluate different scenario approaches in specific development contexts. This paper is a contribution toward such a framework. We lay out a space of representational possibilities for scenarios and enumerate a set of values or criteria that are important for different uses of scenarios. We then summarise several salient representations drawn from the software engineering, HCI, and organisational process design communities to clarify how these representational choices contribute to or detract from the goals of the respective practices. Finally, we discuss how scenario representations from one area of design may be useful in others, and we discuss the relationship between these representations and other significant early-design and requirements engineering practices.     

Information Systems Curricula Evaluated by a Method Engineering Framework

Method engineering (ME) deals with the selection and assembly of situation-specific methods for information systems development. In this paper we use ME with a somewhat unusual perspective, that is, an educational one. We introduce a procedure for the evaluation of information systems curricula within an ME framework. Using this approach it is possible to quantitatively characterise and compare information systems curricula, showing their relative strengths and weaknesses. As an example we evaluate three model curricula (IS’90, IS’97 and ISCC’99) and analyse their differences and similarities.     

Registration of technical drawings and calibrated images for industrial augmented reality

Despite tremendous progress in 3D modelling technology, most sites in traditional industries do not have a computer model of their facilities at their disposal. In these industries, 2D technical drawings are typically the most commonly used documents. In many cases, a database of fully calibrated and oriented photogrammetric images of parts of the plant is also available. These images are often used for metric measurement and 3D as-built modelling. For planning revamps and maintenance, it is necessary to use industrial drawings as well as images and 3D models represented in a common “world” coordinate system. This paper proposes a method for full integration of technical drawings, calibrated images and as-built 3D models. A new algorithm is developed in order to use only a few correspondences between points on a technical drawing and multiple images to estimate a metric planar transformation between the drawing and the world coordinate system. The paper describes the mathematical relationship between this transformation and the set of homographies needed for merging the technical drawing with all the calibrated images. The method is implemented and fully integrated into an industrial software we developed for 3D as-built reconstruction. We present examples of a real application, in which the method is successfully applied to create an augmented reality representation of a waste water plant. Accepted: 13 August 2001     

Rule-based document structure understanding with a fuzzy combination of layout and textual features

Document image processing is a crucial process in office automation and begins at the ‘OCR’ phase with difficulties in document ‘analysis’ and ‘understanding’. This paper presents a hybrid and comprehensive approach to document structure analysis. Hybrid in the sense that it makes use of layout (geometrical) as well as textual features of a given document. These features are the base for potential conditions which in turn are used to express fuzzy matched rules of an underlying rule base. Rules can be formulated based on features which might be observed within one specific layout object. However, rules can also express dependencies between different layout objects. In addition to its rule driven analysis, which allows an easy adaptation to specific domains with their specific logical objects, the system contains domain-independent markup algorithms for common objects (e.g., lists). Received June 19, 2000 / Revised November 8, 2000     

Password hardening based on keystroke dynamics

We present a novel approach to improving the security of passwords. In our approach, the legitimate user’s typing patterns (e.g., durations of keystrokes and latencies between keystrokes) are combined with the user’s password to generate a hardened password that is convincingly more secure than conventional passwords alone. In addition, our scheme automatically adapts to gradual changes in a user’s typing patterns while maintaining the same hardened password across multiple logins, for use in file encryption or other applications requiring a long-term secret key. Using empirical data and a prototype implementation of our scheme, we give evidence that our approach is viable in practice, in terms of ease of use, improved security, and performance. Published online: 26 October 2001