Work domain analysis and sensors II: Pasteurizer II case study

In this paper we establish a foundation for understanding the instrumentation needs of complex dynamic systems if ecological interface design (EID)-based interfaces are to be robust in the face of instrumentation failures. EID-based interfaces often include configural displays which reveal the higher-order properties of complex systems. However, concerns have been expressed that such displays might be misleading when instrumentation is unreliable or unavailable. Rasmussen's abstraction hierarchy (AH) formalism can be extended to include representations of sensors near the functions or properties about which they provide information, resulting in what we call a “sensor-annotated abstraction hierarchy”. Sensor-annotated AHs help the analyst determine the impact of different instrumentation engineering policies on higher-order system information by showing how the data provided from individual sensors propagates within and across levels of abstraction in the AH. The use of sensor-annotated AHs with a configural display is illustrated with a simple water reservoir example. We argue that if EID is to be effectively employed in the design of interfaces for complex systems, then the information needs of the human operator need to be considered at the earliest stages of system development while instrumentation requirements are being formulated. In this way, Rasmussen's AH promotes a formative approach to instrumentation engineering.     

Minimal instrumentation may compromise failure diagnosis with an ecological interface

Interfaces designed according to ecological interface design (EID) display higher-order relations and properties of a work domain so that adaptive operator problem solving can be better supported under unanticipated system conditions. Previous empirical studies of EID have assumed that the raw data required to derive and communicate higher-order information would be available and reliable. The present research examines the relative advantages of an EID interface over a conventional piping-and-instrumentation diagram (PID) when instrumentation is maximally or only minimally adequate. Results show an interaction between interface and the adequacy of the instrumentation. Failure diagnosis performance with the EID interface with maximally adequate instrumentation is best overall. Performance with the EID interface drops more drastically from maximally to minimally adequate instrumentation than does performance with the PID interface, to the point where the EID interface with minimally adequate instrumentation supports nonsignificantly worse performance than does the equivalent PID interface. Actual or potential applications of this research include design of instrumentation and displays for complex industrial processes.     

Ecological interface design for Pasteurizer II: a process description of semantic mapping

Ecological interface design (EID) is proving to be a promising approach to the design of interfaces for complex dynamic systems. Although the principles of EID and examples of its effective use are widely available, few readily available examples exist of how the individual displays that constitute an ecological interface are developed. This paper presents the semantic mapping process within EID in the context of prior theoretical work in this area. The semantic mapping process that was used in developing an ecological interface for the Pasteurizer II microworld is outlined, and the results of an evaluation of the ecological interface against a more conventional interface are briefly presented. Subjective reports indicate features of the ecological interface that made it particularly valuable for participants. Finally, we outline the steps of an analytic process for using EID. The findings presented here can be applied in the design of ecological interfaces or of configural displays for dynamic processes.     

Identifying problems and generating recommendations for enhancing complex systems: applying the abstraction hierarchy framework as an analytical tool

OBJECTIVE: This study adopts J. Rasmussen's (1985) abstraction hierarchy (AH) framework as an analytical tool to identify problems and pinpoint opportunities to enhance complex systems. BACKGROUND: The process of identifying problems and generating recommendations for complex systems using conventional methods is usually conducted based on incompletely defined work requirements. As the complexity of systems rises, the sheer mass of data generated from these methods becomes unwieldy to manage in a coherent, systematic form for analysis. There is little known work on adopting a broader perspective to fill these gaps. METHOD: AH was used to analyze an aircraft-automation system in order to further identify breakdowns in pilot-automation interactions. Four steps follow: developing an AH model for the system, mapping the data generated by various methods onto the AH, identifying problems based on the mapped data, and presenting recommendations. RESULTS: The breakdowns lay primarily with automation operations that were more goal directed. Identified root causes include incomplete knowledge content and ineffective knowledge structure in pilots' mental models, lack of effective higher-order functional domain information displayed in the interface, and lack of sufficient automation procedures for pilots to effectively cope with unfamiliar situations. CONCLUSION: The AH is a valuable analytical tool to systematically identify problems and suggest opportunities for enhancing complex systems. It helps further examine the automation awareness problems and identify improvement areas from a work domain perspective. APPLICATION: Applications include the identification of problems and generation of recommendations for complex systems as well as specific recommendations regarding pilot training, flight deck interfaces, and automation procedures.     

Ecological interface design: progress and challenges

Ecological interface design (EID) is a theoretical framework for designing human-computer interfaces for complex sociotechnical systems. Its primary aim is to support knowledge workers in adapting to change and novelty. This literature review shows that in situations requiring problem solving, EID improves performance when compared with current design approaches in industry. EID has been applied to industry-scale problems in a broad variety of application domains (e.g., process control, aviation, computer network management, software engineering, medicine, command and control, and information retrieval) and has consistently led to the identification of new information requirements. An experimental evaluation of EID using a full-fidelity simulator with professional workers has yet to be conducted, although some are planned. Several significant challenges remain as obstacles to the confident use of EID in industry. Promising paths for addressing these outstanding issues are identified. Actual or potential applications of this research include improving the safety and productivity of complex sociotechnical systems.     

Ecological interface design and computer network management: The effects of network size and fault frequency

This article describes an experiment investigating the impact of ecological interface design (EID) on human performance in computer network management. This work domain is more dynamic than those previously studied under EID because there is a constant potential for the addition and removal of devices, as well as changing configurations, making it important to study the generalizability of the framework. Two interfaces were created for the University of Toronto campus network consisting of 220 nodes: a P interface based on existing design practices which presented primarily physical information and a P+F interface based on EID which presented both physical and functional information identified by an abstraction hierarchy analysis. Participants used one of the two interfaces to detect and diagnose faults or disturbances in the simulated network in real-time. Network size and fault frequency were both manipulated as within-participants variables. The P+F interface led to faster detection times overall, as well as improved fault detection rate and more accurate fault diagnosis under higher fault loads. These results suggest that the EID framework may lead to more robust monitoring performance in computer network management compared to existing interfaces.     

Design of Information Content and Layout for Process Control Based on Goal–Means Domain Analysis

With regard to the design of information content in information display, it is often claimed that the abstraction hierarchy (AH) of the work domain should be considered as a basis for identifying and structuring the information content. The primary advantage of AH-based analysis and design is that functionally abstracted information can systematically be identified and provided to the operator, which has rarely been presented in traditional displays. This study evaluated the effectiveness of providing functional information, which was abstracted and represented based on goal–means analysis along the AH, to the operator in two task situations (fault diagnosis and operation). The results showed that the operator’s performance was improved with the high-level information, and the latter’s utility became greater when the goal–means relations between information at different abstraction levels were exhibited. From the results, three design principles for information display can be drawn. First, information should be identified and displayed at multiple abstraction levels. Second, the goal–means relations among the abstraction levels should be explicitly presented, especially for analytical cognitive tasks. Third, information layout should support information integration along decomposition structure within an abstraction level as well as along abstraction levels.     

Configural display design techniques considered at multiple levels of evaluation

Two studies were conducted to examine issues in the design and evaluation of configural displays. Four design techniques (bar graphs/extenders, scale markers/ scale grids, color coding/color layering/color separation, and annotation with digital values) were applied, alone and in combination, to a baseline configural display, forming 10 displays. Two qualitatively different evaluations assessed performance for (A) low-level data probes (quantitative estimates of individual variables) and (B) system control and fault detection tasks. Three of the four design techniques improved performance significantly for low-level data probes (color coding was the exception). A display with digital values only (i.e., no analog configural display) produced the poorest performance for control/fault detection tasks. When both levels of evaluation are considered, a composite display (configural display with all four techniques applied) was clearly the most effective. Overall, the findings obtained in the two experiments provide very limited evidence for the generalization of results between evaluations. The two levels of evaluation, the display manipulations, and the patterns of results are considered in terms of a cognitive systems engineering evaluation framework. General implications for the evaluation of displays and interfaces are discussed. Actual or potential applications include design techniques to improve graphical displays and methodological insights to focus and improve evaluation efforts.     

Designing for adaptation to novelty and change: functional information,emergent feature graphics,and higher-level control

Ecological interface design (EID) is a theoretical framework that aims to support worker adaptation to change and novelty in complex systems. Previous evaluations of EID have emphasized representativeness to enhance generalizability of results to operational settings. The research presented here is complementary, emphasizing experimental control to enhance theory building. Two experiments were conducted to test the impact of functional information and emergent feature graphics on adaptation to novelty and change in a thermal-hydraulic process control microworld. Presenting functional information in an interface using emergent features encouraged experienced participants to become perceptually coupled to the interface and thereby to exhibit higher-level control and more successful adaptation to unanticipated events. The absence of functional information or of emergent features generally led to lower-level control and less success at adaptation, the exception being a minority of participants who compensated by relying on analytical reasoning. These findings may have practical implications for shaping coordination in complex systems and fundamental implications for the development of a general unified theory of coordination for the technical, human, and social sciences. Actual or potential applications of this research include the design of human-computer interfaces that improve safety in complex sociotechnical systems.     

Accounting for memes in sociotechnical systems: extending the abstraction hierarchy to consider cognitive objects

Work domain analysis (WDA) is used to model the functional structure of sociotechnical systems (STS) through the abstraction hierarchy (AH). By identifying objects, processes, functions and measures that support system purposes, WDA reveals constraints within the system. Traditionally, the AH describes system elements at the lowest level of abstraction as physical objects. Multiple analyses of complex systems reveal that many include objects that exist only at a conceptual level. This paper argues that, by extending the AH to include cognitive objects, the analytical power of WDA is extended, and novel areas of application are enabled. Three case studies are used to demonstrate the role that cognitive objects play within STS. It is concluded that cognitive objects are a valid construct that offer a significant enhancement of WDA and enable its application to some of the world’s most pressing problems. Implications for future applications of WDA and the AH are discussed.

Practitioner summary: Some sociotechnical systems include memes as part of their functional structure. Three case studies were used to evaluate the utility of introducing cognitive objects alongside physical ones in work domain analysis, the first phase of cognitive work analysis. Including cognitive objects increases the scope and accuracy of work domain analysis.     

Display Design of Process Systems Based on Functional Modelling

The prevalent way to present information in industrial computer displays is by using piping and instrumentation diagrams. Such interfaces have sometimes resulted in difficulties for operators because they are not sufficient to fulfil their needs. A systematic way that supports interface design therefore has to be considered. In the new design framework, two questions must be answered. Firstly, a modelling method is required to describe a process system. Such a modelling method can define the information content that must be displayed in interfaces. Secondly, how to communicate this information to operators efficiently must be considered. This will provide a basis for determining the visual forms that the information should take. This study discusses interface design of human–machine systems from these two points of view. Based on other scholars’ work, a comprehensive set of functional primitives is summarised as a basis to build a functional model of process systems. A library of geometrical presentations for these primitives is then developed. To support effective interface design, the concept of ‘functional macro’ is introduced and a way to map functional model to interface display is illustrated by applying several principles. To make our ideas clear, a central heating system is taken as an example and its functional model is constructed. Based on the functional model, the information to be displayed is determined. Several functional macros are then found in the model and their corresponding displays are constructed. Finally, by using the library of geometrical presentations for functional primitives and functional macros, the display hierarchy of the central heating system is developed. Reusability of functional primitives makes it possible to use the methodology to support interface design of different process systems.     

Using multiple cognitive task analysis methods for supervisory control interface design in high-throughput biological screening processes

Cognitive task analysis (CTA) approaches are currently needed in many domains to provide explicit guidance on redesigning existing systems. This study used goal-directed task analysis (GDTA) along with abstraction hierarchy (AH) modeling to characterize the knowledge structure of biopharmacologists in planning, executing and analyzing the results of high-throughput organic compound screening operations, as well as the lab automation and equipment used in these operations. It was hypothesized that combining the results of the GDTA and AH models would provide a better understanding of complex system operator needs and how they may be addressed by existing technologies, as well as facilitate identification of automation and system interface design limitations. We used comparisons of the GDTA and AH models along with taxonomies of usability heuristics and types of automation in order to formulate interface design and automation functionality recommendations for existing software applications used in biological screening experiments. The proposed methodology yielded useful recommendations for improving custom supervisory control applications that led to prototypes of interface redesigns. The approach was validated through an expert usability evaluation of the redesigns and was shown to be applicable to the life sciences domain.     

Putting it all together: improving display integration in ecological displays

Computer displays are being designed for increasingly larger industrial systems. As the application domain scales up, maintaining integration across different kinds of views becomes more challenging. This paper presents the results of a study of three different approaches to integration based on the spatial and temporal proximity of related information objects. The domain used for evaluation was a simulation of an industry-scale conventional power plant. All three displays were ecological displays developed using an abstraction hierarchy analysis. Views were integrated in a high-space/low-time, low-space/high-time, and high-space/high-time integration of means-end related objects. During a fault detection and diagnosis task, it was found that a low level of integration, high-space/ low-time, provided the fastest fault detection time. However, the most integrated condition, high-space/high-time, resulted in the fastest and most accurate fault diagnosis performance. Actual or potential applications of this research include computer displays for large-scale systems such as network management or process control, for which problem solving is critical and integration must be maintained.     

A longitudinal study of the effects of ecological interface design on deep knowledge

Some researchers have argued that providing operators with externalized, graphic representations can lead to a trade-off whereby deep knowledge is sacrificed for cognitive economy and performance. This article provides an initial empirical investigation of this hypothesis by presenting a longitudinal study of the effect of ecological interface design (EID), a framework for designing interfaces for complex industrial systems, on subjects' deep knowledge. The experiment continuously observed the quasi-daily performance of the subjects' over a period of six months. The research was conducted in the context of DURESS II, a real-time, interactive thermal-hydraulic process control simulation that was designed to be representative of industrial systems. The performance of two interfaces was compared, an EID interface based on physical and functional (P+F) system representations and a more traditional interface based solely on a physical (P) representation. Subjects were required to perform several control tasks, including startup, tuning, shutdown and fault management. Occasionally, a set of knowledge elicitation tests was administered to assess the evolution of subjects' deep knowledge of DURESS II. The results suggest that EID can lead to a functionally organized knowledge base as well as superior performance, but only if subjects actively reflect on the feedback they get from the interface. In contrast, if subjects adopt a surface approach to learning, then EID can lead to a shallow knowledge base and poor performance, although no worse than that observed with a traditional interface.     

Distributed information systems: an advanced methodology

Information systems ranging over wide areas show properties that must be carefully analyzed and designed in order to meet the needs of the customers. Thus the development of such information systems is to be guided by software engineering methods that address problems like distribution of data and processes, communication aspects and fault tolerance. This paper shows the basic modeling concepts and the development process employed by the BOS Engineering Method to meet these requirements. The BOS Engineering Method applies the concept of business transactions to specify behavior in the early analysis phase. Appropriate abstraction levels are defined to reduce the complexity of specifying distribution issues. The development of complex distributed information systems needs a rigorous life cycle model. The BOS Engineering Method relaxes the waterfall life cycle model to allow controlled look ahead and feedback up and down the abstraction levels     

Designing for attention with sound: challenges and extensions to ecological interface design

OBJECTIVE: We explore whether ecological interface design (EID) principles can be applied to the design of an auditory display for anesthesia monitoring. BACKGROUND: EID examples focus almost exclusively on visual displays. In the anesthesia work environment, however, auditory displays may provide better individual and team awareness of patient state. METHOD: Using a work domain analysis of physiological monitoring in anesthesia, we identify information to display. Using the skills, rules, and knowledge distinction we identify cognitive control needed. Using semantic mapping we map physiological variables and constraints to auditory dimensions. RESULTS: EID principles do not address when information should be displayed and to whom. An attentional mapping stage helps to specify answers to these questions so that a workable auditory display for anesthesia monitoring is achieved. CONCLUSION: EID principles of representing work domain functional structure and minimizing resource-demanding cognitive control are necessary but insufficient to specify requirements for an effective auditory display. Also needed are analyses of control tasks, strategies, and the social organization of work. Such analyses are an integral part of the broader cognitive work analysis framework from which EID emerged. APPLICATION: Actual or potential uses of this research include the design of displays that support continuous peripheral awareness in collaborative multimodal work environments.     

Object‐oriented design patterns for debugging heterogeneous languages and virtual machines

Debugging multi‐language software systems requires examining and executing these systems at multiple levels of abstraction. Embedded systems, for example, often comprise a mix of assembly language device drivers and C language control code. Embedded systems increasingly utilize Java to support dynamic loading and run‐time reconfiguration. The RTEEM (Research version of the Tcl Environment for Extensible Modeling) debugger employs three design patterns in solving the problems of multi‐language embedded system debugging. The Reflective virtual machine (VM) pattern models a language‐neutral virtual machine abstraction, with language‐specific interfaces extending this abstraction. Reflection allows a debugger to inspect and control a target VM. The Chain of Responsibility is a classic pattern used to arrange language‐specific debugger command interpreters in a delegation chain. All interpreters share a single command syntax, but each interpreter adapts commands to its language abstraction by interacting with its language‐specific VM view. Composite is another classic pattern, used to combine objects into tree structures. RTEEM employs it to aggregate VM debugger chains into a hierarchy that supports uniform command syntax for debugging threads, processes, multiprocessor systems, and compositions of these entities. This paper illustrates how combining two classic design patterns with the VM abstraction as a pattern results in an architecture that is powerful and flexible in adapting to the debugging needs of heterogeneous, distributed embedded systems. Copyright © 2004 John Wiley & Sons, Ltd.     

An empirical comparison of alternative methodologies for the evaluation of configural displays

Two different methodologies (visual, memory) were used to evaluate alternative versions of the same configural display. One version (composite display) had several graphical design techniques applied, whereas the other version (baseline display) did not. Two types of information probes (high-level property, low-level data) were administered. When the displays were visible during completion of the probes (visual methodology), the display manipulation had the largest impact on performance (composite display associated with better performance); when the displays were not visible (memory methodology) the probe manipulation had the largest impact on performance (high-level probes associated with better performance). These results are interpreted in light of the mutually interacting constraints introduced by factors in display design, task requirements, and the participants' cognitive and perceptual capabilities/limitations. Implications for both the design and the evaluation of displays and interfaces in general are discussed. Actual or potential applications of this research include design techniques for improving the quality of graphic displays and methodological insights for interpreting previous research and guiding future experimentation.     

The impending demise of the file system

The file system has been a trusted part of most computers for many years, and will likely continue as such in operating systems for many more. However, several emerging trends in user interfaces indicate that the basic file-system model is inadequate to fully satisfy the needs of new users, despite the flexibility of the underlying code and data structures. There is no need for users to know how their information is stored inside the guts of the computer. Indeed, the notion of a continuous file is itself is an abstraction: It masks the fact that the information is normally stored on noncontiguous sectors of a hard disk. From a user perspective, current file systems are based on three assumptions: Information is partitioned into coherent and disjunct units, each of which is treated as a separate object (file). Users typically manipulate information using a file and are restricted to be “in” one file at a time. Information objects are classified according to a single hierarchy: the subdirectory structure. Each information object is given a single, semi-unique name, which is fixed. This file name is the main way users access information inside the object. Window systems have made these assumptions less intolerable, but they still exist. Modern computing, particularly the Internet, is further undermining these assumptions in several ways     

Hierarchical control system design using approximate simulation

In this paper, we present a new approach for hierarchical control based on the recent notions of approximate simulation and simulation functions, a quantitative version of the simulation relations. Given a complex system that needs to be controlled and a simpler abstraction, we show how the knowledge of a simulation function allows us to synthesize hierarchical control laws by first controlling the abstraction and then lifting the abstract control law to the complex system using an interface. For the class of linear control systems, we give an effective characterization of the simulation functions and of the associated interfaces. This characterization allows us to use algorithmic procedures for their computation. We show how to choose an abstraction for a linear control system such that our hierarchical control approach can be used. Finally, we show the effectiveness of our approach on an example.