Work domain analysis and sensors I: principles and simple example

In this paper we establish a foundation for understanding the instrumentation needs of complex dynamic systems if ecological interface design (EID)-based interfaces are to be robust in the face of instrumentation failures. EID-based interfaces often include configural displays which reveal the higher-order properties of complex systems. However, concerns have been expressed that such displays might be misleading when instrumentation is unreliable or unavailable. Rasmussen's abstraction hierarchy (AH) formalism can be extended to include representations of sensors near the functions or properties about which they provide information, resulting in what we call a “sensor-annotated abstraction hierarchy”. Sensor-annotated AHs help the analyst determine the impact of different instrumentation engineering policies on higher-order system information by showing how the data provided from individual sensors propagates within and across levels of abstraction in the AH. The use of sensor-annotated AHs with a configural display is illustrated with a simple water reservoir example. We argue that if EID is to be effectively employed in the design of interfaces for complex systems, then the information needs of the human operator need to be considered at the earliest stages of system development while instrumentation requirements are being formulated. In this way, Rasmussen's AH promotes a formative approach to instrumentation engineering.     

Minimal instrumentation may compromise failure diagnosis with an ecological interface

Interfaces designed according to ecological interface design (EID) display higher-order relations and properties of a work domain so that adaptive operator problem solving can be better supported under unanticipated system conditions. Previous empirical studies of EID have assumed that the raw data required to derive and communicate higher-order information would be available and reliable. The present research examines the relative advantages of an EID interface over a conventional piping-and-instrumentation diagram (PID) when instrumentation is maximally or only minimally adequate. Results show an interaction between interface and the adequacy of the instrumentation. Failure diagnosis performance with the EID interface with maximally adequate instrumentation is best overall. Performance with the EID interface drops more drastically from maximally to minimally adequate instrumentation than does performance with the PID interface, to the point where the EID interface with minimally adequate instrumentation supports nonsignificantly worse performance than does the equivalent PID interface. Actual or potential applications of this research include design of instrumentation and displays for complex industrial processes.     

Ecological interface design for Pasteurizer II: a process description of semantic mapping

Ecological interface design (EID) is proving to be a promising approach to the design of interfaces for complex dynamic systems. Although the principles of EID and examples of its effective use are widely available, few readily available examples exist of how the individual displays that constitute an ecological interface are developed. This paper presents the semantic mapping process within EID in the context of prior theoretical work in this area. The semantic mapping process that was used in developing an ecological interface for the Pasteurizer II microworld is outlined, and the results of an evaluation of the ecological interface against a more conventional interface are briefly presented. Subjective reports indicate features of the ecological interface that made it particularly valuable for participants. Finally, we outline the steps of an analytic process for using EID. The findings presented here can be applied in the design of ecological interfaces or of configural displays for dynamic processes.     

Identifying problems and generating recommendations for enhancing complex systems: applying the abstraction hierarchy framework as an analytical tool

OBJECTIVE: This study adopts J. Rasmussen's (1985) abstraction hierarchy (AH) framework as an analytical tool to identify problems and pinpoint opportunities to enhance complex systems. BACKGROUND: The process of identifying problems and generating recommendations for complex systems using conventional methods is usually conducted based on incompletely defined work requirements. As the complexity of systems rises, the sheer mass of data generated from these methods becomes unwieldy to manage in a coherent, systematic form for analysis. There is little known work on adopting a broader perspective to fill these gaps. METHOD: AH was used to analyze an aircraft-automation system in order to further identify breakdowns in pilot-automation interactions. Four steps follow: developing an AH model for the system, mapping the data generated by various methods onto the AH, identifying problems based on the mapped data, and presenting recommendations. RESULTS: The breakdowns lay primarily with automation operations that were more goal directed. Identified root causes include incomplete knowledge content and ineffective knowledge structure in pilots' mental models, lack of effective higher-order functional domain information displayed in the interface, and lack of sufficient automation procedures for pilots to effectively cope with unfamiliar situations. CONCLUSION: The AH is a valuable analytical tool to systematically identify problems and suggest opportunities for enhancing complex systems. It helps further examine the automation awareness problems and identify improvement areas from a work domain perspective. APPLICATION: Applications include the identification of problems and generation of recommendations for complex systems as well as specific recommendations regarding pilot training, flight deck interfaces, and automation procedures.     

Ecological interface design: progress and challenges

Ecological interface design (EID) is a theoretical framework for designing human-computer interfaces for complex sociotechnical systems. Its primary aim is to support knowledge workers in adapting to change and novelty. This literature review shows that in situations requiring problem solving, EID improves performance when compared with current design approaches in industry. EID has been applied to industry-scale problems in a broad variety of application domains (e.g., process control, aviation, computer network management, software engineering, medicine, command and control, and information retrieval) and has consistently led to the identification of new information requirements. An experimental evaluation of EID using a full-fidelity simulator with professional workers has yet to be conducted, although some are planned. Several significant challenges remain as obstacles to the confident use of EID in industry. Promising paths for addressing these outstanding issues are identified. Actual or potential applications of this research include improving the safety and productivity of complex sociotechnical systems.     

Methods for parallel computation of complex flow problems

This paper is an overview of some of the methods developed by the Team for Advanced Flow Simulation and Modeling (TAFSM) [http://www.mems.rice.edu/TAFSM/] to support flow simulation and modeling in a number of “Targeted Challenges”. The “Targeted Challenges” include unsteady flows with interfaces, fluid–object and fluid–structure interactions, airdrop systems, and air circulation and contaminant dispersion. The methods developed include special numerical stabilization methods for compressible and incompressible flows, methods for moving boundaries and interfaces, advanced mesh management methods, and multi-domain computational methods. We include in this paper a number of numerical examples from the simulation of complex flow problems.     

Configural display design techniques considered at multiple levels of evaluation

Two studies were conducted to examine issues in the design and evaluation of configural displays. Four design techniques (bar graphs/extenders, scale markers/ scale grids, color coding/color layering/color separation, and annotation with digital values) were applied, alone and in combination, to a baseline configural display, forming 10 displays. Two qualitatively different evaluations assessed performance for (A) low-level data probes (quantitative estimates of individual variables) and (B) system control and fault detection tasks. Three of the four design techniques improved performance significantly for low-level data probes (color coding was the exception). A display with digital values only (i.e., no analog configural display) produced the poorest performance for control/fault detection tasks. When both levels of evaluation are considered, a composite display (configural display with all four techniques applied) was clearly the most effective. Overall, the findings obtained in the two experiments provide very limited evidence for the generalization of results between evaluations. The two levels of evaluation, the display manipulations, and the patterns of results are considered in terms of a cognitive systems engineering evaluation framework. General implications for the evaluation of displays and interfaces are discussed. Actual or potential applications include design techniques to improve graphical displays and methodological insights to focus and improve evaluation efforts.     

Formal Specification and Enactment of Security Policies through Agent Coordination Contexts

Our research moves from three fundamental considerations that concern the modelling and engineering of complex systems. First, organization, coordination and security are strictly related issues that should be modelled in a uniform and coherent framework. Second, models, technologies and methodologies should come hand in hand, so that abstractions used in the analysis and design stages should be still “alive and kicking” at development and execution time. Third, the general non-formalisability of complex systems should not prevent us from using formal tools whenever useful, such as in proving or ensuring properties of limited but meaningful portions of a system.By focussing on multi-agent systems, we discuss the notion of Agent Coordination Context (ACC) as an abstraction that (i) works as an organization and security abstraction, (ii) integrates well with abstractions provided by coordination infrastructures, and (iii) covers the engineering process from design to deployment. In particular, in this paper we study the syntax and semantics of a language for ACCs specification, exploiting typical process algebra techniques. Accordingly, we show that process algebras are a suitable tool for both specification and enactment of security and coordination policies through ACCs.     

Designing for adaptation to novelty and change: functional information,emergent feature graphics,and higher-level control

Ecological interface design (EID) is a theoretical framework that aims to support worker adaptation to change and novelty in complex systems. Previous evaluations of EID have emphasized representativeness to enhance generalizability of results to operational settings. The research presented here is complementary, emphasizing experimental control to enhance theory building. Two experiments were conducted to test the impact of functional information and emergent feature graphics on adaptation to novelty and change in a thermal-hydraulic process control microworld. Presenting functional information in an interface using emergent features encouraged experienced participants to become perceptually coupled to the interface and thereby to exhibit higher-level control and more successful adaptation to unanticipated events. The absence of functional information or of emergent features generally led to lower-level control and less success at adaptation, the exception being a minority of participants who compensated by relying on analytical reasoning. These findings may have practical implications for shaping coordination in complex systems and fundamental implications for the development of a general unified theory of coordination for the technical, human, and social sciences. Actual or potential applications of this research include the design of human-computer interfaces that improve safety in complex sociotechnical systems.     

Ecological interface design and computer network management: The effects of network size and fault frequency

This article describes an experiment investigating the impact of ecological interface design (EID) on human performance in computer network management. This work domain is more dynamic than those previously studied under EID because there is a constant potential for the addition and removal of devices, as well as changing configurations, making it important to study the generalizability of the framework. Two interfaces were created for the University of Toronto campus network consisting of 220 nodes: a P interface based on existing design practices which presented primarily physical information and a P+F interface based on EID which presented both physical and functional information identified by an abstraction hierarchy analysis. Participants used one of the two interfaces to detect and diagnose faults or disturbances in the simulated network in real-time. Network size and fault frequency were both manipulated as within-participants variables. The P+F interface led to faster detection times overall, as well as improved fault detection rate and more accurate fault diagnosis under higher fault loads. These results suggest that the EID framework may lead to more robust monitoring performance in computer network management compared to existing interfaces.     

Design of Information Content and Layout for Process Control Based on Goal–Means Domain Analysis

With regard to the design of information content in information display, it is often claimed that the abstraction hierarchy (AH) of the work domain should be considered as a basis for identifying and structuring the information content. The primary advantage of AH-based analysis and design is that functionally abstracted information can systematically be identified and provided to the operator, which has rarely been presented in traditional displays. This study evaluated the effectiveness of providing functional information, which was abstracted and represented based on goal–means analysis along the AH, to the operator in two task situations (fault diagnosis and operation). The results showed that the operator’s performance was improved with the high-level information, and the latter’s utility became greater when the goal–means relations between information at different abstraction levels were exhibited. From the results, three design principles for information display can be drawn. First, information should be identified and displayed at multiple abstraction levels. Second, the goal–means relations among the abstraction levels should be explicitly presented, especially for analytical cognitive tasks. Third, information layout should support information integration along decomposition structure within an abstraction level as well as along abstraction levels.     

Accounting for memes in sociotechnical systems: extending the abstraction hierarchy to consider cognitive objects

Work domain analysis (WDA) is used to model the functional structure of sociotechnical systems (STS) through the abstraction hierarchy (AH). By identifying objects, processes, functions and measures that support system purposes, WDA reveals constraints within the system. Traditionally, the AH describes system elements at the lowest level of abstraction as physical objects. Multiple analyses of complex systems reveal that many include objects that exist only at a conceptual level. This paper argues that, by extending the AH to include cognitive objects, the analytical power of WDA is extended, and novel areas of application are enabled. Three case studies are used to demonstrate the role that cognitive objects play within STS. It is concluded that cognitive objects are a valid construct that offer a significant enhancement of WDA and enable its application to some of the world’s most pressing problems. Implications for future applications of WDA and the AH are discussed.

Practitioner summary: Some sociotechnical systems include memes as part of their functional structure. Three case studies were used to evaluate the utility of introducing cognitive objects alongside physical ones in work domain analysis, the first phase of cognitive work analysis. Including cognitive objects increases the scope and accuracy of work domain analysis.     

Distributed information systems: an advanced methodology

Information systems ranging over wide areas show properties that must be carefully analyzed and designed in order to meet the needs of the customers. Thus the development of such information systems is to be guided by software engineering methods that address problems like distribution of data and processes, communication aspects and fault tolerance. This paper shows the basic modeling concepts and the development process employed by the BOS Engineering Method to meet these requirements. The BOS Engineering Method applies the concept of business transactions to specify behavior in the early analysis phase. Appropriate abstraction levels are defined to reduce the complexity of specifying distribution issues. The development of complex distributed information systems needs a rigorous life cycle model. The BOS Engineering Method relaxes the waterfall life cycle model to allow controlled look ahead and feedback up and down the abstraction levels     

Display Design of Process Systems Based on Functional Modelling

The prevalent way to present information in industrial computer displays is by using piping and instrumentation diagrams. Such interfaces have sometimes resulted in difficulties for operators because they are not sufficient to fulfil their needs. A systematic way that supports interface design therefore has to be considered. In the new design framework, two questions must be answered. Firstly, a modelling method is required to describe a process system. Such a modelling method can define the information content that must be displayed in interfaces. Secondly, how to communicate this information to operators efficiently must be considered. This will provide a basis for determining the visual forms that the information should take. This study discusses interface design of human–machine systems from these two points of view. Based on other scholars’ work, a comprehensive set of functional primitives is summarised as a basis to build a functional model of process systems. A library of geometrical presentations for these primitives is then developed. To support effective interface design, the concept of ‘functional macro’ is introduced and a way to map functional model to interface display is illustrated by applying several principles. To make our ideas clear, a central heating system is taken as an example and its functional model is constructed. Based on the functional model, the information to be displayed is determined. Several functional macros are then found in the model and their corresponding displays are constructed. Finally, by using the library of geometrical presentations for functional primitives and functional macros, the display hierarchy of the central heating system is developed. Reusability of functional primitives makes it possible to use the methodology to support interface design of different process systems.     

Designing for attention with sound: challenges and extensions to ecological interface design

OBJECTIVE: We explore whether ecological interface design (EID) principles can be applied to the design of an auditory display for anesthesia monitoring. BACKGROUND: EID examples focus almost exclusively on visual displays. In the anesthesia work environment, however, auditory displays may provide better individual and team awareness of patient state. METHOD: Using a work domain analysis of physiological monitoring in anesthesia, we identify information to display. Using the skills, rules, and knowledge distinction we identify cognitive control needed. Using semantic mapping we map physiological variables and constraints to auditory dimensions. RESULTS: EID principles do not address when information should be displayed and to whom. An attentional mapping stage helps to specify answers to these questions so that a workable auditory display for anesthesia monitoring is achieved. CONCLUSION: EID principles of representing work domain functional structure and minimizing resource-demanding cognitive control are necessary but insufficient to specify requirements for an effective auditory display. Also needed are analyses of control tasks, strategies, and the social organization of work. Such analyses are an integral part of the broader cognitive work analysis framework from which EID emerged. APPLICATION: Actual or potential uses of this research include the design of displays that support continuous peripheral awareness in collaborative multimodal work environments.     

On interfaces

The nature of interfaces is discussed. It is recommended that modern software approaches to interfaces (such as abstract data types) be extended to interfaces of all kinds. These interfaces can and should be developed at the level of abstraction appropriate to the interacting elements, rather than at the level of their lowest common denominator, which is frequently the case. It is proposed that for an interface to exist at any level of abstraction, there must be an operational framework that implements that level of abstraction and within which the interfacing elements interact. Four tasks required for the development of interfaces are described. The traditional classification of interfaces as hardware/software, software/software, and hardware/hardware is found wanting and a new partitioning is suggested—structural or interpretive interfaces, symbolic interfaces, and physical interfaces. Standards and scientific and engineering theories are shown to define levels of abstraction for general interfaces in much the same way that programming languages do for software interfaces. Implications for software engineering and concurrent engineering are explored.     

Hierarchical control system design using approximate simulation

In this paper, we present a new approach for hierarchical control based on the recent notions of approximate simulation and simulation functions, a quantitative version of the simulation relations. Given a complex system that needs to be controlled and a simpler abstraction, we show how the knowledge of a simulation function allows us to synthesize hierarchical control laws by first controlling the abstraction and then lifting the abstract control law to the complex system using an interface. For the class of linear control systems, we give an effective characterization of the simulation functions and of the associated interfaces. This characterization allows us to use algorithmic procedures for their computation. We show how to choose an abstraction for a linear control system such that our hierarchical control approach can be used. Finally, we show the effectiveness of our approach on an example.     

An empirical comparison of alternative methodologies for the evaluation of configural displays

Two different methodologies (visual, memory) were used to evaluate alternative versions of the same configural display. One version (composite display) had several graphical design techniques applied, whereas the other version (baseline display) did not. Two types of information probes (high-level property, low-level data) were administered. When the displays were visible during completion of the probes (visual methodology), the display manipulation had the largest impact on performance (composite display associated with better performance); when the displays were not visible (memory methodology) the probe manipulation had the largest impact on performance (high-level probes associated with better performance). These results are interpreted in light of the mutually interacting constraints introduced by factors in display design, task requirements, and the participants' cognitive and perceptual capabilities/limitations. Implications for both the design and the evaluation of displays and interfaces in general are discussed. Actual or potential applications of this research include design techniques for improving the quality of graphic displays and methodological insights for interpreting previous research and guiding future experimentation.     

An evaluation of a "time tunnel" display format for the presentation of temporal information

The time tunnel display design technique combines the benefits of configural displays (salient visual properties corresponding to critical domain semantics) with the benefits of temporal information (i.e., the value of variables and properties over time). In Experiment 1 a baseline configural display and a time tunnel display were evaluated using real-time measures of system control, fault detection, and state estimation in a simulated process control task. The results provided little evidence in support of the time tunnel format. In Experiment 2 access to the temporal context was limited: Participants performed the detection and estimation tasks with static "snapshots" of system states that had been generated in Experiment 1. The overall pattern of results indicates that the time tunnel display was more effective for state estimation tasks than was the baseline configural display and or a trend display. Issues in the design of temporal displays are discussed, including representational formats and the choice of temporal time frames. Issues in the evaluation of temporal displays are also discussed, including the role of temporal information and the critical nature of participants' access to this information. Actual or potential applications of this research include design techniques for improving graphical displays and methodological insights to guide future evaluations.     

Perceptual control and layered protocols in interface design: II. The general protocol grammar

Perceptual control theory (PCT) is a framework theory for psychology, based on the tenet “All behaviour is the control of perception.” Layered protocol theory (LPT) is PCT applied to the special case of communication between cooperating partners, each controlling their own perceptions and many levels of abstraction within a dialogue. This paper discusses some perceptual control processes that occur within a single dialogue level, in the form of a General Protocol Grammar that is asserted to be valid for every level of every dialogue. A companion paper is concerned with LPT applied to the design and analysis of human–machine interfaces.