Using cognitive work analysis to explore system flexibility

The presented work describes a structured approach for identifying flexible working practices in complex socio-technical systems; further, it presents a framework for the elicitation of design changes capable of increasing system adaptability. The proposed approach uses the first three phases of the cognitive work analysis framework: work domain analysis; control task analysis; and strategies analysis. Functions are extracted from an abstraction hierarchy; these are then explored in a number of different situations using a contextual activity template. These functions are then explored in greater detail with the strategies analysis phase. The contextual activity template is used to identify situations where functions are unable to perform; in order to increase the flexibility of the system, new strategies are then proposed and represented in strategies analyses flow diagrams. To communicate the approach, it is introduced using the familiar, domestic domain of an ‘Apple iPod’. The paper presents a structured approach based upon cognitive work analysis for exploring system flexibility. The case study presented, an Apple iPod, shows how design changes to the system can be informed by this approach. This paper takes an existing approach and clarifies the link between analysis and design.     

Advancing the practice of cognitive task analysis: a call for taxonomic research

Cognitive task analysis (CTA) captures unobservable cognitive processes, decisions and judgments of expert performance. Over 100 different CTA methods are identified in prior literature. However, existing classifications typically sort techniques by process rather than outcome, application or causal mechanism. Therefore, techniques can be misapplied and comparative analysis of methods made difficult. Based on the frequency distribution of CTA methods in 1065 studies, a subsample representing 60% of the most frequently published methods was coded based on elicitation and analysis techniques. Consistency of resulting applications was assessed. Inconsistent matching of CTA methods and subsequent applications indicate CTA is currently more craft than technology. Therefore, there is no robust basis for selecting one method over another for research or practice. Specific challenges include comparing the reliability and validity of individual methods and optimising selection of techniques for intended applications. Developing a causal taxonomy may facilitate such advancements.     

Assessing the ‘system’ in safe systems-based road designs: Using cognitive work analysis to evaluate intersection designs

While a safe systems approach has long been acknowledged as the underlying philosophy of contemporary road safety strategies, systemic applications are sparse. This article argues that systems-based methods from the discipline of Ergonomics have a key role to play in road transport design and evaluation. To demonstrate, the Cognitive Work Analysis framework was used to evaluate two road designs – a traditional Melbourne intersection and a cut-through design for future intersections based on road safety safe systems principles. The results demonstrate that, although the cut-through intersection appears different in layout from the traditional intersection, system constraints are not markedly different. Furthermore, the analyses demonstrated that redistribution of constraints in the cut-through intersection resulted in emergent behaviour, which was not anticipated and could prove problematic. Further, based on the lack of understanding of emergent behaviour, similar design induced problems are apparent across both intersections. Specifically, incompatibilities between infrastructure, vehicles and different road users were not dealt with by the proposed design changes. The importance of applying systems methods in the design and evaluation of road transport systems is discussed.     

A formative approach to the strategies analysis phase of cognitive work analysis

Strategies Analysis, the third phase of Cognitive Work Analysis, helps investigators consider the range of ways in which workers can perform control tasks. Most existing approaches to Strategies Analysis identify a limited number of domain-specific strategies. We present a two-phase formative Strategies Analysis method intended to expose the range of strategies possible within a work system and the likelihood that different types of strategies will be selected in different contexts. The first phase, the preparatory phase, identifies generalised constraints that affect the range and selection of strategies, and the categories of strategies that may be applied to any domain. In the second phase, the application phase, investigators use the outputs of the preparatory phase to explore the impact that different work situations, tasks and workers have on the categories of strategies most likely to be adopted.     

Military (in)decision-making process: a psychological framework to examine decision inertia in military operations

Abstract

When understanding how members of the armed forces make decisions in war current military doctrine centers on the military decision-making process (MDMP) – a linear process of identifying, evaluating and choosing the best course of action, while wider theoretical contributions focus on recognition prime models (RPD) of decision-making. In this article, we argue that the SAFE-T model of critical incident decision-making can elucidate the process of decision-making during military operations. The SAFE-T model states that effective decision-making follows a sequential process of situation assessment (SA), plan formulation (F) and plan execution (E) phases, and team learning (T). The central innovation of the SAFE-T model; however, is that it highlights the different ways in which decision-making can de-rail from this optimal strategy, resulting in decision inertia. This article discusses the implications of employing the SAFE-T model as a framework to study military decision-making both in the lab and in the field.     

Hierarchical task analysis vs. cognitive work analysis: comparison of theory,methodology and contribution to system design

The cognitive work analysis framework continues to attract increasing attention from the human factors and ergonomics community. Conversely, hierarchical task analysis has been, and remains, the most popular of all human factors and ergonomics methods. This article compares the two approaches in terms of their theoretical underpinning, methodological approach and potential contributions to system design and evaluation. To do this, recent analyses, involving both approaches, of a military rotary wing mission planning software tool are compared and contrasted in terms of their methodological procedure and analysis outputs. The findings indicate that, despite the very different theoretical and methodological nature of the two approaches, and also the entirely different analyses derived, the two methods provide highly complementary outputs. In conclusion, it is argued that there is benefit in applying both approaches to inform the design and/or evaluation of the same product or system.     

Safe places for pedestrians: Using cognitive work analysis to consider the relationships between the engineering and urban design of footpaths

Footpaths provide an integral component of our urban environments and have the potential to act as safe places for people and the focus for community life. Despite this, the approach to designing footpaths that are safe while providing this sense of place often occurs in silos. There is often very little consideration given to how designing for sense of place impacts safety and vice versa. The aim of this study was to use a systems analysis and design framework to develop a design template for an ‘ideal’ footpath system that embodies both safety and sense of place. This was achieved through using the first phase of the Cognitive Work Analysis framework, Work Domain Analysis, to specify a model of footpaths as safe places for pedestrians. This model was subsequently used to assess two existing footpath environments to determine the extent to which they meet the design requirements specified. The findings show instances where the existing footpaths both meet and fail to meet the design requirements specified. Through utilising a systems approach for footpaths, this paper has provided a novel design template that can inform new footpath design efforts or be used to evaluate the extent to which existing footpaths achieve their safety and sense of place requirements.     

A systems analysis approach to solving office work system health and performance problems

This paper describes a systems analysis approach to human performance in office work systems. The approach, integrating both micro- and macroergonomic aspects, provides a process for more comprehensive, systematic solutions. This systems approach is designed to assess office workers' performance and effectiveness problems within technology intensive office work environments and provide realistic solutions for improving performance. The approach incorporates micro- and macroergonomic factors to adequately address the performance and stress and health-related problems associated with modern office work systems. The seven step approach consists of: defining the problems; setting the objectives and developing alternatives; modelling alternatives; evaluating alternatives; selecting an alternative; planning for implementation; and evaluation, feedback and modification. A detailed schematic presentation of these steps is provided. Solutions or alternatives are proposed to minimize the identified problem factors and to improve performance and the quality of work life.     

Applying cognitive work analysis to the design of rapidly reconfigurable interfaces in complex networks

The objective of this paper is to illustrate the interconnections between the different phases (or tools) within the cognitive work analysis framework; the benefits of extending an analysis across each of the five phases are highlighted through these interconnections. The paper uses a command and control micro-world example to describe how each of the five phases can be used to describe the constraints within the micro-world domain from a different perspective. Based upon the social organisation and cooperation analysis, design requirements are extracted in order to develop role specific customisable interfaces for use within the micro-world. The interfaces have been specifically developed to communicate real time reconfiguration of the network through each of the individual interfaces; the reallocations of functions or roles are communicated to the actors through changes to the interface.     

Eliciting strategies in revolutionary design: exploring the hypothesis of predefined strategy categories

Introducing automation in a human-machine system changes the tasks performed by human operators. It is difficult to analyse systems for which there are no experienced operators. This issue emerged within a project with the aim to develop a human–machine interface for a highly automated long-haul vehicle. To handle the problem, a formative strategies analysis method with promises to enable desktop analyses through predefined strategy categories was adopted. The method was used to investigate strategies for controlling the future long haul vehicle by conducting workshops with today's drivers. The method was shown to be a valuable asset in eliciting strategies for revolutionary design.     

What to do next: Using problem status to determine the course of action

Formal decision support tools are little used in engineering design. This paper explores the reasons for this and presents a method which is tailored to problems characterized by teams of stakeholders with inconsistent views who generate multiple alternatives and criteria, and who work to reach consensus. This method is especially designed to support activity when much of the information is qualitative, immature, and there is a diversity of views. The methodology assists the team in determining which alternative attributes to invest time in refining in their effort to reach consensus. The underlying mathematical structure (a Bayesian model of multi-attribute team decision making) is presented. This model supports team member belief about an alternative's ability to meet a criteria on two dimensions, knowledge and confidence. The methodology forces recording the rationale used to reach the final decision. A running example is used to explain the details.This research has been supported by the National Science Foundation under grant DDM-931996. The opinions in this paper are the authors' and do not reflect the position of the NSF or Oregon State University.     

Reprint of “Safe places for pedestrians: Using cognitive work analysis to consider the relationships between the engineering and urban design of footpaths”

Footpaths provide an integral component of our urban environments and have the potential to act as safe places for people and the focus for community life. Despite this, the approach to designing footpaths that are safe while providing this sense of place often occurs in silos. There is often very little consideration given to how designing for sense of place impacts safety and vice versa. The aim of this study was to use a systems analysis and design framework to develop a design template for an ‘ideal’ footpath system that embodies both safety and sense of place. This was achieved through using the first phase of the Cognitive Work Analysis framework, Work Domain Analysis, to specify a model of footpaths as safe places for pedestrians. This model was subsequently used to assess two existing footpath environments to determine the extent to which they meet the design requirements specified. The findings show instances where the existing footpaths both meet and fail to meet the design requirements specified. Through utilising a systems approach for footpaths, this paper has provided a novel design template that can inform new footpath design efforts or be used to evaluate the extent to which existing footpaths achieve their safety and sense of place requirements.     

How the psychological theory of action identification can offer new advances for research in cognitive engineering

We present the psychological theory of action identification as a framework for a more in-depth understanding of a human operator's cognitive activity in the scope of cognitive engineering. A comparison of theoretical models and findings shows that both frameworks are founded on a similar theory of cognitive control based on an ontological viewpoint of means–ends relationships with the proposal that an individual mentally ‘navigates’ or ‘moves’ through a hierarchical arrangement of these relationships. However, whilst cognitive engineering begins the analysis from a viewpoint on affordances coming from the external work domain, the action identification theory starts from a viewpoint on action identities internally attributed to actions by individuals. We show that the conceptual articulation of these two approaches leads to confirming qualitative findings on an agent's cognitive activity and to proposing general cognitive principles that would explain a single agent's mental navigation through the abstraction hierarchy.     

Measurement for evaluating the learnability and resilience of methods of cognitive work

Some experiments on human–computer interaction are aimed at evaluating hypotheses concerning cognitive work. Other experiments are intended to evaluate the software tools that shape the cognitive work. In both cases, effective experimentation is premised on the control and factorial analysis of sources of variability. This entails programmes of experimentation. However, sociotechnical systems are generally a ‘moving target’ in terms of the pace of change. The objective of this study was to create a general approach to experimental design and the measurement of cognitive work that can satisfy the requirements for experimentation and yet can also provide a ‘fast track’ to the evaluation of software-supported cognitive work. A measure called i-bar is presented, which is the inverse of the mid-range. The statistic is derived from data on trials-to-criterion in tasks that require practice and learning. This single measure is interpreted as a conjoint measurement scale, permitting: (a) evaluation of sensitivity of the principal performance measure (which is used to set the metric for trials to criterion); (b) evaluation of the learnability of the work method (i.e. the goodness of the software tool); (c) evaluation of the resilience of the work method. It is shown that it is possible to mathematically model such order statistics and derive methods for estimating likelihoods. This involves novel ways of thinking about statistical analysis for discrete non-Gaussian distributions. The idea and method presented herein should be applicable to the study of the effects of any training or intervention, including software interventions designed to improve legacy work methods and interventions that involve creating entirely new cognitive work systems.     

Operator monitoring in a complex dynamic work environment: a qualitative cognitive model based on field observations

Complex and dynamic work environments provide a challenging litmus-test with which to evaluate basic and applied theories of cognition. In this work, we were interested in obtaining a better understanding of dynamic decision making by studying how human operators monitored a nuclear power plant during normal operations. Interviews and observations were conducted in situ at three different power plants to enhance the generalizability of results across both individuals and plants. A total of 38 operators were observed for approximately 288 hours, providing an extensive database of qualitative data. Based on these empirical observations, a cognitive model of operator monitoring was developed. This qualitative model has important theoretical implications because it integrates findings from several theoretical perspectives. There is a strong human information processing component in that operators rely extensively on active knowledge-driven monitoring rather than passively reacting to changes after they occur, but there is also a strong distributed cognition component in that operators rely extensively on the external representations to offload cognitive demands. In some cases, they even go so far as to actively shape that environment to make it easier to exploit environmental regularities, almost playing the role of designers. Finally, expert operators use workload regulation strategies, allowing them to prioritize tasks so that they avoid situations that are likely to lead to monitoring errors. These meta-cognitive processes have not received much attention in the human information processing and distributed cognition perspectives, although they have been studied by European psychologists who have studied cognition in complex work environments. Collectively, these findings shed light on dynamic decision making but they also serve an important theoretical function by integrating findings from different theoretical perspectives into one common framework.     

An information trail model for capturing human behaviour in artefact creation and use in complex work systems

This article presents an information trail model. The model is a method for uncovering information transformation using artefacts in complex work systems. We use a patient care scenario in a hospital emergency department to illustrate theoretical, methodological and design elements of the model. In a complex system, humans create and manage complexity, a paradox. The information trail model proposes that humans manage the paradox by creating and evolving artefacts and information cues. They organise purposefully through self-organisation and stigmergical behaviour. Information trail model states that humans leave trails of information as signs or symbols, and the piecing together and transformation of which will lead to final goals. Information trails contain work attributes and strategies, which can be utilised in work system design.     

Analysing activity in complex systems with cognitive work analysis: concepts,guidelines and case study for control task analysis

Cognitive work analysis is gaining momentum as an approach for the analysis, design and evaluation of complex sociotechnical systems. This paper focuses on control task analysis (ConTA), the second phase of cognitive work analysis. The paper consolidates existing approaches to ConTA and extends the basic concepts, in particular, by asserting that activity in some work systems is better characterized by both work situations and work functions and by introducing a new formative representation for ConTA called the contextual activity template. In addition, the paper proposes a set of methodological guidelines for performing ConTA and presents a case study of a ConTA for a first-of-a-kind military system called Airborne Early Warning and Control. As well as illustrating the conceptual extensions and methodological guidelines for ConTA by example, this case study highlights some of the difficulties of conducting ConTA for first-of-a-kind, complex systems during the early stages of development.     

Using reliability analysis to support decision making in phased mission systems

Because of the environments in which they will operate, future autonomous systems must be capable of reconfiguring quickly and safely following faults or environmental changes. Past research has shown how, by considering autonomous systems to perform phased missions, reliability analysis can support decision making by allowing comparison of the probability of success of different missions following reconfiguration. Binary decision diagrams (BDDs) offer fast, accurate reliability analysis that could contribute to real‐time decision making. However, phased mission analysis using existing BDD models is too slow to contribute to the instant decisions needed in time‐critical situations. This paper investigates 2 aspects of BDD models that affect analysis speed: variable ordering and quantification efficiency. Variable ordering affects BDD size, which directly affects analysis speed. Here, a new ordering scheme is proposed for use in the context of a decision‐making process. Variables are ordered before a mission, and reordering is unnecessary no matter how the mission configuration changes. Three BDD models are proposed to address the efficiency and accuracy of existing models. The advantages of the developed ordering scheme and BDD models are demonstrated in the context of their application within a reliability analysis methodology used to support decision making in an unmanned aerial vehicle.