Human factors contributions to knowledge elicitation

OBJECTIVE: The objective of this article is to lay out contributions of human factors to knowledge elicitation (KE) methodology. BACKGROUND: The background is historical, dating to about 1985, and involves the convergence of expert systems with applied psychology and cognitive psychology. METHOD: The method is a literature review, focusing on past issues of Human Factors. RESULTS: Human factors researchers have contributed significantly to KE methodology. However, KE methodology "belongs to" a number of communities of practice and has applications that transcend individual disciplines. CONCLUSION: Knowledge elicitation, thought of as a kind of cognitive task analysis, grows in importance with the increasing use of information technology to form complex sociotechnical work systems and the increasing importance of expertise to knowledge-based organizations. APPLICATION: I discuss some open issues for further research and methodological investigation.     

When paradigms collide at the road rail interface: evaluation of a sociotechnical systems theory design toolkit for cognitive work analysis

The Cognitive Work Analysis Design Toolkit (CWA-DT) is a recently developed approach that provides guidance and tools to assist in applying the outputs of CWA to design processes to incorporate the values and principles of sociotechnical systems theory. In this paper, the CWA-DT is evaluated based on an application to improve safety at rail level crossings. The evaluation considered the extent to which the CWA-DT met pre-defined methodological criteria and aligned with sociotechnical values and principles. Both process and outcome measures were taken based on the ratings of workshop participants and human factors experts. Overall, workshop participants were positive about the process and indicated that it met the methodological criteria and sociotechnical values. However, expert ratings suggested that the CWA-DT achieved only limited success in producing RLX designs that fully aligned with the sociotechnical approach. Discussion about the appropriateness of the sociotechnical approach in a public safety context is provided.

Practitioner Summary: Human factors and ergonomics practitioners need evidence of the effectiveness of methods. A design toolkit for cognitive work analysis, incorporating values and principles from sociotechnical systems theory, was applied to create innovative designs for rail level crossings. Evaluation results based on the application are provided and discussed.     

Accimap and crowd flow in urban infrastructure: Case study of Elphinstone road railway station tragedy,Mumbai, India

The aim of this article is to show how Human Factors and Ergonomics (HFE) methods can be used to improve urban public infrastructure systems in densely populated countries, such as India. In this case, we use Accimap analysis with crowd flow approaches to inform safety and policy. We demonstrate the need for sociotechnical systemic safety by a case study of accident analysis of the Elphinstone Road railway station stampede. On September 29, 2017, the Elphinstone Road, Mumbai, India, railway platform bridge stampede killed at least 23 and injured 39 other commuters who traveled through the Mumbai Suburban railway. In this study, we understand the accident as it is presented in newspaper articles. We analyze the accident by a sociotechnical accident analysis method called the Accimap. The Accimap method helps by identification of various stakeholders and their interactions in the different levels of hierarchy in a sociotechnical system. This ensures moving away from individualistic and blame-based accounts of media reporting to a coherent sociotechnical account based on understanding the dynamics of the situation. The findings from the Accimap analysis identify the problem areas in the commuter transit system and provide recommendations. These recommendations range from commuter flow management to enforcement of rules for supporting pedestrian flow. The article concludes with an emphasis on the development of the sociotechnical dimension of public safety and infrastructure from a human factors perspective, above and beyond what is currently practiced in India.     

A tripartite approach to operator-error evaluation in ballast water management system operation

This paper presents a new methodology developed to quantitatively analyze and prioritize the contributions of Human Factors (HFs) in the human-machine-interaction (HMI) within a complex sociotechnical system such as a Ballast Water Management (BWM) System. The methodology is a combination of the Human Factor Analysis Classification System (HFACS), Analytic Hierarchy Process (AHP), and a modified version of the Theory of Inventive Problem Solving (TRIZ) known as the Radial Dynamics Model (RDM). The methodology (HFACS-AHP-RDM) is based on a five-step algorithm, with which data from experts’ judgment was analyzed. A human-error and system risk minimization hierarchy was subsequently proposed to improve human performance and minimize the likelihood of an unwanted event such as the discharge of harmful aquatic organisms and pathogens (HAOPs). The result from the study in order of hierarchy showed fatigue, training and complex automation to be the HFs with the greatest impacts on BWM operations. Minimizing their impact, therefore, will have the greatest positive contribution on the performance of the system.Relevance to industryThe study's outcome shall help decision makers in prioritizing limited resources (e.g. time and money) allocation to resolving only issues related to HFs with the greatest impact on BWM System's performance. The new methodology could also be applicable in assessing the relative impacts of subjective criteria like HFs in complex sociotechnical systems other than BWM Systems.     

Designing for adaptation to novelty and change: functional information,emergent feature graphics,and higher-level control

Ecological interface design (EID) is a theoretical framework that aims to support worker adaptation to change and novelty in complex systems. Previous evaluations of EID have emphasized representativeness to enhance generalizability of results to operational settings. The research presented here is complementary, emphasizing experimental control to enhance theory building. Two experiments were conducted to test the impact of functional information and emergent feature graphics on adaptation to novelty and change in a thermal-hydraulic process control microworld. Presenting functional information in an interface using emergent features encouraged experienced participants to become perceptually coupled to the interface and thereby to exhibit higher-level control and more successful adaptation to unanticipated events. The absence of functional information or of emergent features generally led to lower-level control and less success at adaptation, the exception being a minority of participants who compensated by relying on analytical reasoning. These findings may have practical implications for shaping coordination in complex systems and fundamental implications for the development of a general unified theory of coordination for the technical, human, and social sciences. Actual or potential applications of this research include the design of human-computer interfaces that improve safety in complex sociotechnical systems.     

Macroergonomics as an organizing process for systems safety

Hendrick is attributed with the formalization of organizational design and management (ODAM) in ergonomics [Hendrick, H.W., Kleiner, B.M., 2001. Macroergonomics: An Introduction to Work System Design. Human Factors and Ergonomics Society, Santa Monica, CA.]. Specifically, the method called “Macroergonomic Analysis of Structure” or MAS provides a framework and analysis of these factors and provides the context for an analysis of organizational design and management process through the MacroErgonomic Analysis and Design method (MEAD). Together, MAS and MEAD represent the formalization of staple methods in macroergonomics and can be used to organize existing tools and methods such as those that exist in systems safety and help to differentiate macroergonomics from other approaches. This article illustrates such an integrative role for macroergonomics with respect to systems safety using the example of the construction sector, a domain in which accidents, injuries and fatalities are all too common.     

Designing sociotechnical systems with cognitive work analysis: putting theory back into practice

Cognitive work analysis (CWA) is a framework of methods for analysing complex sociotechnical systems. However, the translation from the outputs of CWA to design is not straightforward. Sociotechnical systems theory provides values and principles for the design of sociotechnical systems which may offer a theoretically consistent basis for a design approach for use with CWA. This article explores the extent to which CWA and sociotechnical systems theory offer complementary perspectives and presents an abstraction hierarchy (AH), based on a review of literature, that describes an ‘optimal’ CWA and sociotechnical systems theory design system. The optimal AH is used to assess the extent to which current CWA-based design practices, uncovered through a survey of CWA practitioners, aligns with sociotechnical systems theory. Recommendations for a design approach that would support the integration of CWA and sociotechnical systems theory design values and principles are also derived.     

Human factors in patient safety as an innovation

The use of Human Factors and Ergonomics (HFE) tools, methods, concepts and theories has been advocated by many experts and organizations to improve patient safety. To facilitate and support the spread of HFE knowledge and skills in healthcare and patient safety, we propose to conceptualize HFE as innovations whose diffusion, dissemination, implementation and sustainability need to be understood and specified. Using Greenhalgh et al. (2004) model of innovation, we identified various factors that can either hinder or facilitate the spread of HFE innovations in healthcare organizations. Barriers include lack of systems thinking, complexity of HFE innovations and lack of understanding about the benefits of HFE innovations. Positive impact of HFE interventions on task performance and the presence of local champions can facilitate the adoption, implementation and sustainability of HFE innovations. This analysis concludes with a series of recommendations for HFE professionals, researchers and educators.     

A general framework for analyzing techno-behavioural dynamics on networks

A general framework for assessing future impacts of technology on society and environment is presented. The dynamics between human activity and technological systems impact upon many processes in society and nature. This involves non-linear dynamics requiring an understanding of how technology and human behaviour influence each other and co-evolve. Conventionally, technological and behavioural systems are analyzed as separate entities. We develop an integrated theoretical and methodological approach termed techno-behavioural dynamics focussing on networked interactions between technology and behaviour across multiple system states. We find that positive feedback between technology learning, evolving preferences and network effects can lead to tipping points in complex sociotechnical systems. We also demonstrate how mean-field and agent-based models are complimentary for capturing a hierarchy of analytical resolutions in a common problem domain. Assessing and predicting co-evolutionary dynamics between technology and human behaviour can help avoid systems lock-in and inform a range of adaptive responses to environmental and societal risk.     

Situation awareness,human error,and organizational learning in sociotechnical systems

In many organizations, the performance of individuals and teams is negatively affected by human error. Studies have shown that these errors can be reduced or even prevented by learning from them and by developing an understanding of error causation and its consequences. The ability to detect, understand, and anticipate errors refers to situation awareness (SA). Although SA is not limited to human error and it is more closely linked with decision making, it is a prerequisite for error reduction in complex sociotechnical work settings. The main objectives of this study were threefold: First, a model that can explain the interrelations between human error, SA, and organizational learning in sociotechnical systems was developed. Secondly, functional and dysfunctional factors that affect human error, SA, and organizational learning were identified. Thirdly, a research methodology was selected and adapted to empirically test the model in a real‐world sociotechnical task environment. To do so, an SA performance test and a human error questionnaire were used to examine SA and respective learning modes of 108 assembly‐line workers in the manufacturing industry. The final test results supported the central assumptions of the applied model. The article concludes by discussing applications in the field of sociotechnical systems analysis, team training, human performance programs, and high‐reliability organizations.     

The Procurement Woes Revisited

The set of people who are frustrated every day by badly designed information technology is very large. So is the set of people whose dollars pay for the badly designed technology. A conservative estimate ranges in the billions for the cost of large-scale information systems that end up collecting dust because they're not properly human-centered. Yes, billions and still counting-that's the scary part. Within this large set of frustrated customers (see the sidebar "When Systems Development Neglects Human Considerations") is a subset whose job it is to do something about this situation. That subset includes policymakers, program managers, and systems engineers. It also includes a sub-subset comprising cognitive systems engineers, ethnographers, and many others who, in one vernacular or another, advocate human-centered computing. We must show that intelligent technologies-those designed to interact with humans or play a role in the cognitive work conducted in sociotechnical work systems-are usable, useful, and understandable.     

Toward the incorporation of temporal interaction analysis techniques in modeling and understanding sociotechnical systems

The increased complexity of modern sociotechnical systems (STS) necessitates the need for a manageable representation of their attributes, to augment our understanding and enable the development of ways through which we can increase their effectiveness, efficiency, and safety. Although many of the methodologies developed in the Human Factors domain map and investigate system properties and network structures, the inclusion of the temporal dimension in the analysis of STS remains limited. In this paper we present how modeling and visualization of STS can be augmented with the incorporation of temporal interaction analysis techniques that enable a micro-level, fine-grained analysis of data. We provide an overview of temporal analysis techniques by breaking down their main function, requirements, types of research questions they can address, and the visualization properties they offer, attempting to enhance their use in system analysis. This overview can assist researchers in selecting an analysis technique, enabling the incorporation of temporality in STS analysis, and helping towards the design of improved and safer systems and interventions.     

Human factors of complex sociotechnical systems

Increasingly products and services result from interactions among people who work across organizational, geographical, cultural and temporal boundaries. This has major implications for human factors and ergonomics (HFE), in particular, challenging the limits of the systems to be designed, and widening the range of system elements and dimensions that we need to consider. The design of sociotechnical systems that involve work across multiple boundaries requires better integration of the various sub-disciplines or components of HFE, as well as increased collaboration with other disciplines that provide either expertise regarding the domain of application or expertise in concepts that can enrich the system design. In addition, 'customers' contribute significantly to the 'co-production' of products/services, as well as to their quality/safety. The design of sociotechnical systems in collaboration with both the workers in the systems and the customers requires increasing attention not only to the design and implementation of systems, but also to the continuous adaptation and improvement of systems in collaboration with customers. This paper draws from research on human factors in the domains of health care and patient safety and of computer security.     

Organizational-historical analysis of the "failure to respond to alarm" problems

Large healthcare organizations are complex sociotechnical systems in many senses. Understanding how such complex systems adapt to their internal and external environment may help us devise effective strategies in building robust, high-reliability organizations. In this paper, we report findings of an in-depth study of auditory alarms in acute care settings in an attempt to study complex organizations. Investigations into incidents with adverse outcomes often lead to blames of not responding to auditory alarms which, in most cases, sound at audible levels. We interviewed a number of clinicians and stakeholders to understand the reasons why in some circumstances alarms are not responded to. The findings illustrate a spectrum of reasons that predispose care providers in certain responding patterns, such as large numbers of alarms, confusion of alarms, temporary episodes of high workload, and external economic pressures. We also uncovered a number of proactive interventions at unit and organizational levels that sometimes had unanticipated effects.     

What could they have been thinking? How sociotechnical system design influences cognition: a case study of the Stockwell shooting

Understanding why an individual acted in a certain way is of fundamental importance to the human factors community, especially when the choice of action results in an undesirable outcome. This challenge is typically tackled by applying retrospective interview techniques to generate models of what happened, recording deviations from a 'correct procedure'. While such approaches may have great utility in tightly constrained procedural environments, they are less applicable in complex sociotechnical systems that require individuals to modify procedures in real time to respond to a changing environment. For complex sociotechnical systems, a formative approach is required that maps the information available to the individual and considers its impact on performance and action. A context-specific, activity-independent, constraint-based model forms the basis of this approach. To illustrate, an example of the Stockwell shooting is used, where an innocent man, mistaken for a suicide bomber, was shot dead. Transferable findings are then presented. STATEMENT OF RELEVANCE: This paper presents a new approach that can be applied proactively to consider how sociotechnical system design, and the information available to an individual, can affect their performance. The approach is proposed to be complementary to the existing tools in the mental models phase of the cognitive work analysis framework.     

SEIPS 2.0: a human factors framework for studying and improving the work of healthcare professionals and patients

Healthcare practitioners, patient safety leaders, educators and researchers increasingly recognise the value of human factors/ergonomics and make use of the discipline's person-centred models of sociotechnical systems. This paper first reviews one of the most widely used healthcare human factors systems models, the Systems Engineering Initiative for Patient Safety (SEIPS) model, and then introduces an extended model, ‘SEIPS 2.0’. SEIPS 2.0 incorporates three novel concepts into the original model: configuration, engagement and adaptation. The concept of configuration highlights the dynamic, hierarchical and interactive properties of sociotechnical systems, making it possible to depict how health-related performance is shaped at ‘a moment in time’. Engagement conveys that various individuals and teams can perform health-related activities separately and collaboratively. Engaged individuals often include patients, family caregivers and other non-professionals. Adaptation is introduced as a feedback mechanism that explains how dynamic systems evolve in planned and unplanned ways. Key implications and future directions for human factors research in healthcare are discussed.     

Assessment of patient safety research from an organizational ergonomics and structural perspective

The aim of this study is to review patient safety improvement initiatives within a conceptual framework that builds upon principles of organizational ergonomics and emphasizes structural factors that influence patient safety. The literature review included 131 English language published studies of patient safety improvement strategies extracted using Medline, Ovid Healthstar, PubMed and CINAHL searches. Keywords for the search included: ‘patient safety’; ‘medical errors’; ‘adverse event’; ‘iatrogenic’; and truncated options for ‘improve’. The multilevel, hierarchical framework offered in this paper integrates quality management principles and organizational ergonomics theory and organizes patient safety initiatives according to sociotechnical system elements within three structural levels: health policies and associated health care organizations; health care delivery organizations; and health care microsystems. Utilizing the conceptual framework, this review of patient safety improvement initiatives highlights the need for consideration of the impact of all improvement proposals on each structural component within health care systems. The review also supports the need for patient safety research to evolve from exploratory, 1-D reporting to multi-level, integrated research.     

Assessment of patient safety research from an organizational ergonomics and structural perspective

The aim of this study is to review patient safety improvement initiatives within a conceptual framework that builds upon principles of organizational ergonomics and emphasizes structural factors that influence patient safety. The literature review included 131 English language published studies of patient safety improvement strategies extracted using Medline, Ovid Healthstar, PubMed and CINAHL searches. Keywords for the search included: 'patient safety'; 'medical errors'; 'adverse event'; 'iatrogenic'; and truncated options for 'improve'. The multilevel, hierarchical framework offered in this paper integrates quality management principles and organizational ergonomics theory and organizes patient safety initiatives according to sociotechnical system elements within three structural levels: health policies and associated health care organizations; health care delivery organizations; and health care microsystems. Utilizing the conceptual framework, this review of patient safety improvement initiatives highlights the need for consideration of the impact of all improvement proposals on each structural component within health care systems. The review also supports the need for patient safety research to evolve from exploratory, 1-D reporting to multi-level, integrated research.     

Designing a ticket to ride with the Cognitive Work Analysis Design Toolkit

Cognitive work analysis has been applied in the design of numerous sociotechnical systems. The process used to translate analysis outputs into design concepts, however, is not always clear. Moreover, structured processes for translating the outputs of ergonomics methods into concrete designs are lacking. This paper introduces the Cognitive Work Analysis Design Toolkit (CWA-DT), a design approach which has been developed specifically to provide a structured means of incorporating cognitive work analysis outputs in design using design principles and values derived from sociotechnical systems theory. This paper outlines the CWA-DT and describes its application in a public transport ticketing design case study. Qualitative and quantitative evaluations of the process provide promising early evidence that the toolkit fulfils the evaluation criteria identified for its success, with opportunities for improvement also highlighted.