Skills,rules and knowledge in aircraft maintenance: errors in context

Automatic or skill-based behaviour is generally considered to be less prone to error than behaviour directed by conscious control. However, researchers who have applied Rasmussen's skill-rule-knowledge human error framework to accidents and incidents have sometimes found that skill-based errors appear in significant numbers. It is proposed that this is largely a reflection of the opportunities for error which workplaces present and does not indicate that skill-based behaviour is intrinsically unreliable. In the current study, 99 errors reported by 72 aircraft mechanics were examined in the light of a task analysis based on observations of the work of 25 aircraft mechanics. The task analysis identified the opportunities for error presented at various stages of maintenance work packages and by the job as a whole. Once the frequency of each error type was normalized in terms of the opportunities for error, it became apparent that skill-based performance is more reliable than rule-based performance, which is in turn more reliable than knowledge-based performance. The results reinforce the belief that industrial safety interventions designed to reduce errors would best be directed at those aspects of jobs that involve rule- and knowledge-based performance.     

Unsafe acts and unsafe outcomes in aircraft maintenance

Road safety studies using the Driver Behaviour Questionnaire (DBQ) have provided support for a three-way distinction between violations, skill-based errors and mistakes, and have indicated that a tendency to commit driving violations is associated with an increased risk of accident involvement. The aims of this study were to examine whether the three-way distinction of unsafe acts is applicable in the context of aircraft maintenance, and whether involvement in maintenance safety occurrences can be predicted on the basis of self-reported unsafe acts. A Maintenance Behaviour Questionnaire (MBQ) was developed to explore patterns of unsafe acts committed by aircraft maintenance mechanics. The MBQ was completed anonymously by over 1300 Australian aviation mechanics, who also provided information on their involvement in workplace accidents and incidents. Four factors were identified: routine violations, skill-based errors, mistakes and exceptional violations. Violations and mistakes were related significantly to the occurrence of incidents that jeopardized the quality of aircraft maintenance, but were not related to workplace injuries. Skill-based errors, while not related to work quality incidents, were related to workplace injuries. The results are consistent with the three-way typology of unsafe acts described by Reason et al. (1990) and with the DBQ research indicating an association between self-reported violations and accidents. The current findings suggest that interventions addressed at maintenance quality incidents should take into account the role of violations and mistakes, and the factors that promote them. In contrast, interventions directed at reducing workplace injury are likely to require a focus on skill-based errors.     

Unsafe acts and unsafe outcomes in aircraft maintenance

Road safety studies using the Driver Behaviour Questionnaire (DBQ) have provided support for a three-way distinction between violations, skill-based errors and mistakes, and have indicated that a tendency to commit driving violations is associated with an increased risk of accident involvement. The aims of this study were to examine whether the three-way distinction of unsafe acts is applicable in the context of aircraft maintenance, and whether involvement in maintenance safety occurrences can be predicted on the basis of self-reported unsafe acts. A Maintenance Behaviour Questionnaire (MBQ) was developed to explore patterns of unsafe acts committed by aircraft maintenance mechanics. The MBQ was completed anonymously by over 1300 Australian aviation mechanics, who also provided information on their involvement in workplace accidents and incidents. Four factors were identified: routine violations, skill-based errors, mistakes and exceptional violations. Violations and mistakes were related significantly to the occurrence of incidents that jeopardized the quality of aircraft maintenance, but were not related to workplace injuries. Skill-based errors, while not related to work quality incidents, were related to workplace injuries. The results are consistent with the three-way typology of unsafe acts described by Reason et al. (1990) and with the DBQ research indicating an association between self-reported violations and accidents. The current findings suggest that interventions addressed at maintenance quality incidents should take into account the role of violations and mistakes, and the factors that promote them. In contrast, interventions directed at reducing workplace injury are likely to require a focus on skill-based errors.     

Human errors in fatal and serious occupational accidents in Finland

The aim of this study is to examine the effect of human error on industrial accidents. One hundred and seventy-eight fatalities and 99 serious accidents were classified according to Rasmussen's SRK model. Of accidents, 84-94% were due mainly to human error. Most of the errors were skill-based, next came rule-based errors and then knowledge-based errors. The type of error was analyzed by age, work experience and work tasks of the victims. The results showed that the SRK model is suitable for analyzing accidents in industrial work using rather simple technology.     

Effectiveness of user testing and heuristic evaluation as a function of performance classification

For different levels of user performance, different types of information are processed and users will make different types of errors. Based on the error's immediate cause and the information being processed, usability problems can be classified into three categories. They are usability problems associated with skill-based, rule-based and knowledge-based levels of performance. In this paper, a user interface for a Web-based software program was evaluated with two usability evaluation methods, user testing and heuristic evaluation. The experiment discovered that the heuristic evaluation with human factor experts is more effective than user testing in identifying usability problems associated with skill-based and rule-based levels of performance. User testing is more effective than heuristic evaluation in finding usability problems associated with the knowledge-based level of performance. The practical application of this research is also discussed in the paper.     

Using Neural Networks to predict HFACS unsafe acts from the pre-conditions of unsafe acts

Abstract

Human Factors Analysis and Classification System (HFACS) is based upon Reason’s organizational model of human error which suggests that there is a ‘one to many’ mapping of condition tokens (HFACS level 2 psychological precursors) to unsafe act tokens (HFACS level 1 error and violations). Using accident data derived from 523 military aircraft accidents, the relationship between HFACS level 2 preconditions and level 1 unsafe acts was modelled using an artificial neural network (NN). This allowed an empirical model to be developed congruent with the underlying theory of HFACS. The NN solution produced an average overall classification rate of ca. 74% for all unsafe acts from information derived from their level 2 preconditions. However, the correct classification rate was superior for decision- and skill-based errors, than for perceptual errors and violations.

Practitioner Summary: A model to predict unsafe acts (HFACS level 1) from their preconditions (HFACS level 2) was developed from the analysis of 523 military aircraft accidents using an artificial NN. The results could correctly predict approximately 74% of errors.     

The links between human error diversity and software diversity: Implications for fault diversity seeking

Software diversity is known to improve fault tolerance in N-version software systems by independent development. As the leading cause of software faults, human error is considered an important factor in diversity seeking. However, there is little scientific research focusing on how to seek software fault diversity based on human error mechanisms. A literature review was conducted to extract factors that may differentiate people with respect to human error-proneness. In addition, we constructed a conceptual model of the links between human error diversity and software diversity. An experiment was designed to validate the hypotheses, in the form of a programming contest, accompanied by a survey of cognitive styles and personality traits. One hundred ninety-two programs were submitted for the identical problem, and 70 surveys were collected. Code inspection revealed 23 faults, of which 10 were coincident faults. The results show that personality traits seems not effective predictors for fault diversity as a whole model, whereas cognitive styles and program measurements moderately account for the variation of fault density. The results also show causal relations between performance levels and coincident faults: coincident faults are unlikely to occur at skill-based performance level; the coincident faults introduced in rule-based performances show a high probability of occurrence, and the coincident faults introduced in knowledge-based performances are shaped by the content and formats of the task itself. Based on these results, we have proposed a model to seek software diversity and prevent coincident faults.     

The role of error in organizing behaviour

During recent years, the significance of the concept of human error has changed considerably. The reason for this has partly been an increasing interest of psychological research in the analysis of complex real-life phenomena, and partly the changes of modern work conditions caused by advanced information technology. Consequently, the topic of the present contribution is not a definition of the concept or a proper taxonomy. Instead, a review is given of two professional contexts for which the concept of error is important. Three cases of analysis of human-system interaction are reviewed: (1) traditional task analysis and human reliability estimation; (2) causal analysis of accidents after the fact; and, finally, (3) design of reliable work conditions in modern socio-technical systems. It is concluded that ‘;errors’ cannot be studied as a separate category of behaviour fragments; the object of study should be cognitive control of behaviour in complex environments.     

Training Novice Users in Developing Strategies for Responding to Errors When Browsing the Web

Novice users frequently make errors when learning a new computer task and spend a large portion of their time trying to recover from errors. Three methods for helping novice users respond to errors have been presented in the literature: error management training, exploratory training, and conceptual models. In error management training, errors are presented as opportunities for learning, and users are instructed in strategies for coping with errors. In exploration, users are given an overview of their environment and are taught how to navigate through their task environment. Conceptual models are graphical or mathematical representations of a system that correspond closely to the real-world system. This experiment tested the effectiveness of these different approaches on training novice users to use the Internet. In this experiment, users received 3 hr of training on the World Wide Web and then were asked to perform a set of information retrieval tasks. Performance was measured in two ways: task performance and performance time. Participants who received exploratory training had significantly higher task performance. Participants who received exploration and conceptual models, both individually and together, were able to complete the tasks in less time. Error management had no significant effect on the performance of participants. In the task application of Web browsing, exploration seems to be the most appropriate training method for novice users.     

Interleaving tasks to improve performance: Users maximise the marginal rate of return

Technological developments have increased the opportunity for interleaving between tasks, leading to more interruptions and more choices for users. Three experiments tested the interleaving strategies of users completing simple office-based tasks while adjusting access control privileges to documents. Previous work predicted users would switch tasks to enable them to work on the task that produced the greatest current benefit—they would maximise the marginal rate of return. Results found that by interleaving between tasks users were able to focus on shorter tasks and that the interleaving decisions were consistent with a strategy of maximising the marginal rate of return. However, interruptions from access control tasks disrupted the processing involved in this task management and led to errors in task selection (Experiment 2) and task performance (Experiment 3). Task interleaving can therefore have costs in security contexts where errors can be catastrophic. Understanding which strategies maximise the marginal rate of return could predict users’ task management behaviour.     

Contact sensing and grasping performance of compliant hands

Limitations in modern sensing technologies result in large errors in sensed target object geometry and location in unstructured environments. As a result, positioning a robotic end-effector includes inherent error that will often lead to unsuccessful grasps. In previous work, we demonstrated that optimized configuration, compliance, viscosity, and adaptability in the mechanical structure of a robot hand facilitates reliable grasping in unstructured environments, even with purely feedforward control of the hand. In this paper we describe the addition of a simple contact sensor to the fingerpads of the SDM Hand (Shape Deposition Manufactured Hand), which, along with a basic control algorithm, significantly expands the grasp space of the hand and reduces contact forces during the acquisition phase of the grasp. The combination of the passive mechanics of the SDM Hand along with this basic sensor suite enables positioning errors of over 5 cm in any direction. In the context of mobile manipulation, the performance demonstrated here may reduce the need for much of the complex array of sensing currently utilized on mobile platforms, greatly increase reliability, and speed task execution, which can often be prohibitively slow.     

A paradigm for handwriting-based intelligent tutors

This paper presents the interaction design of, and demonstration of technical feasibility for, intelligent tutoring systems that can accept handwriting input from students. Handwriting and pen input offer several affordances for students that traditional typing-based interactions do not. To illustrate these affordances, we present evidence, from tutoring mathematics, that the ability to enter problem solutions via pen input enables students to record algebraic equations more quickly, more smoothly (fewer errors), and with increased transfer to non-computer-based tasks. Furthermore our evidence shows that students tend to like pen input for these types of problems more than typing. However, a clear downside to introducing handwriting input into intelligent tutors is that the recognition of such input is not reliable. In our work, we have found that handwriting input is more likely to be useful and reliable when context is considered, for example, the context of the problem being solved. We present an intelligent tutoring system for algebra equation solving via pen-based input that is able to use context to decrease recognition errors by 18% and to reduce recognition error recovery interactions to occur on one out of every four problems. We applied user-centered design principles to reduce the negative impact of recognition errors in the following ways: (1) though students handwrite their problem-solving process, they type their final answer to reduce ambiguity for tutoring purposes, and (2) in the small number of cases in which the system must involve the student in recognition error recovery, the interaction focuses on identifying the student’s problem-solving error to keep the emphasis on tutoring. Many potential recognition errors can thus be ignored and distracting interactions are avoided. This work can inform the design of future systems for students using pen and sketch input for math or other topics by motivating the use of context and pragmatics to decrease the impact of recognition errors and put user focus on the task at hand.     

Stability of digital control systems implemented in error-recoverable computers

Safety-critical, real-time applications make use of fault-tolerant digital control systems to achieve their performance goals. Even though these digital control systems are fault-tolerant, they are susceptible to errors induced by common-mode faults, since these errors cannot be masked by standard redundancy provisions. Such errors can be handled by special error correction mechanisms, which could require stopping the control law computations while the errors are removed. Thus, the effect of these special error recovery mechanisms on digital control systems needs to be understood. This paper presents a comprehensive study of the effect of three error recovery mechanisms–rollback, reset and cold restart–on the stability of digital closed-loop control systems. The effect of the faults, detected and handled by these error recovery mechanisms, on the digital control system is modelled by a set of interference maps. It is assumed that the arrival and departure of common-mode faults can be represented by a Markov process. The overall system behaviour is then described by a Markov jump linear model, whose stability is explored using standard techniques from the literature. The result of this analysis is a new metric, the minimum average interarrival spacing (MAIS), which is useful for comparing the performance of different error recovery mechanisms and for designing new fault-tolerant controllers. The theoretical results are illustrated via Monte Carlo simulations that show the effects of common-mode faults on the closed-loop stability of the longitudinal dynamics of an AFTI/F-16 aircraft.     

Situation awareness,human error,and organizational learning in sociotechnical systems

In many organizations, the performance of individuals and teams is negatively affected by human error. Studies have shown that these errors can be reduced or even prevented by learning from them and by developing an understanding of error causation and its consequences. The ability to detect, understand, and anticipate errors refers to situation awareness (SA). Although SA is not limited to human error and it is more closely linked with decision making, it is a prerequisite for error reduction in complex sociotechnical work settings. The main objectives of this study were threefold: First, a model that can explain the interrelations between human error, SA, and organizational learning in sociotechnical systems was developed. Secondly, functional and dysfunctional factors that affect human error, SA, and organizational learning were identified. Thirdly, a research methodology was selected and adapted to empirically test the model in a real‐world sociotechnical task environment. To do so, an SA performance test and a human error questionnaire were used to examine SA and respective learning modes of 108 assembly‐line workers in the manufacturing industry. The final test results supported the central assumptions of the applied model. The article concludes by discussing applications in the field of sociotechnical systems analysis, team training, human performance programs, and high‐reliability organizations.     

Task structure and postcompletion error in the execution of a routine procedure

OBJECTIVE: To replicate a successful laboratory slip-class error paradigm and, more importantly, to further understand the underlying causes of errors made in that paradigm. BACKGROUND: Routine procedural errors are facts of everyday life but have received limited controlled empirical study, despite the sometimes severe consequences associated with such errors. This research concerns one such error, postcompletion error (M. D. Byrne & S. Bovair, 1997), which is a lapse that occurs after the main goal of a task has been satisfied. METHOD: In the two experiments conducted, participants were trained to criterion on a routine procedural task and were then brought back to the lab for a later session or sessions in which performance on task execution was measured. In the second experiment, a variety of motivational manipulations, retraining, and task redesign were compared. RESULTS: Experiment 1 demonstrated a substantial reduction of error rate generated by a simple design change (alteration of when feedback about goal completion occurred). Furthermore, the reduction in error rate came with no penalty in terms of overall speed of performance. Experiment 2 showed that this more appropriate design is superior to motivationally oriented interventions, retraining, and even midtask redesign. As in Experiment 1, Experiment 2 revealed no speed-accuracy tradeoff. CONCLUSION: These experiments provide evidence that controlled laboratory studies of slip-class errors can be meaningful and highlight the centrality of cognitive factors (particularly goal structure) in such errors. APPLICATION: Potential applications include design of interfaces and their related procedures as well as error-mitigation techniques.     

Cognitive failure analysis for aircraft accident investigation

Abstract

The present studies were undertaken to investigate the applicability of an information processing approach to human failure in the aircraft cockpit. Using data obtained from official aircraft accident investigation reports, a database of accidents and incidents involving New Zealand civil aircraft between 1982 and 1991 was compiled. In the first study, reports were coded into one of three error stages proposed by Nagel (1988) and for the presence of any of 61 specific errors noted by Gerbert and Kemmler (1986). The importance of decisional factors in fatal crashes was noted. Principal components analysis suggested the presence of five different varieties of human failure. In the second study, a more detailed error taxonomy derived from the work of Rasmussen (1982) was applied to the data. Goal selection errors emerged as the most frequent kind of cognitive error in fatal accidents. Aircraft accident reports can be a useful source of information about cognitive failures if probed with an appropriate, theoretically-based, analysis of information processing errors. Such an approach could provide the accident investigators with a useful tool, and lead to a more complete understanding of human error in aviation.     

Predicting separation errors of air traffic controllers through integrated sequence analysis of multimodal behaviour indicators

Predicting separation errors in the daily tasks of air traffic controllers (ATCOs) is essential for the timely implementation of mitigation strategies before performance declines and the prevention of loss of separation and aircraft collisions. However, three challenges impede accurate separation errors forecasting: 1) compounding relationships between many human factors and control processes require sufficient operation process data to capture how separation errors occur and propagate within controller-in-the-loop processes; 2) previous human factor measurement approaches are disruptive to controllers’ daily operations because they use invasive sensors, such as electroencephalography (EEG) and electrocardiography (ECG), 3) errors accumulated in using the tasks and human behaviors for estimating system dynamics challenge accurate separation error predictions with sufficient leading time for proactive control actions. This study proposed a separation error prediction framework with a long leading time (>50 s) to address the above challenges, including 1) a multi-factorial model that characterizes the inter-relationships between task complexity, behavioral activity, cognitive load, and operational performance; 2) a multimodal data analytics approach to non-intrusively extract the task features (i.e., traffic density) from high-fidelity simulation systems and visual behavioral features (i.e., head pose, eyelid movements, and facial expressions) from ATCOs’ facial videos; 3) an encoder-decoder Long Short-Term Memory (LSTM) network to predict long-time-ahead separation errors by integrating multimodal features for reducing accumulated errors. A user study with six experienced ATCOs tested the proposed framework using the Phoenix Terminal Radar Approach Control (TRACON) simulator. The authors evaluated the model performance through two types of metrics: 1) point-level metrics, including precision, recall, and F1-score, and 2) sequence-level metrics, including alignment accuracy and sequence similarity. The results showed that 1) the model using the task and visual behavioral features significantly improved the prediction performance compared to the model using one single feature (eyelid movements), with an improvement of up to 26.95% in alignment accuracy for 10s-ahead prediction; 2) the model that combined task and visual behavioral features had a higher or comparable performance to models with different hybrid features, achieving an alignment accuracy of 82.38% for 50s-ahead error prediction; and (3) the proposed method outperformed three baseline models – Convolutional Neural Network (CNN), Gated Recurrent Unit (GRU), and classic LSTM – by 8.21%, 3.47%, and 3.14% in alignment accuracy, respectively, for predicting 50s-ahead separation errors. These results suggest that the proposed model can effectively predict separation errors in air traffic control.     

Human error identification in human reliability assessment. Part 1: Overview of approaches

This paper reviews a number of approaches which can be used to identify human errors in human-machine systems. These approaches range from simple error classifications to sophisticated software packages based on models of human performance. However, the prediction of human behaviour in complex environments is far from being an easy task itself, and there is significant scope for improvement in human error identification 'technology'. This first paper in a series of two reviews the available techniques of human error identification, highlighting their strengths and weaknesses. The second paper will review the validation of such approaches, and likely future trends in this area of human reliability assessment.     

Knowledge-based control for finite element analysis

This paper shows that control logic may be separated from analysis software and that a knowledge-based expert system can use this logic to perform interactive computation. Heuristics that control a simple interactive finite element analysis program are represented using a rule-based format and are used by a goal-driven logic processor to invoke analysis activity.Traditional algorithm-oriented control and the proposed knowledge-based control are compared in a simple displacement computation scenario to identify the advantages/disadvantages of the two approaches. General activities and constraints, practical methods of reasoning and representation, and knowledge-based expert systems are discussed with emphasis on applications to interactive finite element analysis.An analysis control expert system has been developed for use in the numerical analysis of two-dimensional linear problems in solid and structural mechanics. An example problem is used to clarify the methods used to direct activity and to identify the problems associated with conditional task processing for interactive analysis.The main difference between the analysis program described in this paper and conventional analysis programs is related to the control architecture. The general conclusion of this paper is that knowledge-based control is more effective and flexible than algorithm-oriented control.     

Language-oriented information retrieval

There is no task that computers regularly perform that is more affected by the nature of human language than the retrieval of texts in response to a human need. Despite this, the techniques actually in use for this task, as well as most of the techniques proposed by information retrieval (IR) researchers, make little use of knowledge about language. In this article we take the view that IR is an inference task, and that natural language processing (NLP) techniques can produce text representations that enable more accurate inferences about document content. By considering previous work on language-based and knowledge-based techniques from this perspective, some clear lessons are apparent, and we are applying these lessons in the ADRENAL (Augmented Document REtrieval using NAtural Language processing) project. Our initial experiments with hand-coded representations suggest that using NLP-produced representations can result in significant performance increases in IR systems, and also demonstrate the attention that must be given to representational issues in language-oriented IR.