The step complexity measure for emergency operating procedures: measure verification

In complex systems, such as nuclear power plants (NPPs) or airplane control systems, human errors play a major role in many accidents. Therefore, to prevent an occurrence of accidents or to ensure system safety, extensive effort has been made to identify significant factors that can cause human errors. According to related studies, written manuals or operating procedures are revealed as one of the most important factors, and the understandability is pointed out as one of the major reasons for procedure-related human errors.Many qualitative checklists are suggested to evaluate emergency operating procedures (EOPs) of NPPs. However, since qualitative evaluations using checklists have some drawbacks, a quantitative measure that can quantify the complexity of EOPs is very necessary to compensate for them.In order to quantify the complexity of steps included in EOPs, Park et al. suggested the step complexity (SC) measure. In addition, to ascertain the appropriateness of the SC measure, averaged step performance time data obtained from emergency training records for the loss of coolant accident and the excess steam dump event were compared with estimated SC scores. Although averaged step performance time data show good correlation with estimated SC scores, conclusions for some important issues that have to be clarified to ensure the appropriateness of the SC measure were not properly drawn because of lack of backup data.In this paper, to clarify remaining issues, additional activities to verify the appropriateness of the SC measure are performed using averaged step performance time data obtained from emergency training records. The total number of available records is 36, and training scenarios are the steam generator tube rupture and the loss of all feedwater. The number of scenarios is 18 each. From these emergency training records, averaged step performance time data for 30 steps are retrieved.As the results, the SC measure shows statistically meaningful correlation with averaged step performance time data. In addition, since it is observed that the SC measure seems to have the procedure independent property (i.e. steps that have similar SC scores, whether they are included in different procedures or not, would have similar step performance time), it can be concluded that the SC measure can represent the complexity of steps included in EOPs.     

Development of the step complexity measure for emergency operating procedures using entropy concepts

For a nuclear power plant (NPP), symptom-based emergency operating procedures (EOPs) have been adopted to enhance the safety of NPPs through reduction of operators’ workload under emergency conditions. Symptom-based EOPs, however, could place a workload on operators because they have to not only identify related symptoms, but also understand the context of steps that should be carried out. Therefore, many qualitative checklists are suggested to ensure the appropriateness of steps included in EOPs. However, since these qualitative evaluations have some drawbacks, a quantitative measure that can roughly estimate the complexity of EOP steps is imperative to compensate for them.In this paper, a method to evaluate the complexity of an EOP step is developed based on entropy measures that have been used in software engineering. Based on these, step complexity (SC) measure that can evaluate SC from various viewpoints (such as the amount of information/operators’ actions included in each EOP step, and the logic structure of each EOP step) was developed.To verify the suitableness of the SC measure, estimated SC values are compared with subjective task load scores obtained from the NASA-TLX (task load index) method and step performance time obtained from a full scope simulator. From these comparisons, it was observed that estimated SC values generally agree with the NASA-TLX scores and step performance time data. Thus, it could be concluded that the developed SC measure would be considered for evaluating SC of an EOP step.     

The operators' non-compliance behavior to conduct emergency operating procedures—comparing with the work experience and the complexity of procedural steps

Many kinds of procedures have been used to reduce the operators' workload throughout various industries, such as in the aviation, the chemical and the nuclear industry. It is remarkable that, however, significant portion of accidents or incidents was caused by procedure related human error due to non-compliance of procedures.In this study, to investigate the operators' non-compliance behavior, emergency-training records were collected using a full scope simulator. And three types of the operators' behavior (such as strict adherence, skipping redundant actions and modifying action sequences) observed from collected emergency training records were compared with both their work experience and the complexity of procedural steps.As the results, three remarkable relationships are obtained. They are: (1) the operators who have an intermediate work experience seem to frequently adopt non-compliance behavior to conduct the procedural steps, (2) the operators seem to frequently adopt non-compliance behavior to conduct the procedural steps that have an intermediate procedural complexity, and (3) the senior reactor operators seem to accommodate their non-compliance behavior based on the complexity of procedural steps. Therefore, it is expected that these relationships can be used as meaningful clues not only to scrutinize the reason for non-compliance behavior but also to suggest appropriate remedies for the reduction of non-compliance behavior that can result in procedure related human error.     

Influence of step complexity and presentation style on step performance of computerized emergency operating procedures

With the development of information technology, computerized emergency operating procedures (EOPs) are taking the place of paper-based ones. However, ergonomics issues of computerized EOPs have not been studied adequately since the industrial practice is quite limited yet. This study examined the influence of step complexity and presentation style of EOPs on step performance. A simulated computerized EOP system was developed in two presentation styles: Style A: one- and two-dimensional flowcharts combination; Style B: two-dimensional flowchart and success logic tree combination. Step complexity was quantified by a complexity measure model based on an entropy concept. Forty subjects participated in the experiment of EOP execution using the simulated system. The results of data analysis on the experiment data indicate that step complexity and presentation style could significantly influence step performance (both step error rate and operation time). Regression models were also developed. The regression analysis results imply that operation time of a step could be well predicted by step complexity while step error rate could only partly predicted by it. The result of a questionnaire investigation implies that step error rate was influenced not only by the operation task itself but also by other human factors. These findings may be useful for the design and assessment of computerized EOPs.     

A study on the development of a task complexity measure for emergency operating procedures of nuclear power plants

In this study, a measure called task complexity (TACOM) that can quantify the complexity of tasks stipulated in emergency operating procedures of nuclear power plants is developed. The TACOM measure consists of five sub-measures that can cover remarkable complexity factors: (1) amount of information to be managed by operators, (2) logical entanglement due to the logical sequence of the required actions, (3) amount of actions to be accomplished by operators, (4) amount of system knowledge in recognizing the problem space, and (5) amount of cognitive resources in establishing an appropriate decision criterion. The appropriateness of the TACOM measure is investigated by comparing task performance time data with the associated TACOM scores. As a result, it is observed that there is a significant correlation between TACOM scores and task performance time data. Therefore, it is reasonable to expect that the TACOM measure can be used as a meaningful tool to quantify the complexity of tasks.     

Identifying cognitive complexity factors affecting the complexity of procedural steps in emergency operating procedures of a nuclear power plant

In complex systems such as a nuclear and chemical plant, it is well known that the provision of understandable procedures that allow operators to clarify what needs to be done and how to do it is one of the requisites to secure their safety.As a previous study in providing understandable procedures, the step complexity (SC) measure that can quantify the complexity of procedural steps in emergency operating procedures (EOPs) of a nuclear power plant (NPP) was suggested. However, the necessity of additional complexity factors that can consider a cognitive aspect in evaluating the complexity of procedural steps is raised.To this end, the comparisons between operators' performance data measured by the form of a step performance time with their behavior in carrying out the prescribed activities of procedural steps are conducted in this study. As a result, two kinds of complexity factors (the abstraction level of knowledge and the level of engineering decision) that could affect an operator's cognitive burden are identified.Although a well-designed experiment is indispensable for confirming the appropriateness of the additional complexity factors, it is strongly believed that the change of operators' performance data can be more authentically explained if the additional complexity factors are taken into consideration.     

A study on the validity of a task complexity measure for emergency operating procedures of nuclear power plants—Comparing task complexity scores with two sets of operator response time data obtained under a simulated SGTR

The appropriateness of the task complexity (TACOM) measure that can quantify the complexity of procedural tasks was validated in this study. To this end, two sets of task performance time data that have been extracted under the simulated steam generator tube rupture (SGTR) conditions of the reference nuclear power plant (NPP) A and B were compared with the associated TACOM scores. As a result, it was observed that two sets of task performance time data seem to be soundly explained by the associated TACOM scores. Although more additional activities should be conducted to clarify the appropriateness of the TACOM measure, the result of this study provides a crucial clue supporting that the complexity of emergency tasks stipulated in emergency operating procedures (EOPs) can be properly quantified by the TACOM measure.     

Comparing the complexity of procedural steps with the operators' performance observed under stressful conditions

In this study, to clarify the appropriateness of the step complexity (SC) measure, step performance time data are empirically collected under stressful conditions, and then they are compared with the SC scores. As for the result, when the operators have to accomplish their tasks using the procedures, it is found that there is a statistically meaningful correlation between the SC scores and the operators' performance data under stressful conditions. Therefore, although the data used in this study are not obtained from designed experiments but from empirical observations, it is quite positive to expect that the SC measure could be a useful method to scrutinize the operators' performance under stressful conditions. It is also expected that the SC measure can be used to improve both the procedures and the quality of the emergency training because the complicated tasks that need careful consideration are effectively distinguished by it.     

The effect of two complexity factors on the performance of emergency tasks—An experimental verification

It is well known that the use of procedures is very important in securing the safety of process systems, since good procedures effectively guide human operators by providing ‘what should be done’ and ‘how to do it’, especially under stressful conditions. At the same time, it has been emphasized that the use of complicated procedures could drastically impair operators’ performance. This means that a systematic approach that can properly evaluate the complexity of procedures is indispensable for minimizing the side effects of complicated procedures. For this reason, Park et al. have developed a task complexity measure called TACOM that can be used to quantify the complexity of tasks stipulated in emergency operating procedures (EOPs) of nuclear power plants (NPPs). The TACOM measure consists of five sub-measures that can cover five important factors making the performance of emergency tasks complicated. However, a verification activity for two kinds of complexity factors—the level of abstraction hierarchy (AH) and engineering decision (ED)—seems to be insufficient. In this study, therefore, an experiment is conducted by using a low-fidelity simulator in order to clarify the appropriateness of these complexity factors. As a result, it seems that subjects’ performance data are affected by the level of AH as well as ED. Therefore it is anticipate that both the level of AH and ED will play an important role in evaluating the complexity of EOPs.     

The requisite characteristics for diagnosis procedures based on the empirical findings of the operators' behavior under emergency situations

In this study, to suggest requisite characteristics for diagnosis procedures, the SROs' (senior reactor operators) behaviors in conducting the diagnosis procedure under simulated conditions are observed and analyzed in order to identify factors that make the performance of it difficult. As the results, four important characteristics that may be helpful in constructing effective and useful diagnosis procedures are suggested. These are: (1) the use of connection symbols should be minimized, (2) when connection symbols are used, precise information about entry symbols should be provided to the operators, (3) the use of negative expressions should be avoided, and (4) the diagnosis procedure should be constructed through the integration of both the non-compensatory and the compensatory approach. Although these characteristics are deduced, not from well-designed experiments, but from empirical observations, it is highly expected that the operators' diagnostic burden could be significantly reduced, if the above characteristics are properly embodied in the diagnosis procedures.     

The appropriateness of the systematic framework to develop diagnosis procedures of nuclear power plants—an experimental verification

It has been well recognized that a diagnosis procedure that allows operators to successfully identify the nature of an on-going event is inevitable for an effective and appropriate recovery. Unfortunately, studies for a framework that can suggest a unified and consistent process in constructing a serviceable diagnosis procedure seem to be scant. Thus, Park et al. have suggested a systematic framework that can be used to construct a useful diagnosis procedure. In addition, the diagnosis procedure that is currently in use at the reference nuclear power plant (NPP) is reformed in order to demonstrate the appropriateness of the suggested framework. However, the necessity of a well-designed experiment is proposed to confirm the appropriateness of the suggested framework.In this regard, in this study, an experiment is conducted using a full-scope simulator of the reference NPP. From the experiment, two sets of operators' diagnosis performance data are collected, and then they are compared to investigate the change of an operator's diagnosis performance with respect to two types of diagnosis procedures.As a result, it is shown that an operator's diagnosis performance is improved when the revised diagnosis procedure is used. Therefore, it is reasonable to conclude that the suggested framework is useful in constructing an effective diagnosis procedure.