A study on the validity of a task complexity measure for emergency operating procedures of nuclear power plants—Comparing task complexity scores with two sets of operator response time data obtained under a simulated SGTR

The appropriateness of the task complexity (TACOM) measure that can quantify the complexity of procedural tasks was validated in this study. To this end, two sets of task performance time data that have been extracted under the simulated steam generator tube rupture (SGTR) conditions of the reference nuclear power plant (NPP) A and B were compared with the associated TACOM scores. As a result, it was observed that two sets of task performance time data seem to be soundly explained by the associated TACOM scores. Although more additional activities should be conducted to clarify the appropriateness of the TACOM measure, the result of this study provides a crucial clue supporting that the complexity of emergency tasks stipulated in emergency operating procedures (EOPs) can be properly quantified by the TACOM measure.     

The effect of two complexity factors on the performance of emergency tasks—An experimental verification

It is well known that the use of procedures is very important in securing the safety of process systems, since good procedures effectively guide human operators by providing ‘what should be done’ and ‘how to do it’, especially under stressful conditions. At the same time, it has been emphasized that the use of complicated procedures could drastically impair operators’ performance. This means that a systematic approach that can properly evaluate the complexity of procedures is indispensable for minimizing the side effects of complicated procedures. For this reason, Park et al. have developed a task complexity measure called TACOM that can be used to quantify the complexity of tasks stipulated in emergency operating procedures (EOPs) of nuclear power plants (NPPs). The TACOM measure consists of five sub-measures that can cover five important factors making the performance of emergency tasks complicated. However, a verification activity for two kinds of complexity factors—the level of abstraction hierarchy (AH) and engineering decision (ED)—seems to be insufficient. In this study, therefore, an experiment is conducted by using a low-fidelity simulator in order to clarify the appropriateness of these complexity factors. As a result, it seems that subjects’ performance data are affected by the level of AH as well as ED. Therefore it is anticipate that both the level of AH and ED will play an important role in evaluating the complexity of EOPs.     

Development of the step complexity measure for emergency operating procedures using entropy concepts

For a nuclear power plant (NPP), symptom-based emergency operating procedures (EOPs) have been adopted to enhance the safety of NPPs through reduction of operators’ workload under emergency conditions. Symptom-based EOPs, however, could place a workload on operators because they have to not only identify related symptoms, but also understand the context of steps that should be carried out. Therefore, many qualitative checklists are suggested to ensure the appropriateness of steps included in EOPs. However, since these qualitative evaluations have some drawbacks, a quantitative measure that can roughly estimate the complexity of EOP steps is imperative to compensate for them.In this paper, a method to evaluate the complexity of an EOP step is developed based on entropy measures that have been used in software engineering. Based on these, step complexity (SC) measure that can evaluate SC from various viewpoints (such as the amount of information/operators’ actions included in each EOP step, and the logic structure of each EOP step) was developed.To verify the suitableness of the SC measure, estimated SC values are compared with subjective task load scores obtained from the NASA-TLX (task load index) method and step performance time obtained from a full scope simulator. From these comparisons, it was observed that estimated SC values generally agree with the NASA-TLX scores and step performance time data. Thus, it could be concluded that the developed SC measure would be considered for evaluating SC of an EOP step.     

Comparing the complexity of procedural steps with the operators' performance observed under stressful conditions

In this study, to clarify the appropriateness of the step complexity (SC) measure, step performance time data are empirically collected under stressful conditions, and then they are compared with the SC scores. As for the result, when the operators have to accomplish their tasks using the procedures, it is found that there is a statistically meaningful correlation between the SC scores and the operators' performance data under stressful conditions. Therefore, although the data used in this study are not obtained from designed experiments but from empirical observations, it is quite positive to expect that the SC measure could be a useful method to scrutinize the operators' performance under stressful conditions. It is also expected that the SC measure can be used to improve both the procedures and the quality of the emergency training because the complicated tasks that need careful consideration are effectively distinguished by it.     

The step complexity measure for emergency operating procedures: measure verification

In complex systems, such as nuclear power plants (NPPs) or airplane control systems, human errors play a major role in many accidents. Therefore, to prevent an occurrence of accidents or to ensure system safety, extensive effort has been made to identify significant factors that can cause human errors. According to related studies, written manuals or operating procedures are revealed as one of the most important factors, and the understandability is pointed out as one of the major reasons for procedure-related human errors.Many qualitative checklists are suggested to evaluate emergency operating procedures (EOPs) of NPPs. However, since qualitative evaluations using checklists have some drawbacks, a quantitative measure that can quantify the complexity of EOPs is very necessary to compensate for them.In order to quantify the complexity of steps included in EOPs, Park et al. suggested the step complexity (SC) measure. In addition, to ascertain the appropriateness of the SC measure, averaged step performance time data obtained from emergency training records for the loss of coolant accident and the excess steam dump event were compared with estimated SC scores. Although averaged step performance time data show good correlation with estimated SC scores, conclusions for some important issues that have to be clarified to ensure the appropriateness of the SC measure were not properly drawn because of lack of backup data.In this paper, to clarify remaining issues, additional activities to verify the appropriateness of the SC measure are performed using averaged step performance time data obtained from emergency training records. The total number of available records is 36, and training scenarios are the steam generator tube rupture and the loss of all feedwater. The number of scenarios is 18 each. From these emergency training records, averaged step performance time data for 30 steps are retrieved.As the results, the SC measure shows statistically meaningful correlation with averaged step performance time data. In addition, since it is observed that the SC measure seems to have the procedure independent property (i.e. steps that have similar SC scores, whether they are included in different procedures or not, would have similar step performance time), it can be concluded that the SC measure can represent the complexity of steps included in EOPs.     

The step complexity measure for emergency operating procedures — comparing with simulation data

In complex systems, such as nuclear power plants (NPPs) or airplane control systems, human errors play a major role in many accidents. Therefore, to prevent occurrences of accidents or to ensure system safety, extensive effort has been made to identify significant factors that cause human errors. According to related studies, written manuals or operating procedures are revealed as one of the most important factors, and complexity or understandability of a procedure is pointed out as one of the major reasons that make procedure-related human errors.Many qualitative checklists are suggested to evaluate emergency operating procedures (EOPs) of NPPs. However, since qualitative evaluations using checklists have some drawbacks, a quantitative measure that can quantify the complexity of EOPs is imperative to compensate for them.In order to quantify the complexity of EOPs, Park et al. suggested the step complexity (SC) measure to quantify the complexity of a step included in EOPs. In this paper, to ensure the appropriateness of the SC measure, SC scores are compared with averaged step performance time data obtained from emergency training records. The total number of available records is 36, and training scenarios are the loss of coolant accident and the excess steam dump event. The number of scenario is 18 each. From these emergency training records, step performance time data for 39 steps are retrieved, and they are compared with estimated SC scores of them. In addition, several questions that are needed to clarify the appropriateness of the SC measure are also discussed.As a result, it was observed that estimated SC scores and step performance time data have a statistically meaningful correlation. Thus, it can be concluded that the SC measure can quantify the complexity of steps included in EOPs.     

Identifying cognitive complexity factors affecting the complexity of procedural steps in emergency operating procedures of a nuclear power plant

In complex systems such as a nuclear and chemical plant, it is well known that the provision of understandable procedures that allow operators to clarify what needs to be done and how to do it is one of the requisites to secure their safety.As a previous study in providing understandable procedures, the step complexity (SC) measure that can quantify the complexity of procedural steps in emergency operating procedures (EOPs) of a nuclear power plant (NPP) was suggested. However, the necessity of additional complexity factors that can consider a cognitive aspect in evaluating the complexity of procedural steps is raised.To this end, the comparisons between operators' performance data measured by the form of a step performance time with their behavior in carrying out the prescribed activities of procedural steps are conducted in this study. As a result, two kinds of complexity factors (the abstraction level of knowledge and the level of engineering decision) that could affect an operator's cognitive burden are identified.Although a well-designed experiment is indispensable for confirming the appropriateness of the additional complexity factors, it is strongly believed that the change of operators' performance data can be more authentically explained if the additional complexity factors are taken into consideration.     

Task analysis of discrete and continuous skills: a dual methodology approach to human skills capture for automation

There is a growing requirement within the field of intelligent automation for a formal methodology to capture and classify explicit and tacit skills deployed by operators during complex task performance. This paper describes the development of a dual methodology approach which recognises the inherent differences between continuous tasks and discrete tasks and which proposes separate methodologies for each. Both methodologies emphasise capturing operators’ physical, perceptual, and cognitive skills, however, they fundamentally differ in their approach. The continuous task analysis recognises the non-arbitrary nature of operation ordering and that identifying suitable cues for subtask is a vital component of the skill. Discrete task analysis is a more traditional, chronologically ordered methodology and is intended to increase the resolution of skill classification and be practical for assessing complex tasks involving multiple unique subtasks through the use of taxonomy of generic actions for physical, perceptual, and cognitive actions.     

Collaboration and complexity: an experiment on the effect of multi-actor coupled design

Design of complex systems requires collaborative teams to overcome limitations of individuals; however, teamwork contributes new sources of complexity related to information exchange among members. This paper formulates a human subjects experiment to quantify the relative contribution of technical and social sources of complexity to design effort using a surrogate task based on a parameter design problem. Ten groups of 3 subjects each perform 42 design tasks with variable problem size and coupling (technical complexity) and team size (social complexity) to measure completion time (design effort). Results of a two-level regression model replicate past work to show completion time grows geometrically with problem size for highly coupled tasks. New findings show the effect of team size is independent from problem size for both coupled and uncoupled tasks considered in this study. Collaboration contributes a large fraction of total effort, and it increases with team size: about 50–60 % of time and 70–80 % of cost for pairs and 60–80 % of time and 90 % of cost for triads. Conclusions identify a role for improved design methods and tools to anticipate and overcome the high cost of collaboration.     

An ergonomics study of computerized emergency operating procedures: Presentation style, task complexity, and training level

Emergency operating procedures (EOPs) are widely used in nuclear power plants (NPPs). With the development of information technology, computerized EOPs are taking the place of paper-based ones. Unlike paper-based EOPs, the industrial practice of computerized EOPs is still quite limited. Ergonomics issues of computerized EOPs have not been studied adequately. This study focuses on the effects of EOP presentation style, task complexity, and training level on the performance of the operators in the execution of computerized EOPs. One simulated computerized EOP system was developed to present two EOPs, each with different task complexity levels, by two presentation styles (Style A: one- and two-dimensional flowcharts combination; Style B: two-dimensional flowchart and success logic tree combination). Forty subjects participated in the experiment of EOP execution using the simulated system. Statistical analysis of the experimental results indicates that: (1) complexity, presentation style, and training level all can significantly influence the error rate. High-complexity tasks and lack of sufficient training may lead to a higher error rate. Style B caused a significantly higher error rate than style A did in the skilled phase. So the designers of computerized procedures should take the presentation styles of EOPs into account. (2) Task complexity and training level can significantly influence operation time. No significant difference was found in operation time between the two presentation styles. (3) Training level can also significantly influence the subjective workload of EOPs operations. This implies that adequate training is very important for the performance of computerized EOPs even if emergency responses with computerized EOPs are much more simple and easy than that with paper-based EOPs.     

Cognitive Task Performance Time And Accuracy In Supervisory Process Control

Research is reported on several types of cognitive tasks that are performed by industrial process controllers. Statistics of performance time and accuracy are examined as a function of task repetitions and complexity variables. Performance plateaus were found in these statistics when the tasks were held at an average complexity level. A distribution of performance time is identified. As the reference and/or field size increased or other complexity variables changed, the means and variances of performance time for within- and between-persons changed with those variables in different ways with the type of cognitive task. Some of these complexity variables accounted for about 90 percent of the time variation within a typical subject. Accuracy was found to be about 95 percent, varying slightly by the type of task but not with the complexity variables.     

基于近红外光学脑成像实验的轻型装配作业的中断效应研究

中断任务的复杂度和相似性对作业者操作绩效和脑认知负荷有不同程度影响。通过设计不同复杂度和不同相似性中断情景下的乐高模型模拟轻型装配实验，记录作业者操作绩效指标，并利用近红外光学脑成像设备监测其左右脑前额叶含氧血红蛋白浓度变化，分析中断类型对轻型装配作业者操作绩效和脑认知负荷的影响规律及其内在机理。结果表明，中断任务的复杂度不会导致作业者操作绩效和脑认知负荷出现显著差异，相似中断会导致作业者操作绩效下降，不相似中断会导致作业者脑认知负荷增加。研究发现了作业者应对中断时的注意力资源分配规律，并发展了动作技能型任务注意力焦点转换过程中的工作记忆理论，从而指导轻型装配企业合理制定标准作业时间和中断管理策略，降低作业者脑认知负荷，以实现可持续高效生产。     

A case for information theory-based modelling of human multitasking performance

Technological advances have led to increased prevalence of human multitasking. Traditionally, multitasking performance is measured for each individual task, utilising multiple performance metrics. Recently, the human operator informatic model (HOIM) has been developed as a quantitative model of human–machine interaction capable of measuring multitasking performance with a single metric. In this study, we investigated the performance effects of both a different number of tasks and five different task combinations while maintaining a constant level of overall difficulty. Results indicate that, at a constant difficulty level, neither number of tasks nor task combination significantly affects multitasking performance, indicating the reliability of input baud rate as a measure of system complexity. No task interaction effects were found among the different tasks. This study demonstrates the application of information theory for modelling of human multitasking performance and makes a case for the use of the HOIM in future studies of human information processing.     

Automation-induced complacency for monitoring highly reliable systems: the role of task complexity,system experience,and operator trust

The increase in quantity and complexity of advanced automated systems has generated new concerns surrounding automation-induced complacency, or the difficulties operators have monitoring the status of automated systems. The present investigation consists of two studies that assessed the impact of system reliability, monitoring complexity, operator trust, and system experience on automation-induced complacency. In both studies, participants operated a manually controlled flight task while monitoring several simulated aircraft displays for failures. The ability of operators to detect a single automation failure over three experimental sessions was also assessed. Results indicated that realistic levels of system reliability severely impaired an operator's ability to monitor effectively. Further, as system experience increased, operator monitoring performance declined. The results also indicated that the complexity of the monitoring task heavily influenced operator monitoring, with poorer performance associated with more cognitively demanding tasks. Finally, results from both studies indicated that operator trust increased and monitoring performance decreased as a function of increasing system reliability. These results suggest that for highly reliable systems, increasing task complexity and extensive experience may severely impair an operator's ability to monitor for unanticipated system states.     

Chasing the silver bullet: measuring driver fatigue using simple and complex tasks

Driver fatigue remains a significant cause of motor-vehicle accidents worldwide. New technologies are increasingly utilised to improve road safety, but there are no effective on-road measures for fatigue. While simulated driving tasks are sensitive, and simple performance tasks have been used in industrial fatigue management systems (FMS) to quantify risk, little is known about the relationship between such measures. Establishing a simple, on-road measure of fatigue, as a fitness-to-drive tool, is an important issue for road safety and accident prevention, particularly as many fatigue related accidents are preventable. This study aimed to measure fatigue-related performance decrements using a simple task (reaction time - RT) and a complex task (driving simulation), and to determine the potential for a link between such measures, thus improving FMS success. Fifteen volunteer participants (7 m, 8 f) aged 22-56 years (mean 33.6 years), underwent 26 h of supervised wakefulness before an 8h recovery sleep opportunity. Participants were tested using a 30-min interactive driving simulation test, bracketed by a 10-min psychomotor vigilance task (PVT) at 4, 8, 18 and 24h of wakefulness, and following recovery sleep. Extended wakefulness caused significant decrements in PVT and driving performance. Although these measures are clearly linked, our analyses suggest that driving simulation cannot be replaced by a simple PVT. Further research is needed to closely examine links between performance measures, and to facilitate accurate management of fitness to drive, which requires more complex assessments of performance than RT alone.     

Using Rasch scaled stage scores to validate orders of hierarchical complexity of balance beam task sequences

These studies examine the relationship between the analytic basis underlying the hierarchies produced by the Model of Hierarchical Complexity and the probabilistic Rasch scales that places both participants and problems along a single hierarchically ordered dimension. A Rasch analysis was performed on data from the balance-beam task series. This yielded scaled stage of performance for each of the items. The items formed a series of clusters along this same dimension, according to their order of hierarchical complexity. We sought to ascertain whether there was a significant relationship between the order of hierarchical complexity (a task property variable) of the tasks and the corresponding Rasch scaled difficulty of those same items (a performance variable). It was found that The Model of Hierarchical Complexity was highly accurate in predicting the Rasch Stage scores of the performed tasks, therefore providing an analytic and developmental basis for the Rasch scaled stages.     

The impact of secondary task cognitive processing demand on driving performance

Crash causation research has identified inattention as a major source of driver error leading to crashes. The series of experiments presented herein investigate the characteristics of an in-vehicle information system (IVIS) task that could hinder driving performance due to uncertainty buildup and cognitive capture. Three on-road studies were performed that used instrumented passenger and tractor-trailer vehicles to obtain real-world driving performance data. Participants included young, middle-aged, and older passenger vehicle drivers and middle-aged and older commercial vehicle operators. While driving, they were presented with IVIS tasks with various information densities, decision-making elements, presentation formats, and presentation modalities (visual or auditory). The experiments showed that, for both presentation modalities, the presence of multiple decision-making elements in a task had a substantial negative impact on driving performance of both automobile drivers and truck drivers when compared to conventional tasks or tasks with only one decision-making element. The results from these experiments can be used to improve IVIS designs, allowing for potential IVIS task phenomena such as uncertainty buildup and cognitive capture to be avoided.     

The role of reward and effort over time in task switching

When multitasking in a demanding environment, operators strategically switch between tasks. Two influences on this multi-task management behaviour are the perceived rewards gained from performing a task, and the perceived effort a task requires in order to be completed. Proportionally, reward over effort is ‘rate of return’, expressing that either reward or effort changing over time spent performing a task, may influence multitasking behaviour. In the current experiment, two of four tasks within a multitasking paradigm provided constant or diminishing reward and required either constant or increasing effort in their performance. For on-going tasks, decreasing reward and increasing effort required both increased the probability of a task switch. The theory contributed here supports the study of multi-task management, and task switching model development. We discuss each in the context of safety-critical applications.