Effects of training on short- and long-term skill retention in a complex multiple-task environment

The paper reports the results of an experiment on the performance and retention of a complex task. This was a computer-based simulation of the essential elements of a spacecraft's life support system. It allowed the authors to take a range of measures, including primary and secondary task performance, system intervention and information sampling strategies, mental model structure, and subjective operator state. The study compared the effectiveness of two methods of training, based on low level (procedure-based) and high level (system-based) understanding. Twenty-five participants were trained extensively on the task, then given a 1-h testing session. A second testing session was carried out 8 months after the first (with no intervening practice) with 17 of the original participants. While training had little effect on control performance, there were considerable effects on system management strategies, as well as in structure of operator's mental model. In the second testing session, the anticipated general performance decrement did not occur, though for complex faults there was an increase in selectivity towards the primary control task. The relevance of the findings for training and skill retention in real work environments is discussed in the context of a model of compensatory control.     

Cognitive diversity and team performance in a complex multiple task environment

This article examines the multiple effects of cognitive diversity in teams operating complex human-machine-systems. The study employed a PC-based multiple-task environment, called the Cabin Air Management System, which models a process control task in the operational context of a spacecraft's life support system. Two types of cognitive diversity were examined: system understanding and team specialization. System understanding referred to the depth of understanding team members were given during training (low-level procedure-oriented vs. high level knowledge-oriented training). Team specialization referred to the degree to which knowledge about system fault scenarios was distributed between team members (specialized vs. non-specialized). A total of 72 participants took part in the study. After having received 4.5 h of training on an individual basis, participants completed a 1-h experimental session, in which they worked in two-person teams on a series of fault scenarios of varying difficulty. Measures were taken of primary and secondary task performance, system intervention and information sampling strategies, system knowledge, subjective operator state, communication patterns and conflict. The results provided evidence for the benefits of cognitive diversity with regard to system understanding. This manifested itself in better primary task performance and more efficient manual system control. No advantages were found for cognitive diversity with regard to specialization. There was no effect of cognitive diversity on intra-team conflict, with conflict levels generally being very low. The article concludes with a discussion of the implications of the findings for the engineering of cognitive diversity in teams operating complex human-machine-systems.     

Cognitive diversity and team performance in a complex multiple task environment

This article examines the multiple effects of cognitive diversity in teams operating complex human-machine-systems. The study employed a PC-based multiple-task environment, called the Cabin Air Management System, which models a process control task in the operational context of a spacecraft's life support system. Two types of cognitive diversity were examined: system understanding and team specialization. System understanding referred to the depth of understanding team members were given during training (low-level procedure-oriented vs. high level knowledge-oriented training). Team specialization referred to the degree to which knowledge about system fault scenarios was distributed between team members (specialized vs. non-specialized). A total of 72 participants took part in the study. After having received 4.5 h of training on an individual basis, participants completed a 1-h experimental session, in which they worked in two-person teams on a series of fault scenarios of varying difficulty. Measures were taken of primary and secondary task performance, system intervention and information sampling strategies, system knowledge, subjective operator state, communication patterns and conflict. The results provided evidence for the benefits of cognitive diversity with regard to system understanding. This manifested itself in better primary task performance and more efficient manual system control. No advantages were found for cognitive diversity with regard to specialization. There was no effect of cognitive diversity on intra-team conflict, with conflict levels generally being very low. The article concludes with a discussion of the implications of the findings for the engineering of cognitive diversity in teams operating complex human-machine-systems.     

The predictive qualities of operator characteristics for process control performance: the influence of personality and cognitive variables

This article examines the relationship between operator characteristics and process control performance. Thirty-nine trainee operators participated in a 4-h training session of a simulated process control task and a testing session in which various system faults had to be managed. Cognitive ability, cognitive flexibility, self-efficacy and personality traits were measured as operator characteristics. Cognitive ability related positively to system control performance but not to diagnostic performance. Participants with low cognitive flexibility performed best on system control, whereas participants with high cognitive flexibility performed best on diagnostic performance. A hierarchical regression analysis revealed that cognitive ability, cognitive flexibility and declarative knowledge accounted for about 30% of the variability of system control. The findings suggest that consideration of cognitive ability and cognitive flexibility be increased in personnel selection for complex work environments.     

Modeling and analysis of a dynamic judgment task using a lens modelapproach

We investigated performance in a complex, dynamic decision-making task. Individual judgment performance in a command and control environment was modeled as linear combinations of environmental cue values using a lens model approach. Examination of the judgment models indicated that participants had similar judgment policies, while correlational and error analyzes indicated that performance differences were due to participants' abilities to execute judgment strategies rather than their knowledge of the task environment. The research demonstrated how the lens model approach can be extended to account for dynamic aspects of decision making in complex environments, through the use of individual, time dependent environmental models for each participant. Additionally, the research showed that a lens model approach is useful for characterizing factors in individual performance in complex judgment tasks. Empirically, this research suggests that training on task knowledge should be supplemented by training which focuses on the consistent execution of judgment strategies     

The effects of heuristic rule training on operator performance in a simulated process control environment

In complex work environments, the occurrence of novel system states represents a particular challenge for the design of training. This article is concerned with the use of heuristic rules to prepare operators for the management of unfamiliar fault states. An experiment was carried out to examine the effects of heuristic rule training on operator performance and system management behaviour. Thirty-nine trainee operators from the chemical industry took part in the study. They were trained for 4 h on a PC-based simulation of a process control task. Operators in the experimental group received training on heuristic rules while operators in the control group did not. One week later the operators participated in a 70-min testing session. While the results showed that heuristic rules training led to better diagnostic performance, it was also associated with increased operator fatigue and impaired secondary task performance. The implications of the findings for using heuristic rule training are discussed.     

Situational awareness ability and cognitive skills training in a complex real-world task

Successful performance in complex dynamic environments depends on domain-dependent factors, such as situational awareness (SA). Underlying SA in a domain are domain-independent cognitive abilities in perception, memory, attention and executive control. Individuals with lower underlying ability perform relatively poorly in complex dynamic real-world tasks. The first experiment examined whether cognitive skills training could overcome limitations in underlying SA ability that impact on complex dynamic task performance. Participants were taught a mix of cognitive management strategies (e.g. divided and focused attention and visual search) in a simulated air traffic control task. A second experiment investigated the link between underlying SA ability, TRACON and SAGAT, a widely used measure of domain-specific SA. In a third experiment, the focus was on encouraging participants to plan ahead and consider the interrelations of elements (aircraft) in the environment. Whilst both training methods ameliorated the negative impact that lower SA ability had on complex dynamic task performance, the results of the third study indicated that this may have been achieved through improved planning behaviour. Finally, participants with higher underlying SA ability performed well irrespective of training condition.     

Situational awareness ability and cognitive skills training in a complex real-world task

Successful performance in complex dynamic environments depends on domain-dependent factors, such as situational awareness (SA). Underlying SA in a domain are domain-independent cognitive abilities in perception, memory, attention and executive control. Individuals with lower underlying ability perform relatively poorly in complex dynamic real-world tasks. The first experiment examined whether cognitive skills training could overcome limitations in underlying SA ability that impact on complex dynamic task performance. Participants were taught a mix of cognitive management strategies (e.g. divided and focused attention and visual search) in a simulated air traffic control task. A second experiment investigated the link between underlying SA ability, TRACON and SAGAT, a widely used measure of domain-specific SA. In a third experiment, the focus was on encouraging participants to plan ahead and consider the interrelations of elements (aircraft) in the environment. Whilst both training methods ameliorated the negative impact that lower SA ability had on complex dynamic task performance, the results of the third study indicated that this may have been achieved through improved planning behaviour. Finally, participants with higher underlying SA ability performed well irrespective of training condition.     

The effects of heuristic rule training on operator performance in a simulated process control environment

In complex work environments, the occurrence of novel system states represents a particular challenge for the design of training. This article is concerned with the use of heuristic rules to prepare operators for the management of unfamiliar fault states. An experiment was carried out to examine the effects of heuristic rule training on operator performance and system management behaviour. Thirty-nine trainee operators from the chemical industry took part in the study. They were trained for 4 h on a PC-based simulation of a process control task. Operators in the experimental group received training on heuristic rules while operators in the control group did not. One week later the operators participated in a 70-min testing session. While the results showed that heuristic rules training led to better diagnostic performance, it was also associated with increased operator fatigue and impaired secondary task performance. The implications of the findings for using heuristic rule training are discussed.     

A neurophysiological approach to assess training outcome under stress: A virtual reality experiment of industrial shutdown maintenance using Functional Near-Infrared Spectroscopy (fNIRS)

Shutdown maintenance, i.e., turning off a facility for a short period for renewal or replacement operations is a highly stressful task. With the limited time and complex operation procedures, human stress is a leading risk. Especially shutdown maintenance workers often need to go through excessive and stressful on-site trainings to digest complex operation information in limited time. The challenge is that workers’ stress status and task performance are hard to predict, as most trainings are only assessed after the shutdown maintenance operation is finished. A proactive assessment or intervention is needed to evaluate workers’ stress status and task performance during the training to enable early warning and interventions. This study proposes a neurophysiological approach to assess workers’ stress status and task performance under different virtual training scenarios. A Virtual Reality (VR) system integrated with the eye-tracking function was developed to simulate the power plant shutdown maintenance operations of replacing a heat exchanger in both normal and stressful scenarios. Meanwhile, a portable neuroimaging device – Functional Near-Infrared Spectroscopy (fNIRS) was also utilized to collect user’s brain activities by measuring hemodynamic responses associated with neuron behavior. A human–subject experiment (n = 16) was conducted to evaluate participants’ neural activity patterns and physiological metrics (gaze movement) related to their stress status and final task performance. Each participant was required to review the operational instructions for a pipe maintenance task for a short period and then perform the task based on their memory in both normal and stressful scenarios. Our experiment results indicated that stressful training had a strong impact on participants’ neural connectivity patterns and final performance, suggesting the use of stressors during training to be an important and useful control factors. We further found significant correlations between gaze movement patterns in review phase and final task performance, and between the neural features and final task performance. In summary, we proposed a variety of supervised machine learning classification models that use the fNIRS data in the review session to estimate individual’s task performance. The classification models were validated with the k-fold (k = 10) cross-validation method. The Random Forest classification model achieved the best average classification accuracy (80.38%) in classifying participants’ task performance compared to other classification models. The contribution of our study is to help establish the knowledge and methodological basis for an early warning and estimating system of the final task performance based on the neurophysiological measures during the training for industrial operations. These findings are expected to provide more evidence about an early performance warning and prediction system based on a hybrid neurophysiological measure method, inspiring the design of a cognition-driven personalized training system for industrial workers.     

Performance in a complex multiple-task environment during a laboratory-based simulation of occasional night work

A study was carried out to examine the impact of occasional night work on simulated process control using a complex task environment. The 21 student participants were tested during 2 6-hr simulated shifts (daytime and night). In addition to the primary system management task, the simulation allowed measurement of fault diagnosis behavior, monitoring and control actions, and two secondary tasks--alarm reaction time and system status checks (prospective memory)--as well as subjective state. Consistent with predictions from compensatory control theory, night work did not impair system performance, although monitoring and control were reduced (supported by subjective reports of increased use of risky "corner-cutting" strategies). Secondary tasks showed an increase in alarm reaction time during night work, but there was no effect on prospective memory and no clear pattern of change in subjective state. Actual or potential applications of this research include the design of complex systems for nighttime operation.     

Learning while multitasking: short and long-term benefits of brain stimulation

Abstract

We employed a simulated production task that mimics the real-world skill acquisition required of operators working in control rooms of power plants to assess short and long-term effects of transcranial random noise stimulation (tRNS). tRNS has shown potential for enhancing learning and performance of cognitive skills. Forty subjects (24 female) learned how to execute the simulated production task during the training phase and were required to perform a secondary task during the skill acquisition phase while they received active (12?min) or sham tRNS on DLPFC. After 2 weeks they had to recall the task again without any stimulation. The results demonstrate that tRNS promoted better multitasking as reflected by better performance in a secondary task during and immediately after tRNS. However, 2 weeks later, beneficial effect of tRNS on retention was moderated by general mental ability. Particularly, tRNS benefited those with lower general mental ability.

Practitioner summary: By using a simulated production task, we assessed the effects of tRNS on learning and skill retention. The study indicates that neurostimulation can enhance the learning of multiple complex tasks. Moreover, it shows that retention of those tasks can be supported by neurostimulation, especially for those with lower general mental ability.     

Adaptive interaction in a 3D product structure browsing system for maintenance training

The traditional maintenance training method with a physical model of the product is costly and inconvenient. Computer‐aided instruction (CAI) technology along with multimedia can provide much help in the training but provides limited interaction between the user and the system. In this article, a 3D model‐based product structure browsing system for maintenance training, CAMT, is developed for complex products adopting desktop virtual reality technology. To improve training performance, the interaction between the trainee and the CAMT system is enhanced by adaptive change of the zoom level, mouse sensitivity, and rotation origin. Details about the implementation of adaptive interaction are discussed. Experiments were conducted to test the effectiveness of this adaptive interaction. Seventy participants were arranged randomly into two groups assigned to perform product structure learning tasks using software with or without adaptive interaction functions. Statistical analysis showed that there were significant differences between the groups in task time, operation convenience, and learning satisfaction. Most participants preferred to use the system with adaptive interaction. It may be concluded that using adaptive interaction with maintenance training systems can significantly improve the usability of the systems and the efficiency of interactive learning. Although adaptive interaction has obvious advantages, our experiment also suggested that it is not a good idea to provide only the adaptive interaction mode. It is better to set adaptive interaction as the default mode but also to provide the possibility for a user to switch to a mode without adaptive interaction because a static view scope is also helpful to learn the 3D structure of a complex product. © 2007 Wiley Periodicals, Inc.     

Applying minimalist design principles to the problem of computer anxiety

Computer anxiety is one possible barrier to acquisition of computer skills. To test whether an instructional intervention could decrease computer anxiety while training subjects in basic word-processing skills, instruction was developed according to the minimalist design principles of John Carroll and was compared with a commercially available instructional unit.The design of the experiment was a repeated measures control group with random assignment of subjects. Two instructional sessions were held, separated by 1 month. Subjects were pre- and posttested with respect to computer anxiety during each instructional session. Subjects worked on two similar word-processing tasks, a practice and a transfer task, during the first instructional session and on another similar word-processing task during the second instructional session. Subjects' performances on the word-processing tasks were scored according to a performance checklist. For the experimental group, there was a statistically significant decrease in computer anxiety immediately following the first instructional treatment. For the control group, there was also a decrease in computer anxiety, but this occurred over the 1-month experimental time-frame. Both experimental and control treatments were equally effective in training subjects to perform the word-processing tasks.     

Does training in the cold improve cold performance?

Despite the well-described physiological response to cold, little is known about how to best train persons to perform motor tasks in cold conditions. It is unclear if principles of training specificity would apply to motor skill training when cold because cold exposure reduces tactile sensitivity and cognitive function, which may reduce rather than enhance training efficacy. The purpose of this study is to determine whether training in the cold facilitates performance in the cold. To do this, we investigated the effect of cold or thermoneutral training on performance of the Grooved Pegboard Task. Twenty persons (11M, 9F, mean age 31.2 ± 5.44 years) visited the lab on two separate days and were randomly assigned to either a cold (5M, 5F) or thermoneutral (6M, 4F) training group. On day one, participants were tested at baseline and then performed 20 repetitions of the task according to their respective group assignment. Prior to each task repetition during training, the cold training group immersed their hand in cold water (2 °C) while the thermoneutral group immersed their hand in thermoneutral water (34 °C). Following training, participants were tested once again (immediate retention test). During testing, participants performed the task twice; once after immersing their hands in cold water and once after immersing their hands in thermoneutral water. On day two, delayed retention tests were performed. Time to completion and number of errors were recorded during testing. There were no differences in time to completion at delayed retention tests between thermoneutral and cold-trained groups (p = 0.434). The incidence of errors was significantly less in the cold-trained group than the thermoneutral training group at delayed retention testing (p = 0.035). The main finding of this study was that dexterity but not speed was improved by cold training. Further research is required but the findings presented here suggest that industries that require motor tasks in the cold could benefit from training employees in cold conditions.     

Effects of different physical workload parameters on mental workload and performance

The design and evaluation of an occupational task should include an assessment of mental workload, since excessive levels of mental workload can cause errors or delayed information processing. Physically demanding work that is performed concurrently with a cognitive task may impact mental workload by impairing mental processing or decreasing performance. The primary objective of this study was to determine whether there is a differential effect of various types of physical activity on both mental workload and cognitive performance. Objective and subjective assessment tools (heart rate variability and visual analog scale) were used as indicators of mental workload, while correct responses during an arithmetic task reflected levels of performance. Thirty participants (ages 18-24 years) performed a combination of tasks inducing both physical and mental workload. Type of physical effort, frequency of movement, and force exertion level were manipulated to alter the workload associated with the physical activity. Changes in subjective ratings generally corresponded to changes in both performance on the arithmetic task and objective mental workload assessment. Some discrepancies occurred at the highest physical force exertion level as participants perceived an increase in effort to maintain the same level of performance. Further research is needed to determine the force exertion threshold, beyond which the physical effort required interferes with mental workload and/or cognitive performance.

Relevance to industry

Technological advancements have increased the requirement for many workers to execute cognitive tasks concurrently with physical activity. When designing and evaluating such situations it is important to determine the interactive effects of these activities. A simple, uni-dimensional tool is suggested as a screening tool to identify situations requiring excessive or increased mental workload that many degrade performance or place additional stress on the individual.     

The impact of knowledge representation on cognitive‐oriented task performance

This research examines the impact of training style and operator individual differences on the task representation developed, automatized task performance, and controlled task performance. Results indicate that performance on relatively straightforward repetitive tasks usually associated with automatization is influenced by training style and the mental task representation held by operators. Also, domain representation is a significant determinant of performance on complex cognitive‐oriented tasks requiring controlled processes. Therefore, the task representation is identified as a high‐level performance determinant for both simple and complex task performance. No effect for training style or individual differences was found. It is concluded that training programs for systems requiring human‐computer interaction must account for this factor in order to facilitate the learning process and enhance task performance.     

Dynamic problem selection in air traffic control training: a comparison between performance,mental effort and mental efficiency

The differential effects on training and training outcome of four methods of problem selection were investigated in a computer-based training for air traffic control. In one method, training problems were given to the participants in a fixed predetermined sequence, from simple to complex. In the other methods, problems were selected dynamically, based on three different learner variables. These were mental effort, performance and mental efficiency, which is a variable that combines mental effort and performance measures to determine problem efficiency. After the training, transfer was measured. The hypothesis that dynamic problem selection would lead to more efficient training than non-dynamic problem selection was confirmed. The second hypothesis, that dynamic problem selection based on mental efficiency would lead to more efficient training and better transfer than dynamic problem selection based on performance or mental effort alone was not supported. However, the efficiency measures of the three variables were distorted by the differential effects of these variables on the acquisition phase. A possible explanation for the results is that selection based on performance stimulates rule automation, whereas selection based on mental effort or mental efficiency leads to schema acquisition.     

Comparison of a brain-based adaptive system and a manual adaptable system for invoking automation

OBJECTIVE: Two experiments are presented examining adaptive and adaptable methods for invoking automation. BACKGROUND: Empirical investigations of adaptive automation have focused on methods used to invoke automation or on automation-related performance implications. However, no research has addressed whether performance benefits associated with brain-based systems exceed those in which users have control over task allocations. METHOD: Participants performed monitoring and resource management tasks as well as a tracking task that shifted between automatic and manual modes. In the first experiment, participants worked with an adaptive system that used their electroencephalographic signals to switch the tracking task between automatic and manual modes. Participants were also divided between high- and low-reliability conditions for the system-monitoring task as well as high- and low-complacency potential. For the second experiment, participants operated an adaptable system that gave them manual control over task allocations. RESULTS: Results indicated increased situation awareness (SA) of gauge instrument settings for individuals high in complacency potential using the adaptive system. In addition, participants who had control over automation performed more poorly on the resource management task and reported higher levels of workload. A comparison between systems also revealed enhanced SA of gauge instrument settings and decreased workload in the adaptive condition. CONCLUSION: The present results suggest that brain-based adaptive automation systems may enhance perceptual level SA while reducing mental workload relative to systems requiring user-initiated control. APPLICATION: Potential applications include automated systems for which operator monitoring performance and high-workload conditions are of concern.     

Adaptability of training in simulated process control: knowledge- versus rule-based guidance under task changes and environmental stress

OBJECTIVE: The study examined the adaptability of different types of process control training across changes in task and environmental stress. BACKGROUND: The literature on training leads us to expect greater flexibility for system-based training, as opposed to procedure-based training. However, the stress literature suggests that knowledge-based strategies (making use of executive control) may be more vulnerable under stress conditions. METHOD: Two groups were given 6 hr of training on the Cabin Air Management System (CAMS), a complex, multilevel, PC-based process control task, emphasizing either system knowledge or use of procedures. They were then required to carry out the task for 3 hr (with noise during the middle 1 hr) across a range of both familiar and unfamiliar fault scenarios. RESULTS: For the primary control task, the system-trained group performed better, especially for less familiar and complex faults. However, for lower priority tasks requiring executive control, procedure-trained operators performed better and were less impaired by noise. CONCLUSION: System training was more effective for managing unexpected task events, whereas procedural training was better under noise. The results are interpreted in terms of the rationale for instructing operators in the range of strategies required for effective process skills in complex work environments. APPLICATION: Training methodologies for safety critical applications should aim to develop skill in the use of both procedural and system knowledge strategies. Operators should be trained in the most effective deployment of these strategies during unfamiliar task events and environmental stress and given stress exposure training.