The interaction of drill and practice and error training with individual differences

This article examines the question of whether the effectiveness of training methods in vocational training settings is modified by trainee characteristics. General mental abilities, cognitive style and conscientiousness were measured as person variables that were expected to influence the effectiveness of training methods. Being trained on a simulated process control environment as an example for a complex technological task for 5 h, N = 38 participants received either drill & practice (D&P) or error training (ET) as two distinct training methods. A 70-minute transfer test followed 9 weeks later, in which the trainees had to manage familiar (near transfer) as well as unfamiliar fault states (far transfer). The results showed that high-ability trainees benefited more from D&P than from ET, whereas low-ability trainees benefited equally well from either training method. Furthermore, participants with a highly flexible cognitive style benefited more from D&P rather than ET. Also participants low in conscientiousness profited more from D&P. The findings are indicative of the tentative benefits of using learner-tailored training methods, suggesting that this person-centered approach is worthy of further exploration.     

Experience of automation failures in training: effects on trust,automation bias,complacency and performance

This work examined the effects of operators’ exposure to various types of automation failures in training. Forty-five participants were trained for 3.5 h on a simulated process control environment. During training, participants either experienced a fully reliable, automatic fault repair facility (i.e. faults detected and correctly diagnosed), a misdiagnosis-prone one (i.e. faults detected but not correctly diagnosed) or a miss-prone one (i.e. faults not detected). One week after training, participants were tested for 3?h, experiencing two types of automation failures (misdiagnosis, miss). The results showed that automation bias was very high when operators trained on miss-prone automation encountered a failure of the diagnostic system. Operator errors resulting from automation bias were much higher when automation misdiagnosed a fault than when it missed one. Differences in trust levels that were instilled by the different training experiences disappeared during the testing session.

Practitioner Summary: The experience of automation failures during training has some consequences. A greater potential for operator errors may be expected when an automatic system failed to diagnose a fault than when it failed to detect one.     

Adaptability of training in simulated process control: knowledge- versus rule-based guidance under task changes and environmental stress

OBJECTIVE: The study examined the adaptability of different types of process control training across changes in task and environmental stress. BACKGROUND: The literature on training leads us to expect greater flexibility for system-based training, as opposed to procedure-based training. However, the stress literature suggests that knowledge-based strategies (making use of executive control) may be more vulnerable under stress conditions. METHOD: Two groups were given 6 hr of training on the Cabin Air Management System (CAMS), a complex, multilevel, PC-based process control task, emphasizing either system knowledge or use of procedures. They were then required to carry out the task for 3 hr (with noise during the middle 1 hr) across a range of both familiar and unfamiliar fault scenarios. RESULTS: For the primary control task, the system-trained group performed better, especially for less familiar and complex faults. However, for lower priority tasks requiring executive control, procedure-trained operators performed better and were less impaired by noise. CONCLUSION: System training was more effective for managing unexpected task events, whereas procedural training was better under noise. The results are interpreted in terms of the rationale for instructing operators in the range of strategies required for effective process skills in complex work environments. APPLICATION: Training methodologies for safety critical applications should aim to develop skill in the use of both procedural and system knowledge strategies. Operators should be trained in the most effective deployment of these strategies during unfamiliar task events and environmental stress and given stress exposure training.     

Use of genetic algorithms in training diagnostic rules for process fault diagnosis

The self learning of diagnostic rules can ease knowledge-acquisition effort, and it is more desirable in cases where experience about certain faults is not available. Applications of genetic algorithms to the self learning of diagnostic rules for a pilot-scale mixing process and a continuous stirred-tank reactor system are described in the paper. In this method, a set of training data, which could be obtained from simulations and/or from the recorded data of the previous operations of the real process, is required. The training data is divided into various groups corresponding to various faults and the normal operating condition. Corresponding to each fault, there is a group of initial rules which are coded into binary strings. These rules are evaluated by a genetic algorithm which contains the three basic operators, reproduction, crossover and mutation, and an added operator which preserves the best rule ever discovered. Through this biological-type evaluation, new fitted rules are discovered. The results demonstrate that diagnostic rules fitted with a given set of training data can be efficiently discovered through genetic learning, and, hence, that genetic algorithms provide a means for the automatic creation of rules from a set of training data. It is also demonstrated that bad training data and the inappropriate formulation of rules could degrade the performance of the learning system.     

The predictive qualities of operator characteristics for process control performance: the influence of personality and cognitive variables

This article examines the relationship between operator characteristics and process control performance. Thirty-nine trainee operators participated in a 4-h training session of a simulated process control task and a testing session in which various system faults had to be managed. Cognitive ability, cognitive flexibility, self-efficacy and personality traits were measured as operator characteristics. Cognitive ability related positively to system control performance but not to diagnostic performance. Participants with low cognitive flexibility performed best on system control, whereas participants with high cognitive flexibility performed best on diagnostic performance. A hierarchical regression analysis revealed that cognitive ability, cognitive flexibility and declarative knowledge accounted for about 30% of the variability of system control. The findings suggest that consideration of cognitive ability and cognitive flexibility be increased in personnel selection for complex work environments.     

Effects of training on short- and long-term skill retention in a complex multiple-task environment

The paper reports the results of an experiment on the performance and retention of a complex task. This was a computer-based simulation of the essential elements of a spacecraft's life support system. It allowed the authors to take a range of measures, including primary and secondary task performance, system intervention and information sampling strategies, mental model structure, and subjective operator state. The study compared the effectiveness of two methods of training, based on low level (procedure-based) and high level (system-based) understanding. Twenty-five participants were trained extensively on the task, then given a 1-h testing session. A second testing session was carried out 8 months after the first (with no intervening practice) with 17 of the original participants. While training had little effect on control performance, there were considerable effects on system management strategies, as well as in structure of operator's mental model. In the second testing session, the anticipated general performance decrement did not occur, though for complex faults there was an increase in selectivity towards the primary control task. The relevance of the findings for training and skill retention in real work environments is discussed in the context of a model of compensatory control.     

Situational awareness ability and cognitive skills training in a complex real-world task

Successful performance in complex dynamic environments depends on domain-dependent factors, such as situational awareness (SA). Underlying SA in a domain are domain-independent cognitive abilities in perception, memory, attention and executive control. Individuals with lower underlying ability perform relatively poorly in complex dynamic real-world tasks. The first experiment examined whether cognitive skills training could overcome limitations in underlying SA ability that impact on complex dynamic task performance. Participants were taught a mix of cognitive management strategies (e.g. divided and focused attention and visual search) in a simulated air traffic control task. A second experiment investigated the link between underlying SA ability, TRACON and SAGAT, a widely used measure of domain-specific SA. In a third experiment, the focus was on encouraging participants to plan ahead and consider the interrelations of elements (aircraft) in the environment. Whilst both training methods ameliorated the negative impact that lower SA ability had on complex dynamic task performance, the results of the third study indicated that this may have been achieved through improved planning behaviour. Finally, participants with higher underlying SA ability performed well irrespective of training condition.     

Situational awareness ability and cognitive skills training in a complex real-world task

Successful performance in complex dynamic environments depends on domain-dependent factors, such as situational awareness (SA). Underlying SA in a domain are domain-independent cognitive abilities in perception, memory, attention and executive control. Individuals with lower underlying ability perform relatively poorly in complex dynamic real-world tasks. The first experiment examined whether cognitive skills training could overcome limitations in underlying SA ability that impact on complex dynamic task performance. Participants were taught a mix of cognitive management strategies (e.g. divided and focused attention and visual search) in a simulated air traffic control task. A second experiment investigated the link between underlying SA ability, TRACON and SAGAT, a widely used measure of domain-specific SA. In a third experiment, the focus was on encouraging participants to plan ahead and consider the interrelations of elements (aircraft) in the environment. Whilst both training methods ameliorated the negative impact that lower SA ability had on complex dynamic task performance, the results of the third study indicated that this may have been achieved through improved planning behaviour. Finally, participants with higher underlying SA ability performed well irrespective of training condition.     

Effects of training on short- and long-term skill retention in a complex multiple-task environment

The paper reports the results of an experiment on the performance and retention of a complex task. This was a computer-based simulation of the essential elements of a spacecraft's life support system. It allowed the authors to take a range of measures, including primary and secondary task performance, system intervention and information sampling strategies, mental model structure, and subjective operator state. The study compared the effectiveness of two methods of training, based on low level (procedure-based) and high level (system-based) understanding. Twenty-five participants were trained extensively on the task, then given a 1-h testing session. A second testing session was carried out 8 months after the first (with no intervening practice) with 17 of the original participants. While training had little effect on control performance, there were considerable effects on system management strategies, as well as in structure of operator's mental model. In the second testing session, the anticipated general performance decrement did not occur, though for complex faults there was an increase in selectivity towards the primary control task. The relevance of the findings for training and skill retention in real work environments is discussed in the context of a model of compensatory control.     

基于化工仿真培训器专家评分系统的研究和开发

讨论了仿真培训专家评分系统的设计及实现。介绍了仿真培训系统评分方法的结构和实现方法,在现有基础上开发了的专家评分系统,并较详细的论述了专家评分系统的结构和功能。在装置的培训过程中该专家评分系统能够记录操作失误并提出建议,该系统的知识库可以根据不足及时获取知识,不断增加诊断能力,满足了工厂对工人培训和提高技能的需求。     

The effect of training systemic information on the retention of fault‐finding skills in manufacturing industries

This study examines how the process of skill retention (including both recall and reconstruction) can benefit from systemic training information that covers several structural, functional, physical (e.g., causal links), and supervisory features (e.g., system control and instrumentation). An experiment was carried out in which a group of participants who received systemic information was compared to a group trained in plant theory and to another three groups trained in heuristics that ranged in terms of high‐ and low‐level diagnostic information. Learning curves revealed that all groups achieved high accuracy scores in diagnosing faults in a distillation plant. Six weeks later, a retention test showed that the systemic group (S) had a better recall score than the groups trained in theory (T) and high‐level heuristics (HH). In addition, the S group had a better reconstruction score than the groups receiving the high‐ and low‐level heuristics (H+) and the low‐level diagnostic information (HL). An interesting finding was that high‐level diagnostic information seemed to support the reconstruction phase of the retention process, whereas low‐level information better supported the recall phase. Overall, it appeared that systemic information enabled trainees to acquire task cues to help recall and reconstruct diagnostic heuristics. © 2004 Wiley Periodicals, Inc. Hum Factors Man 14: 197–217, 2004.     

The effects of heuristic rule training on operator performance in a simulated process control environment

In complex work environments, the occurrence of novel system states represents a particular challenge for the design of training. This article is concerned with the use of heuristic rules to prepare operators for the management of unfamiliar fault states. An experiment was carried out to examine the effects of heuristic rule training on operator performance and system management behaviour. Thirty-nine trainee operators from the chemical industry took part in the study. They were trained for 4 h on a PC-based simulation of a process control task. Operators in the experimental group received training on heuristic rules while operators in the control group did not. One week later the operators participated in a 70-min testing session. While the results showed that heuristic rules training led to better diagnostic performance, it was also associated with increased operator fatigue and impaired secondary task performance. The implications of the findings for using heuristic rule training are discussed.     

A software for performance evaluation and comparison of people detection and tracking methods in video processing

Digital video content analysis is an important item for multimedia content-based indexing (MCBI), content-based video retrieval (CBVR) and visual surveillance systems. There are some frequently-used generic object detection and/or tracking (D&T) algorithms in the literature, such as Background Subtraction (BS), Continuously Adaptive Mean Shift (CMS), Optical Flow (OF) and etc. An important problem for performance evaluation is the absence of stable and flexible software for comparison of different algorithms. This software is able to compare them with the same metrics in real-time and at the same platform. In this paper, we have designed and implemented the software for the performance comparison and the evaluation of well-known video object D&T algorithms (for people D&T) at the same platform. The software works as an automatic and/or semi-automatic test environment in real-time, which uses the image and video processing essentials, e.g. morphological operations and filters, and ground-truth (GT) XML data files, charting/plotting capabilities and etc.     

The effects of heuristic rule training on operator performance in a simulated process control environment

In complex work environments, the occurrence of novel system states represents a particular challenge for the design of training. This article is concerned with the use of heuristic rules to prepare operators for the management of unfamiliar fault states. An experiment was carried out to examine the effects of heuristic rule training on operator performance and system management behaviour. Thirty-nine trainee operators from the chemical industry took part in the study. They were trained for 4 h on a PC-based simulation of a process control task. Operators in the experimental group received training on heuristic rules while operators in the control group did not. One week later the operators participated in a 70-min testing session. While the results showed that heuristic rules training led to better diagnostic performance, it was also associated with increased operator fatigue and impaired secondary task performance. The implications of the findings for using heuristic rule training are discussed.     

Who benefits from simulator training: Personality and heart rate variability in relation to situation awareness during navigation training

Objective

This study tested the relationship between individual differences and Situation Awareness (SA) during training in a navigation simulator.

Background

Simulators have become an important tool in the training and education of maritime personnel with a view to improving decisions and performance. There are a few, inconclusive empirical studies that have focused on mapping personality characteristics and their link to the ability to generate and maintain SA during simulator training.

Method

Thirty-six first-year students from the Royal Norwegian Navy Officer Candidate School participated in the study. SA was measured using both subjective and observer ratings.

Results

The results indicated that low scores on Neuroticism and high scores on Extraversion and Conscientiousness (resilient personality type) predicted both subjective and observer-rated SA. Furthermore, participants with high SA were able to modulate their Heart Rate Variability (HRV), with suppression of HRV during navigation training as well showing recovery of HRV.

Application

The potential applications of this research include the assessment of personality differences as a tool in selecting navigators, and the use of HRV as an objective index of adaptability to environmental demands.     

New construction for expert system based on innovative knowledge discovery technology

Knowledge acquisition is the bottleneck of expert system. To solve this problem, KD (D&K), which is a comprehensive knowledge discovery process model cooperating both database and knowledge base, and related technology are proposed. Then based on KD (D&K) and related technology, the new construction of Expert System based on Knowledge Discovery (ESKD) is proposed. As the key knowledge acquisition component of ESKD, KD (D&K) is composed of KDD* and KDK*. KDD*—the new process model based on double bases cooperating mechanism; KDK*—the new process model based on double-basis fusion mechanism are introduced, respectively. The overall framework of ESKD is proposed. Some sub-systems and dynamic knowledge base system are discussed. Finally, the effectiveness and advantages of ESKD are tested in a real-world agriculture database. We hope that ESKD may be useful for the new generation of expert systems. Supported by the National Natural Science Foundation of China (Grant No. 69835001), the Ministry of Education of China (Grant No. [2000] 175), and the Science Foundation of Beijing (Grant No. 4022008).     

Abstract and reduced-context representations in fault-finding training

The ability to transfer problem-solving expertise to new situations is regarded as an important practical skill but it is known to be sensitive to contextual differences. Employing abstract and reduced-context training tasks has been proposed as a method of fostering transferable skill but the evidence supporting this approach is limited. The research reported here aimed at assessing the effectiveness of training with diagnostic tasks of varying degrees of contextual detail. The criterion test used to measure training effectiveness was a context-rich diagnostic task that involved locating faults in a representation of a water supply and drainage system (WSDS). In the first study reported (baseline study), subjects were trained and tested on the WSDS task. Various training interventions were employed and a set of diagnostic heuristics proved to be the most effective. Two pilot studies were then carried out to determine whether training with abstract diagnostic tasks could produce transfer to the criterion test, the WSDS. Little evidence of transfer was found; therefore, it was decided to incorporate some context into training but not so much as to obscure important, conceptual task features. This was achieved by constructing reduced-context representations (RCO) of the WSDS at more than one level of abstraction (main study). Training with those representations took less time than training with the WSDS but transfer to the criterion test (the WSDS) was equally efficient. During training with the RCO representations, a technique of refusing costly test information was employed which is thought to have helped subjects discover efficient search strategies and consistently transfer them to the criterion test. This was contrasted with the verbal diagnostic heuristics of the baseline study, which had proved to be effective, but which were not always transferred consistently to the criterion test. This raises the question as to whether diagnostic strategies are necessarily best conveyed solely in verbal form, if the intention is to train transferable skill.     

Measuring information systems success: models,dimensions, measures,and interrelationships

Since DeLone and McLean (D&M) developed their model of IS success, there has been much research on the topic of success as well as extensions and tests of their model. Using the technique of a qualitative literature review, this research reviews 180 papers found in the academic literature for the period 1992–2007 dealing with some aspect of IS success. Using the six dimensions of the D&M model – system quality, information quality, service quality, use, user satisfaction, and net benefits – 90 empirical studies were examined and the results summarized. Measures for the six success constructs are described and 15 pairwise associations between the success constructs are analyzed. This work builds on the prior research related to IS success by summarizing the measures applied to the evaluation of IS success and by examining the relationships that comprise the D&M IS success model in both individual and organizational contexts.     

Abstract and reduced-context representations in fault-finding training

The ability to transfer problem-solving expertise to new situations is regarded as an important practical skill but it is known to be sensitive to contextual differences. Employing abstract and reduced-context training tasks has been proposed as a method of fostering transferable skill but the evidence supporting this approach is limited. The research reported here aimed at assessing the effectiveness of training with diagnostic tasks of varying degrees of contextual detail. The criterion test used to measure training effectiveness was a context-rich diagnostic task that involved locating faults in a representation of a water supply and drainage system (WSDS). In the first study reported (baseline study), subjects were trained and tested on the WSDS task. Various training interventions were employed and a set of diagnostic heuristics proved to be the most effective. Two pilot studies were then carried out to determine whether training with abstract diagnostic tasks could produce transfer to the criterion test, the WSDS. Little evidence of transfer was found; therefore, it was decided to incorporate some context into training but not so much as to obscure important, conceptual task features. This was achieved by constructing reduced-context representations (RCO) of the WSDS at more than one level of abstraction (main study). Training with those representations took less time than training with the WSDS but transfer to the criterion test (the WSDS) was equally efficient. During training with the RCO representations, a technique of refusing costly test information was employed which is thought to have helped subjects discover efficient search strategies and consistently transfer them to the criterion test. This was contrasted with the verbal diagnostic heuristics of the baseline study, which had proved to be effective, but which were not always transferred consistently to the criterion test. This raises the question as to whether diagnostic strategies are necessarily best conveyed solely in verbal form, if the intention is to train transferable skill.     

Observational assessment of situation awareness,team differences and training implications

The two goals were to investigate, first, the practicability and reliability of observational assessment of team situation awareness (SA) and, second, the nature of any team differences, their consistency and training implications. Five shift teams tackled three scenarios, each with three probe events concerning SA, and three observers viewed and rated concurrently each shift. This methodology was found to be practicable and achieved satisfactory rater reliability as indicated by intraclass and inter-rater correlations. Team differences in SA emerged although there was no consistent pattern. A retrospective analysis of individual and team behaviours relating to SA was performed using the Critical Incident Technique. A total of 75 incidents and 20 behavioural dimensions relevant to SA were identified and these were subsumed under planning, problem solving, team coordination, attention, communication and knowledge. These findings are discussed with regard to the nature and measurement of SA, and the content of training to improve SA for control room teams.