Explicit control of adaptive automation under different levels of environmental stress

This article examines the effectiveness of three different forms of explicit control of adaptive automation under low- and high-stress conditions, operationalised by different levels of noise. In total, 60 participants were assigned to one of three types of automation design (free, prompted and forced choice). They were trained for 4?h on a highly automated simulation of a process control environment, called AutoCAMS. This was followed by a 4-h testing session under noise exposure and quiet conditions. Measures of performance, psychophysiology and subjective reactions were taken. The results showed that all three modes of explicit control of adaptive automation modes were able to attenuate the negative effects of noise. This was partly due to the fact that operators opted for higher levels of automation under noise. It also emerged that forced choice showed marginal advantages over the two other automation modes. Statement of Relevance: This work is relevant to the design of adaptive automation since it emphasises the need to consider the impact of work-related stressors during task completion. During the presence of stressors, different forms of operator support through automation may be required than under more favourable working conditions.     

Effects of adaptive task allocation on monitoring of automated systems

The effects of adaptive task allocation on monitoring for automation failure during multitask flight simulation were examined. Participants monitored an automated engine status task while simultaneously performing tracking and fuel management tasks over three 30-min sessions. Two methods of adaptive task allocation, both involving temporary return of the automated engine status task to the human operator ("human control"), were examined as a possible countermeasure to monitoring inefficiency. For the model-based adaptive group, the engine status task was allocated to all participants in the middle of the second session for 10 min, following which it was again returned to automation control. The same occurred for the performance-based adaptive group, but only if an individual participant's monitoring performance up to that point did not meet a specified criterion. For the nonadaptive control groups, the engine status task remained automated throughout the experiment. All groups had low probabilities of detection of automation failures for the first 40 min spent with automation. However, following the 10-min intervening period of human control, both adaptive groups detected significantly more automation failures during the subsequent blocks under automation control. The results show that adaptive task allocation can enhance monitoring of automated systems. Both model-based and performance-based allocation improved monitoring of automation. Implications for the design of automated systems are discussed.     

Comparison of a brain-based adaptive system and a manual adaptable system for invoking automation

OBJECTIVE: Two experiments are presented examining adaptive and adaptable methods for invoking automation. BACKGROUND: Empirical investigations of adaptive automation have focused on methods used to invoke automation or on automation-related performance implications. However, no research has addressed whether performance benefits associated with brain-based systems exceed those in which users have control over task allocations. METHOD: Participants performed monitoring and resource management tasks as well as a tracking task that shifted between automatic and manual modes. In the first experiment, participants worked with an adaptive system that used their electroencephalographic signals to switch the tracking task between automatic and manual modes. Participants were also divided between high- and low-reliability conditions for the system-monitoring task as well as high- and low-complacency potential. For the second experiment, participants operated an adaptable system that gave them manual control over task allocations. RESULTS: Results indicated increased situation awareness (SA) of gauge instrument settings for individuals high in complacency potential using the adaptive system. In addition, participants who had control over automation performed more poorly on the resource management task and reported higher levels of workload. A comparison between systems also revealed enhanced SA of gauge instrument settings and decreased workload in the adaptive condition. CONCLUSION: The present results suggest that brain-based adaptive automation systems may enhance perceptual level SA while reducing mental workload relative to systems requiring user-initiated control. APPLICATION: Potential applications include automated systems for which operator monitoring performance and high-workload conditions are of concern.     

Evaluation of operators' performance for automation design in the fully digital control room of nuclear power plants

Recent technical developments in computer hardware and software have meant that human–machine systems can be automated in many respects. If automation fails, however, human operators can have difficulty in recognizing the existence of a problem, identifying what has failed, and taking corrective action to remedy these out‐of‐the‐loop (OOTL) performance problems. Several studies have suggested that taxonomies of levels of automation (LOAs) and types of automation (TOAs) can be used to solve OOTL problems. This study examined the impact of LOAs in process control automation within the context of nuclear power plants (NPPs). A simulation experiment in an NPP is performed to validate this framework using an automatic mode and a semiautomatic mode. Mental demand is found to be significantly reduced under the automatic mode; however, participants felt frustrated with this LOA. Situation awareness is found to be similar in the two modes. The results of an end‐of‐experiment subjective rating reveal that participants were evenly divided between the two modes with respect to generating and selecting functions. It is therefore suggested that human operators be involved in generating and selecting functions under an automatic mode. © 2009 Wiley Periodicals, Inc.     

Designing automation for complex work environments under different levels of stress

This article examines the effectiveness of different forms of static and adaptable automation under low- and high-stress conditions. Forty participants were randomly assigned to one of four experimental conditions, comparing three levels of static automation (low, medium and high) and one level of adaptable automation, with the environmental stressor (noise) being varied as a within-subjects variable. Participants were trained for 4?h on a simulation of a process control environment, called AutoCAMS, followed by a 2.5-h testing session. Measures of performance, psychophysiology and subjective reactions were taken. The results showed that operators preferred higher levels of automation under noise than under quiet conditions. A number of parameters indicated negative effects of noise exposure, such as performance impairments, physiological stress reactions and higher mental workload. It also emerged that adaptable automation provided advantages over low and intermediate static automation, with regard to mental workload, effort expenditure and diagnostic performance. The article concludes that for the design of automation a wider range of operational scenarios reflecting adverse as well as ideal working conditions needs to be considered.     

Effects of modes of cockpit automation on pilot performance and workload in a next generation flight concept of operation

The objective of this study was to compare the effects of various forms of advanced cockpit automation for flight planning on pilot performance and workload under a futuristic concept of operation. A lab experiment was conducted in which airline pilots flew simulated tailored arrivals to an airport using three modes of automation (MOAs), including a control‐display unit (CDU) to the aircraft flight management system, an enhanced CDU (CDU+), and a continuous descent approach (CDA) tool. The arrival scenario required replanning to avoid convective activity and was constrained by a minimum fuel requirement at the initial approach fix. The CDU and CDU+ modes allowed for point‐by‐point path planning or selection among multiple standard arrivals, respectively. The CDA mode completely automated the route replanning for pilots. It was expected that the higher‐level automation would significantly reduce pilot workload and improve overall flight performance. In general, results indicated that the MOAs influenced pilot performance and workload responses according to hypotheses. This study provides new knowledge about the relationship of cockpit automation and interface features with pilot performance and workload in a novel next generation–style flight concept of operation. © 2012 Wiley Periodicals, Inc.     

Task partitioning effects in semi-automated human–machine system performance

Twelve professional pilots performed a flight simulation consisting of three component sub-tasks: (i) tracking, (ii) monitoring and (iii) targeting, respectively. The targeting sub-task required (i) target identification, (ii) weapon selection and then (iii) weapon release. Pilots performed in a fully manual condition, a partial automation condition or a fully automated condition. Automated assistance was provided for the targeting sub-task only, while tracking and monitoring sub-tasks were always performed manually. During full automation, the computer located the target, identified it and released the appropriate weapon without any pilot input. During partial automation, the computer located and identified the target while the pilot retained final control over weapon release. Significantly higher levels of tracking error distinguished manual from both automated conditions and also between the two levels of automation. Monitoring response times were also sensitive to the degree of automation engaged, with the partial-automation condition exhibiting faster responses than full automation. Findings support a design principle in which pilots retain control over final weapons release directly on the basis of objective performance outcome. These collective results support the contention that effective and principled task-partitioning should represent a central strategy for the evolution of complex human–machine systems.

Practitioner Summary: Advantages of partitioning tasks between human and automated control are contingent upon the overall context of performance and the actual way the partitioning is accomplished. Simple algorithms, for example, automate on every feasible occasion, are poor design heuristics and may even prove actively harmful to overall response capacity. Transitioning humans from active controllers to passive monitors can be a problematic design choice, especially when that individual is socially deemed to retain overall responsibility for ultimate system effects in the real world.     

The effects of communication modes on performance and discourse organization with an adaptive interface

This paper reports a study of different communication patterns on performance with a simulated adaptive interface that created the impression of a talking and listening computer which would help participants solve problems with a computer. There were four levels of communication modes which differed in the restrictions placed on human-computer communication. Dependent measures included tasks completed per minute as well as participants' utterances, which were assessed for verbosity, disfluencies, and indices of common ground. The largest performance differences were found between the groups that could communicate freely and those where communication was restricted or denied. As restriction increased, performance decreased. Further, as restriction increased, the computer assumed greater control and verbosity decreased. Performance on the simple tasks declined as communication restriction increased, but no differences were observed for complex tasks. The results are discussed with respect to differences between human-human and human-computer communication as well as research on adaptive environments.     

Sensitivity of candidate markers of psychophysiological strain to cyclical changes in manual control load during simulated process control

Complex systems are vulnerable to unpredictable breakdowns in operator performance. Although primary task goals are typically protected by compensatory effort, such protection may break down under fatigue and high strain. Detection of strain states would enable prediction of increased operational risk through adaptive automation, triggering a switch of control from human to computer. A simulated process control task was used to identify markers of strain under a cyclic loading procedure, which forced performance breakdown through stepwise changes in control load. Four trained participants provided data on control performance and a range of candidate psychophysiological markers of strain (two EEG power ratios and HRV). Within-individual analyses showed the strongest sensitivity for ‘task load index’ (TLI), an EEG measure based on executive control activity in frontal brain areas, though all measures were sensitive for some participants. The implications of such findings for the development of a closed loop system for adaptive automation are discussed.     

The effects of errors on system trust, self-confidence, and the allocation of control in route planning

The concept of trust is believed by some to compensate for feelings of uncertainty. Therefore, trust is considered to be crucial in people's decision to rely on a complex automated system to perform tasks for them. This experiment aimed to study the effects of errors on control allocation, and the mediating role of trust and self-confidence in the domain of route planning. Using a computer-based route planner, participants completed 10 route-planning trials in manual mode, and 10 in automatic mode, allowing participants to become equally experienced in operating both modes. During these so-called fixed trials, the numbers of errors in automatic as well as manual mode were systematically varied. Subsequently, participants completed six free trials, during which they were free to choose between modes. Our results showed that high automation error rates (AERs) decreased levels of system trust compared to low AERs. Conversely, high manual error rates (MERs) resulted in lower levels of self-confidence compared to low MERs, although to a lesser extent. Moreover, the difference between measures of trust and self-confidence proved to be highly predictive of the number of times automatic mode was selected during the six free trials. Additionally, results suggest a fundamental bias to trust one's own abilities over those of the system. Finally, evidence indicating a relationship between trust and self-confidence is discussed.     

Level of automation effects on performance, situation awareness and workload in a dynamic control task

Various levels of automation (LOA) designating the degree of human operator and computer control were explored within the context of a dynamic control task as a means of improving overall human/machine performance. Automated systems have traditionally been explored as binary function allocations; either the human or the machine is assigned to a given task. More recently, intermediary levels of automation have been discussed as a means of maintaining operator involvement in system performance, leading to improvements in situation awareness and reductions in out-of-the-loop performance problems. A LOA taxonomy applicable to a wide range of psychomotor and cognitive tasks is presented here. The taxonomy comprises various schemes of generic control system function allocations. The functions allocated to a human operator and/or computer included monitoring displays, generating processing options, selecting an 'optimal' option and implementing that option. The impact of the LOA taxonomy was assessed within a dynamic and complex cognitive control task by measuring its effect on human/system performance, situation awareness and workload. Thirty subjects performed simulation trials involving various levels of automation. Several automation failures occurred and out-of-the-loop performance decrements were assessed. Results suggest that, in terms of performance, human operators benefit most from automation of the implementation portion of the task, but only under normal operating conditions; in contrast, removal of the operator from task implementation is detrimental to performance recovery if the automated system fails. Joint human/system option generation significantly degraded performance in comparison to human or automated option generation alone. Lower operator workload and higher situation awareness were observed under automation of the decision making portion of the task (i.e. selection of options), although human/system performance was only slightly improved. The implications of these findings for the design of automated systems are discussed.     

Detecting automation failures in a simulated supervisory control environment

Abstract

The goal of this research was to determine how individuals perform and allocate their visual attention when monitoring multiple automated displays that differ in automation reliability. Ninety-six participants completed a simulated supervisory control task where each automated display had a different level of reliability (namely 70%, 85% and 95%). In addition, participants completed a high and low workload condition. The performance data revealed that (1) participants’ failed to detect automation misses approximately 2.5 times more than automation false alarms, (2) participants’ had worse automation failure detection in the high workload condition and (3) participant automation failure detection remained mostly static across reliability. The eye tracking data revealed that participants spread their attention relatively equally across all three of the automated displays for the duration of the experiment. Together, these data support a system-wide trust approach as the default position of an individual monitoring multiple automated displays.

Practitioner Summary: Given the rapid growth of automation throughout the workforce, there is an immediate need to better understand how humans monitor multiple automated displays concurrently. The data in this experiment support a system-wide trust approach as the default position of an individual monitoring multiple automated displays.

Abbreviations: DoD: Department of Defense; UA: unmanned aircraft; SCOUT: Supervisory Control Operations User Testbed; UAV: unmanned aerial vehicle; AOI: areas of interest     

大规模集群运维自动化的探索与实践

本文从东方公司研究院的IT现状和未来设备管理入手,对大规模集群自动化运维模式和工具进行分析,提出IT运维自动化概念。通过目前存在的问题,分析运维自动化的目标和方法,介绍建立运维自动化的步骤;以图解方式介绍自动化安装、自动化配置、自动化监控等使用情况,并据此说明未来大规模集群运维自动化的远景规划。     

Discipline differences in mental models: How mechanical engineers and automation engineers evaluate machine processes

Interdisciplinary collaboration frequently comes into play when existing problems cannot be solved by one discipline alone. However, the interlocking of contributions from different disciplines is by no means trivial. This exploratory study examines one foundation of successful teamwork, namely shared mental models. To this end, we compared the contents of mental models between members of different but interdependent disciplines who collaboratively solve knowledge-intensive, creative tasks. Five automation and five mechanical engineers were recruited from a company that produces packaging machines. In semi-structured interviews, participants reported their approach to evaluating the process behavior of a packaging machine, and their mental models were represented in concept maps. Quantitative analyses revealed that the maps of automation engineers were smaller than those of mechanical engineers. In qualitative analyses, the focus on different levels of abstraction and on contents from the two disciplines was examined. Automation engineers represented a large proportion of rather abstract machine functions, whereas mechanical engineers additionally represented the physical implementation of these functions. The disciplinary focus also differed in the sense that automation engineers mainly attended to automated machine processes, while mechanical engineers attended to both mechanical and automated processes. Overall, automation engineers' focus was narrower than that of mechanical engineers. We explain these results by considering typical tasks and reasoning processes in both disciplines, and discuss how shared mental models can aid the integration of different disciplinary perspectives, for instance, during Systems Engineering.     

Designing for flexible interaction between humans and automation: delegation interfaces for supervisory control

OBJECTIVE: To develop a method enabling human-like, flexible supervisory control via delegation to automation. BACKGROUND: Real-time supervisory relationships with automation are rarely as flexible as human task delegation to other humans. Flexibility in human-adaptable automation can provide important benefits, including improved situation awareness, more accurate automation usage, more balanced mental workload, increased user acceptance, and improved overall performance. METHOD: We review problems with static and adaptive (as opposed to "adaptable") automation; contrast these approaches with human-human task delegation, which can mitigate many of the problems; and revise the concept of a "level of automation" as a pattern of task-based roles and authorizations. We argue that delegation requires a shared hierarchical task model between supervisor and subordinates, used to delegate tasks at various levels, and offer instruction on performing them. A prototype implementation called Playbook is described. RESULTS: On the basis of these analyses, we propose methods for supporting human-machine delegation interactions that parallel human-human delegation in important respects. We develop an architecture for machine-based delegation systems based on the metaphor of a sports team's "playbook." Finally, we describe a prototype implementation of this architecture, with an accompanying user interface and usage scenario, for mission planning for uninhabited air vehicles. CONCLUSION: Delegation offers a viable method for flexible, multilevel human-automation interaction to enhance system performance while maintaining user workload at a manageable level. APPLICATION: Most applications of adaptive automation (aviation, air traffic control, robotics, process control, etc.) are potential avenues for the adaptable, delegation approach we advocate. We present an extended example for uninhabited air vehicle mission planning.     

Situation awareness and driving performance in a simulated navigation task

The objective of this study was to identify task and vehicle factors that may affect driver situation awareness (SA) and its relationship to performance, particularly in strategic (navigation) tasks. An experiment was conducted to assess the effects of in-vehicle navigation aids and reliability on driver SA and performance in a simulated navigation task. A total of 20 participants drove a virtual car and navigated a large virtual suburb. They were required to follow traffic signs and navigation directions from either a human aid via a mobile phone or an automated aid presented on a laptop. The navigation aids operated under three different levels of information reliability (100%, 80% and 60%). A control condition was used in which each aid presented a telemarketing survey and participants navigated using a map. Results revealed perfect navigation information generally improved driver SA and performance compared to unreliable navigation information and the control condition (task-irrelevant information). In-vehicle automation appears to mediate the relationship of driver SA to performance in terms of operational and strategic (navigation) behaviours. The findings of this work support consideration of driver SA in the design of future vehicle automation for navigation tasks.     

无人作战飞机系统的辅助决策等级实验研究

无人作战飞机的辅助决策系统包括两部分：任务规划与控制站为操作者呈现清晰明确的任务界面和操作提示;机载智能体的变权限任务管理策略为操作者和无人机动态分配任务和功能,这些辅助都基于人与自动化的交互关系,分为召唤、建议和否决多个辅助决策等级。为了定量评测各级辅助决策方式的效用和辅助不可靠的负面影响,设计开发了等级实验。通过比较操作者手动和三种辅助操作方式的反应时间和正确率,得到了三种辅助决策等级的效用基本相等,辅助可靠性超过35%时就可以采用等实验结论,结论可以用来指导无人作战飞机辅助决策系统的设计。     

How does people's trust in automated vehicles change after automation errors occur? An empirical study on dynamic trust in automated driving

When errors of automated vehicles (AVs) occur, drivers' trust can easily be destroyed, resulting in the reduction of the use of AVs. This study aims to examine how error of AVs declines driver's trust by impacting their subjective perceptions. A driving simulator experiment is conducted, in which 104 participants (male = 58; female = 46) experienced automated driving with automation errors and rated their trust. The results indicate that automation error will affect the driver's perceived predictability, perceived reliability, and perceived safety, which will lead to the decline of trust and abandonment of automated driving. With the occurrence of automation error of AVs, perceived safety plays a more critical role in drivers' trust. In addition, when automation errors occur in specific tasks with low risk, the trust of drivers will drop faster than that in high-risk tasks. This paper has explored the internal effects of the decline of driver's trust after automation errors of AVs, and further considers the influence of different external risks on these perception factors and trust. This study can help AVs manufacturers to formulate different degrees of trust repair strategies according to different driving tasks and accident severity.     

Effects of System Automation Management Strategies and Multi-mission Operator-to-vehicle Ratio on Operator Performance in UAV Systems

It has been documented that the military intends to increase the number of Unmanned Aerial Vehicles (UAVs) in service while at the same time reducing the number of operators (Dixon et al. 2004). To meet this demand, many of the current UAV operator function will need to be automated. Levels of automation exist along a continuum from fully manual to fully automatic. Different automation strategies have been applied widely in UAV systems. Management by Consent (MBC), where the operator selects the task to be executed, and Management by Exception (MBE), where the computer selects the task to be executed are two proposed levels of automation for future UAV systems. Meanwhile, the optimum operator-to-vehicle ratio for future UAV systems is not yet known. It is expected that the optimum operator-to-vehicle ratio will vary with the level of automation applied to the system. Future UAV systems may require the use of adaptive automation to ensure maximum human–machine performance across varying operator-to-vehicle ratios. This study aims to help determine what levels of automation are most appropriate for different operator-to-vehicle ratios and how adaptive automation should be applied in future UAV systems. We investigated the effect of various operator-to-vehicle ratios and the two automation strategies on UAV mission tasks, results were analyzed using Analysis of Variance (ANOVA) and discussed in the last section of the paper.     

A function-to-task process model for adaptive automation system design

Adaptive automation systems allow the user to complete a task seamlessly with a computer performing tasks at which the human operator struggles. Unlike traditional systems that allocate functions to either the human or the machine, adaptive automation varies the allocation of functions during system operation. Creating these systems requires designers to consider issues not present during static system development. To assist in adaptive automation system design, this paper presents the concept of inherent tasks and takes advantage of this concept to create the function-to-task design process model. This process model helps the designer determine how to allocate functions to the human, machine, or dynamically between the two. An illustration of the process demonstrates the potential complexity within adaptive automation systems and how the process model aids in understanding this complexity during early stage design.