Information complexity--mental workload and performance in combat aircraft

The purpose of the present study was to analyse the effects of information complexity on Pilot Mental WorkLoad (PMWL) and Pilot Performance (PP), and to analyse the structure of PMWL. Eighteen pilots performed 72 simulated low level-high speed emissions. The complexity of the Head Down Display (HDD) information varied as a function of the tactical situation. Flight data were recorded continuously. The pilots' eye movements were video taped and psychophysiological activation data, Heart Rate (HR), were obtained. The pilots rated PMWL according to the psychological content of three scales (Bedford Rating Scale, Subjective Workload Assessment Technique, NASA-Task Load indeX) and answered a questionnaire tapping aspects of performance, information load, motivation and mood. It was found that even a moderate complexity of information interfered with the flight task. Altitude and variation in altitude were increased and corrections of altitude errors were delayed, when complexity increased. Changes in information load reached its maximum influence on flight performance (r = 0.60, p < 0.001) after a time delay of 20 to 40 seconds. Performance of the flight task correlated positively (r = 0.59, p <0.001) with the performance of the information handling task (Tactical Situation Awareness, TSA). Durations and frequencies of eye fixations Head Up (HU) versus Head Down (HD) changed as a function of information load. A structural equation model implied that PMWL was affected by mission complexity and that PMWL affected objective and subjective aspects of flight performance and information handling. Heart rate (sortie means) correlated positively with PMWL (r = 0.34, p <0.05) and perceived complexity of mission (r = 0.37, p < 0.01). Heart rate (running means) covaried with variations in information complexity for those pilots who performed well. From spectral analyses of cardiac interval times it was found that the amplitude of the 0.10 Hz component tended to decrease during high as compared to low levels of information load.     

The influence of training level on manual flight in connection to performance,scan pattern,and task load

This work focuses on the analysis of pilots’ performance during manual flight operations in different stages of training and their influence on gaze strategy. The secure and safe operation of air traffic is highly dependent on the individual performances of the pilots. Before becoming a pilot, he/she has to acquire a broad set of skills by training to pass all the necessary qualification and licensing standards. A basic skill for every pilot is manual control operations, which is a closed-loop control process with several cross-coupled variables. Even with increased automation in the cockpit, the manual control operations are essential for every pilot as a last resort in the event of automation failure. A key element in the analysis of manual flight operations is the development over time in relation to performance and visual perception. An experiment with 28 participants (including 11 certified pilots) was conducted in a Boeing 737 simulator. For defined flight phases, the dynamic time warping method was applied to evaluate the performance for selected criteria, and eye-tracking methodology was utilized to analyze the gaze-pattern development. The manipulation of workload and individual experience influences the performance and the gaze pattern at the same time. Findings suggest that the increase of workload has an increased influence on pilots depending on the flight phase. Gaze patterns from experienced pilots provide insights into the training requirements of both novices and experts. The connection between workload, performance and gaze pattern is complex and needs to be analyzed under as many differing conditions. The results imply the necessity to evaluate manual flight operations with respect to more flight phases and a detailed selection of performance indications.

     

Examining the Effects of Conformal Terrain Features in Advanced Head‐Up Displays on Flight Performance and Pilot Situation Awareness

Synthetic vision systems (SVS) render terrain features for pilots through cockpit displays using a GPS database and three‐dimensional graphical models. Enhanced vision systems (EVS) present infrared imagery of terrain using a forward‐looking sensor in the nose of an aircraft. The ultimate goal of SVS and EVS technologies is to support pilots in achieving safety under low‐visibility and night conditions comparable to clear, day conditions. This study assessed pilot performance and situation awareness (SA) effects of SVS and EVS imagery in an advanced head‐up display (HUD) during a simulated landing approach under instrument meteorological conditions. Videos of the landing with various HUD configurations were presented to eight pilots with a superimposed tracking task. The independent variables included four HUD feature configurations (baseline [no terrain imagery], SVS, EVS, and a combination of SVS and EVS), two visibility conditions, and four legs of the flight. Results indicated that SVS increased overall SA but degraded flight path control performance because of visual confusion with other display features. EVS increased flight path control accuracy but decreased system (aircraft) awareness because of visual distractions. The combination of SVS and EVS generated offsetting effects. Display configurations did not affect pilot spatial awareness. Flight performance was not different among phases of the approach, but levels and types of pilot SA did vary from leg to leg. These results are applicable to development of adaptive HUD features to support pilot performance. They support the use of multidimensional measures of SA for insight on pilot information processing with advanced aviation displays. © 2012 Wiley Periodicals, Inc.     

Pilot maneuver choice and workload in free flight

Two experiments examined pilots' maneuver choice and visual workload in a free-flight simulation. In Experiment 1, 12 pilots flew a high-fidelity flight simulator with a cockpit display of traffic information and maneuvered to avoid traffic in a simulated free-flight environment. Pilots' choices reflected a preference to make vertical rather than lateral avoidance maneuvers and to climb rather than descend. Pilots avoided both complex maneuvers and airspeed maneuvers. The data were modeled in terms of how pilots traded off factors related to safety, efficiency, mental effort, and prior habits. In Experiment 2, 10 pilots flew the same maneuvers as the pilots in Experiment 1 but followed ATC instructions rather than using the CDTI. The CDTI in Experiment 1 occupied 25% of the pilots' visual attention. A comparison of scanning with Experiment 2 suggested that the CDTI pulled visual attention away from the outside world, but this attention diversion did not leave pilots vulnerable to missing traffic not annunciated on the CDTI. Actual or potential applications of the results include understanding the safety implications of presenting traffic displays in the cockpit, and the impact of pilot maneuver preferences on airspace procedures.     

Effects of modes of cockpit automation on pilot performance and workload in a next generation flight concept of operation

The objective of this study was to compare the effects of various forms of advanced cockpit automation for flight planning on pilot performance and workload under a futuristic concept of operation. A lab experiment was conducted in which airline pilots flew simulated tailored arrivals to an airport using three modes of automation (MOAs), including a control‐display unit (CDU) to the aircraft flight management system, an enhanced CDU (CDU+), and a continuous descent approach (CDA) tool. The arrival scenario required replanning to avoid convective activity and was constrained by a minimum fuel requirement at the initial approach fix. The CDU and CDU+ modes allowed for point‐by‐point path planning or selection among multiple standard arrivals, respectively. The CDA mode completely automated the route replanning for pilots. It was expected that the higher‐level automation would significantly reduce pilot workload and improve overall flight performance. In general, results indicated that the MOAs influenced pilot performance and workload responses according to hypotheses. This study provides new knowledge about the relationship of cockpit automation and interface features with pilot performance and workload in a novel next generation–style flight concept of operation. © 2012 Wiley Periodicals, Inc.     

Pilot ability to anticipate the consequences of flight actions as a function of expertise

The study offers insights into pilot ability to anticipate consequences of actions and how this ability changes with experience. Novice and expert pilots completed trials in which 3 screens depicted a control movement (or control movements), a cockpit flight situation, or a change in flight situation. Changes depicted in the 3rd screen of each trial were consistent, inconsistent with the mental model of the effect of the control movement or movements, or inconsistent with the application of the control movement(s) to the current flight situation. Pilots indicated whether the depicted change was inconsistent or consistent with their expectations, and accuracy of consistency judgments was greater for mental-model than for situation-model inconsistent statements. Experts are more accurate than novices, particularly for trials that involve multiple, meaningfully related control movements. Expert ability to organize information into meaningful units appears to facilitate future flight state projections, and projection failures appear to result from situation- rather than mental-model failures. Actual or potential applications of this research include analysis of flight situation awareness and flight performance errors.     

心理运动能力测评系统的开发应用

飞行事故的发生,有一部分原因是因为飞行员自身不适合飞行活动。为了减少这种原因导致的飞行事故发生的几率,同时为了避免民航公司因为飞行学员高强度训练而被淘汰造成的培训费用浪费,需要对飞行员候选者开展相关的心理选拔,考察他们是否具备合格飞行员所需的各种心理品质。目前,我国民航飞行员心理选拔大多沿用传统的纸笔测验,对应聘者进行人格和基本能力测验,对飞行特殊能力的考察和检测不够重视。文中研究的主要目的是为了实现民航飞行员心理选拔的仪器检测化,以达到选拔过程的客观化和标准化。本单位自行开发了一套计算机化的心理运动能力检测系统,并将该软件用于民航飞行员招聘中,并对结果进行统计分析,通过对现役飞行员和应聘者之间内容效度的比较,发现其成绩之间有较大差异,表明该软件对民航飞行员心理运动能力选拔具有实际应用价值。该测试软件操作简单方便,界面友好,结果准确可靠。     

A virtual reality cognitive health screening tool for aviation: Managing accident risk for older pilots

To address elevated risk for older pilots, we examined the efficacy of a virtual reality (VR) cognitive health screening tool (integrated into simulated flight scenarios) in identifying general aviation pilots who experienced a critical incident during flight in a full-scale Cessna 172 simulator. Performance data were obtained from 51 certified pilots (17–71 years). Machine learning classification algorithms, based on key data from the VR flight, were used to validate the utility of the screening tool for identifying pilot risk. The results showed that aviation-relevant cognitive factors obtained in the VR screening tool, including situation awareness and prospective memory, predicted risk of a critical incident with good sensitivity (0.83) and specificity (0.85), AUC = 0.82. These results support VR-based cognitive screening to identify at-risk older pilots. The present findings inform procedures for optimizing safety and reducing critical incidents at any point in the pilot lifespan and are timely in view of the impending pilot workforce shortage.     

Eye movement as a mediator of the relationships among time pressure,feedback, and learning performance

The goal of this study is to examine the effects of time pressure and feedback on learning performance, as mediated by eye movement. Time pressure is one of main causes of human error in the workplace. Providing participants with feedback about their performance before task completion has been shown to reduce human error in diverse domains. Since both time pressure and feedback induce motivation, which is closely related to attention, we measured participants' eye movements to trace their attention and information acquisition coupled with a visual display. Time-to-deadline (long and short) and the presence of feedback were the independent factors used while measuring participants’ performance and eye movements as they learned new information about the subject of project management and answered multiple-choice questions via self-paced online learning systems. Using structural equation modeling, we found a mediating effect of eye movement on the relationships among time-to-deadline, feedback, and learning performance. Insufficient time-to-deadline accelerated the number of fixations on the screen, which resulted in longer task completion times and increased correct rates for participants learning about project management. The models in this study suggest the possibility of predicting performance from eye movement under time-to-deadline and feedback conditions. The structural equation model in the study can be applied to online and remote learning systems, in which time management is one of the main challenges for individual learners.     

The influence of low frequency vibration on pilot performance (as measured in a fixed base simulator)

This report describes a study on the effects of low frequency vibration on pilot performance. Army pilots acted as subjects and flew simulated helicopter missions in a realistic fixed base simulator environment. While flying the two-hour missions pilots were exposed to vibration stimuli varying in frequency from 6 to 12 Hz and in amplitude from ±0·1 to ±0·3g (measured at the floor). Measurements were taken of the vibration tramsmissibility of the pilot's seat so that the vibration actually felt by the subject could be determined.

Missions involved transporting external loads in a simulated logistics environment. Performance was evaluated by measuring flight path deviations from prescribed en route, approach, and hover parameters.

The vibration stimuli used did not degrade performance. In fact, performance tended to improve with increased stress. It is hypothesized that this trend was due to motivation, i.e. as subjects felt the onset of fatigue they compensated by working harder and thus tended to improve their performance.

On about 6% of their scores pilots exhibited sudden lapses in their ability to respond to display indications. This resulted in poor scores in the midst of otherwise normal data. These lapses are probably of very short duration (seconds in length) and seem to occur randomly. This same effect was also observed in a previous study by the author. It may be possible that lapses of this type are related to so called ‘pilot error’ accidents. Further research is needed to verify this.     

A Method for Classification and Evaluation of Pilot’s Mental States Based on CNN

How to accurately recognize the mental state of pilots is a focus in civil aviation safety. The mental state of pilots is closely related to their cognitive ability in piloting. Whether the cognitive ability meets the standard is related to flight safety. However, the pilot's working state is unique, which increases the difficulty of analyzing the pilot's mental state. In this work, we proposed a Convolutional Neural Network (CNN) that merges attention to classify the mental state of pilots through electroencephalography (EEG). Considering the individual differences in EEG, semi-supervised learning based on improved K-Means is used in the model training to improve the generalization ability of the model. We collected the EEG data of 12 pilot trainees during the simulated flight and compared the method in this paper with other methods on this data. The method in this paper achieved an accuracy of 86.29%, which is better than 4D-aNN and HCNN etc. Negative emotion will increase the probability of fatigue appearing, and emotion recognition is also meaningful during the flight. Then we conducted experiments on the public dataset SEED, and our method achieved an accuracy of 93.68%. In addition, we combine multiple parameters to evaluate the results of the classification network on a more detailed level and propose a corresponding scoring mechanism to display the mental state of the pilots directly.     

Vigilance on the civil flight deck: incidence of sleepiness and sleep during long-haul flights and associated changes in physiological parameters

The study investigated sleepiness and sleep in aircrew during long-haul flights. The objectives were to identify loss of alertness and to recommend a practical approach to the design of an alerting system to be used by aircrew to prevent involuntary sleep. The flights were between London and Miami, covering both day- and night-time sectors, each with a duration of ~9 h. The subjects were 12 British Airways pilots. Various physiological variables were measured that could potentially be used to indicate the presence of drowsiness and involuntary sleep: brain electrical activity (electroencephalogram, EEG), eye movements via the electro-oculogram (EOG), wrist activity, head movements and galvanic skin resistance. The EEG and EOG identified sleepiness and sleep, as well as being potential measures on which to base an alarm system. Ten pilots either slept or showed evidence of sleepiness as assessed by the EEG and EOG. Many of the episodes of sleepiness lasted < 20 s, which could mean that the subjects were unaware of their occurrence and of the potential consequences on performance and vigilance. All physiological parameters showed changes during sleep, although only the EEG and EOG were modified by sleepiness. During sleep, skin resistance was increased, and wrist activity and head movements were absent for long periods. The study indicated that the measurement of eye movements (either alone or in combination with the EEG), wrist activity or head movement may be used as the basis of an alarm system to prevent involuntary sleep. Skin resistance is considered to be unsuitable, however, being related in a more general way to fatigue rather than to sleep episodes. The optimal way to monitor the onset of sleep would be to measure eye movements; however, this is not feasible in the flight deck environment at the present time due to the intrusive nature of the recording methodology. Wrist activity is therefore recommended as the basis of an alertness alarm. Such a device would alert the pilot after ~4–5 min of wrist inactivity, since this duration has been shown by the present study to be associated with sleep. The possibility that sleep inertia (reduced alertness immediately after awakening from sleep) could follow periods of sleep lasting 5 min needs to be considered. The findings reported here might be applicable to other occupational environments where fatigue and sleepiness are known to occur.     

Vigilance on the civil flight deck: incidence of sleepiness and sleep during long-haul flights and associated changes in physiological parameters

The study investigated sleepiness and sleep in aircrew during long-haul flights. The objectives were to identify loss of alertness and to recommend a practical approach to the design of an alerting system to be used by aircrew to prevent involuntary sleep. The flights were between London and Miami, covering both day- and night-time sectors, each with a duration of approximately 9 h. The subjects were 12 British Airways pilots. Various physiological variables were measured that could potentially be used to indicate the presence of drowsiness and involuntary sleep: brain electrical activity (electroencephalogram, EEG), eye movements via the electro-oculogram (EOG), wrist activity, head movements and galvanic skin resistance. The EEG and EOG identified sleepiness and sleep, as well as being potential measures on which to base an alarm system. Ten pilots either slept or showed evidence of sleepiness as assessed by the EEG and EOG. Many of the episodes of sleepiness lasted < 20 s, which could mean that the subjects were unaware of their occurrence and of the potential consequences on performance and vigilance. All physiological parameters showed changes during sleep, although only the EEG and EOG were modified by sleepiness. During sleep, skin resistance was increased, and wrist activity and head movements were absent for long periods. The study indicated that the measurement of eye movements (either alone or in combination with the EEG), wrist activity or head movement may be used as the basis of an alarm system to prevent involuntary sleep. Skin resistance is considered to be unsuitable, however, being related in a more general way to fatigue rather than to sleep episodes. The optimal way to monitor the onset of sleep would be to measure eye movements; however, this is not feasible in the flight deck environment at the present time due to the intrusive nature of the recording methodology. Wrist activity is therefore recommended as the basis of an alertness alarm. Such a device would alert the pilot after approximately 4-5 min of wrist inactivity, since this duration has been shown by the present study to be associated with sleep. The possibility that sleep inertia (reduced alertness immediately after awakening from sleep) could follow periods of sleep lasting 5 min needs to be considered. The findings reported here might be applicable to other occupational environments where fatigue and sleepiness are known to occur.     

Pilots strategically compensate for display enlargements in surveillance and flight control tasks

OBJECTIVE: Experiments were conducted to assess the impact of display size on flight control, airspace surveillance, and goal-directed target search. BACKGROUND: Research of 3-D displays has shown that display scale compression influences the perception of flight path deviation, though less is known about the causes that drive this effect. In addition, research on attention-based tasks has shown that information displaced to significant eccentricities can amplify effort, but it is unclear whether the effect generates a performance difference in complex displays. METHOD: In Experiment 1, 16 pilots completed a low-fidelity flight control task under single- and dual-axis control. In Experiment 2, the control task from Experiment 1 was scaled up to a more realistic flight environment, and pilots performed hazard surveillance and target search tasks. RESULTS: For flight control, pilots exhibited less path error and greater stick activity with a large display, which was attributed both to greater enhanced resolution and to the fact that larger depictions of error lead to greater urgency in correcting deviations. Size did not affect hazard surveillance or search, as pilots were adaptive in altering scanning patterns in response to the enlargement of the displays. CONCLUSION: Although pilots were adaptive to display changes in search and surveillance, display size reduction diminished estimates of flight path deviation and control performance because of lowered resolution and control urgency. APPLICATION: Care should be taken when manipulating display size, as size reduction can diminish control performance.     

Eye tracking as a debriefing tool in upset prevention and recovery training (UPRT) for general aviation pilots

Abstract

Upset prevention and recovery training (UPRT) is intended to improve the ability of pilots to recognize and avoid situations that can lead to airplane upsets and to improve their ability to recover control of an airplane that has exceeded the normal flight envelope. To this end, a set of different training contents – from theoretical knowledge of aerodynamics and human factors to practice-based flight training – is necessary. In order to support the debriefing with an objective feedback, and because visual scanning is a core competence, two studies on subjective evaluation of aviation pilots – one conducted in a flight simulator and the other one in-flight – focussed on the practical application of eye tracking as a debriefing tool in UPRT. From a practitioner’s perspective, eye tracking appeared to be a useful method in terms of visualising instrument scanning techniques, supporting the instructor with objective debriefing material and fostering self-awareness in human processes. The discussion recommends adjusted UPRT instructor training and further improvements to eye tracking hardware and software.

Practitioner Summary: The article focuses on pilot evaluations of eye tracking as a debriefing tool in UPRT and the identification of critical elements in its use. Eye tracking is a promising debriefing tool for UPRT. The discussion points to desirable improvements of eye tracking hardware and software as well as adjustments to instructor training that are pertinent.     

Helicopter pilots’ views of air traffic controller responsibilities: a mismatch

Abstract

Controllers and pilots must work together to ensure safe and efficient helicopter flight within the London control zone. Subjective ratings of pilot perception of controller responsibility for five key flight tasks were obtained from thirty helicopter pilots. Three types of airspace were investigated. Results indicate that there is variation in pilot understanding of controller responsibility compared to the formal regulations that define controller responsibility. Significant differences in the perception of controller responsibility were found for the task of aircraft separation in class D airspace and along helicopter routes. Analysis of the patterns of response suggests that task type rather than the airspace type may be the key factor. Results are framed using the concept of a shared mental model. This research demonstrates that pilots flying in complex London airspace have an expectation of controller responsibility for certain flight tasks, in certain airspace types that is not supported by aviation regulation.

Practitioner Summary: The responsibility for tasks during flight varies according to the flight rules used and airspace type. Helicopter pilots may attribute responsibility to controllers for tasks when controllers have no responsibility as defined by regulation. This variation between pilot perceptions of controller responsibility could affect safety within the London control zone.     

Physiological measurement of anxiety to evaluate performance in simulation training

The ability to control emotion is a skill which contributes to performance in the same way as cognitive and technical skills do to the successful completion of high stress operations. The interdependence between emotion, problem-solving and decision-making makes a negative emotion such as anxiety of interest in evaluating trainee performance in simulations which replicate stressful work conditions. Self-report measures of anxiety require trainees to interrupt the simulation experience to either complete psychological scales or make verbal reports of state anxiety. An uninterrupted, continuous measure of anxiety is, therefore, preferable for simulation environments. During this study, the anxiety levels of trainee pilots were tracked via electromyography, eye movements and pupillometry while undertaking required tasks in a flight simulation. Fixation duration and saccade rate corresponded reliably to pilot self-reports of anxiety, while pupil size and saccade amplitude did not show a strong comparison to changes in affective state. Large increases in muscle activation where recorded when higher anxiety was reported. The results suggest that a combination of physiological measures could provide a robust, continuous indicator of anxiety level. The implications of the current study on further development of physiological measures to support tracking anxiety as a tool for simulation training assessment are discussed.     

Affects on visual search performance of individual differences in fixation time and number of fixations

Abstract

The purpose of this study was to provide a basis for improving individual visual performance of inspectors. The relationship between the correct count rate and eye movements of subjects when they counted dots arranged on samples presented for different lengths of time were analysed mainly to determine individual differences. Subjects' eye movements were measured with a corneal reflectance eye camera and analysed frame by frame with a video motion analyser. It was found that accuracy of visual inspection does not depend on length of search time and that a fast search time is not incompatible with a slow search speed. Furthermore, fixation time and number of fixations were considered the main factors governing accuracy of visual inspection. When limited time is allowed for search, a search strategy of prolonging the fixation time leads to high performance and consequently shorter inspection time. Several other findings were obtained which appear important in obtaining accurate information rapidly.     

Task partitioning effects in semi-automated human–machine system performance

Twelve professional pilots performed a flight simulation consisting of three component sub-tasks: (i) tracking, (ii) monitoring and (iii) targeting, respectively. The targeting sub-task required (i) target identification, (ii) weapon selection and then (iii) weapon release. Pilots performed in a fully manual condition, a partial automation condition or a fully automated condition. Automated assistance was provided for the targeting sub-task only, while tracking and monitoring sub-tasks were always performed manually. During full automation, the computer located the target, identified it and released the appropriate weapon without any pilot input. During partial automation, the computer located and identified the target while the pilot retained final control over weapon release. Significantly higher levels of tracking error distinguished manual from both automated conditions and also between the two levels of automation. Monitoring response times were also sensitive to the degree of automation engaged, with the partial-automation condition exhibiting faster responses than full automation. Findings support a design principle in which pilots retain control over final weapons release directly on the basis of objective performance outcome. These collective results support the contention that effective and principled task-partitioning should represent a central strategy for the evolution of complex human–machine systems.

Practitioner Summary: Advantages of partitioning tasks between human and automated control are contingent upon the overall context of performance and the actual way the partitioning is accomplished. Simple algorithms, for example, automate on every feasible occasion, are poor design heuristics and may even prove actively harmful to overall response capacity. Transitioning humans from active controllers to passive monitors can be a problematic design choice, especially when that individual is socially deemed to retain overall responsibility for ultimate system effects in the real world.