Effects of modes of cockpit automation on pilot performance and workload in a next generation flight concept of operation

The objective of this study was to compare the effects of various forms of advanced cockpit automation for flight planning on pilot performance and workload under a futuristic concept of operation. A lab experiment was conducted in which airline pilots flew simulated tailored arrivals to an airport using three modes of automation (MOAs), including a control‐display unit (CDU) to the aircraft flight management system, an enhanced CDU (CDU+), and a continuous descent approach (CDA) tool. The arrival scenario required replanning to avoid convective activity and was constrained by a minimum fuel requirement at the initial approach fix. The CDU and CDU+ modes allowed for point‐by‐point path planning or selection among multiple standard arrivals, respectively. The CDA mode completely automated the route replanning for pilots. It was expected that the higher‐level automation would significantly reduce pilot workload and improve overall flight performance. In general, results indicated that the MOAs influenced pilot performance and workload responses according to hypotheses. This study provides new knowledge about the relationship of cockpit automation and interface features with pilot performance and workload in a novel next generation–style flight concept of operation. © 2012 Wiley Periodicals, Inc.     

基于负荷均衡的飞机驾驶舱动态功能分配

为了提高动态复杂环境下飞机驾驶舱人机系统的安全性,提出一种基于负荷均衡的动态功能分配方法。结合飞行任务和飞行员生理特征预测飞行员的工作负荷,将预测结果作为动态功能分配的触发条件,基于模糊推理调整驾驶舱自动化等级,实现飞行任务在飞行员和自动化系统之间的协调分配。基于Simulink建立飞行任务需求、飞行员生理特征、飞行员工作负荷预测、模糊推理、任务分析与再分配模块,通过仿真测试对提出的方法进行分析和验证,结果表明该方法能够及时有效地调整任务分配,避免飞行员工作负荷过高或过低。     

Hate to interrupt you,but… analyzing turn-arounds from a cockpit perspective

This study aimed to analyze aircraft ground operation processes from a human factors perspective with special emphases on the occurrence and influence of interruptions on pilots’ workload. Interruptions have been shown to increase workload and error probability as well as to contribute to fatal accidents in various fields. Countermeasures have been initiated especially in high-risk environments such as those involving medical issues. In aviation, more explicitly during turn-around processes, interruptions might occur frequently and impair flight safety. One hundred and sixty fully certified pilots working for a European airline were observed during their turn-around while performing real operations. Pilots’ interruptions were documented and classified in order to predict subjectively perceived workload by use of multiple linear regression analysis. External factors such as weather conditions, technical problems, and time pressure were considered as covariates. On average, a pilot experienced about eight interruptions during a turn-around. Overall workload estimates showed a level comparable to that of manual flying in a simulator. Interruptions from colleagues or from outside the cockpit were found to predict pilots’ workload; however, further external factors such as poor weather conditions impacted workload even more strongly. We suggest two approaches based on our results to handling the high rate of interruptions. We first recommend procedural changes to diminish the interruption rate; second, we recommend comprehensive, line-oriented flight training for airline and ground staff to raise awareness about the negative influence of interruptions.     

Task partitioning effects in semi-automated human–machine system performance

Twelve professional pilots performed a flight simulation consisting of three component sub-tasks: (i) tracking, (ii) monitoring and (iii) targeting, respectively. The targeting sub-task required (i) target identification, (ii) weapon selection and then (iii) weapon release. Pilots performed in a fully manual condition, a partial automation condition or a fully automated condition. Automated assistance was provided for the targeting sub-task only, while tracking and monitoring sub-tasks were always performed manually. During full automation, the computer located the target, identified it and released the appropriate weapon without any pilot input. During partial automation, the computer located and identified the target while the pilot retained final control over weapon release. Significantly higher levels of tracking error distinguished manual from both automated conditions and also between the two levels of automation. Monitoring response times were also sensitive to the degree of automation engaged, with the partial-automation condition exhibiting faster responses than full automation. Findings support a design principle in which pilots retain control over final weapons release directly on the basis of objective performance outcome. These collective results support the contention that effective and principled task-partitioning should represent a central strategy for the evolution of complex human–machine systems.

Practitioner Summary: Advantages of partitioning tasks between human and automated control are contingent upon the overall context of performance and the actual way the partitioning is accomplished. Simple algorithms, for example, automate on every feasible occasion, are poor design heuristics and may even prove actively harmful to overall response capacity. Transitioning humans from active controllers to passive monitors can be a problematic design choice, especially when that individual is socially deemed to retain overall responsibility for ultimate system effects in the real world.     

Perceived vs. measured effects of advanced cockpit systems on pilot workload and error: Are pilots' beliefs misaligned with reality?

Four types of advanced cockpit systems were tested in an in-flight experiment for their effect on pilot workload and error. Twelve experienced pilots flew conventional cockpit and advanced cockpit versions of the same make and model airplane. In both airplanes, the experimenter dictated selected combinations of cockpit systems for each pilot to use while soliciting subjective workload measures and recording any errors that pilots made. The results indicate that the use of a GPS navigation computer helped reduce workload and errors during some phases of flight but raised them in others. Autopilots helped reduce some aspects of workload in the advanced cockpit airplane but did not appear to reduce workload in the conventional cockpit. Electronic flight and navigation instruments appeared to have no effect on workload or error. Despite this modest showing for advanced cockpit systems, pilots stated an overwhelming preference for using them during all phases of flight.     

Workload benefits of colour coded head-up flight symbology during high workload flight

The manipulation of colour in display symbology design has been recognised as a method to improve operator experience and performance. An earlier paper by the authors demonstrated that redundantly colour coding head-up flight symbology supported the manual flying performance of both professional and non-professional pilots during low-workload flying scenarios. In this study the workload and performance of 12 professional airline pilots was evaluated in high workload conditions whilst they flew manoeuvres and an instrument landing system (ILS) approach with and without the presence of colour feedback on a head up display (HUD). Workload was manipulated by presenting pilots with a concurrent auditory n-back task. Colour coded flight symbology reduced the subjective workload of the pilots during high workload conditions. In contrast, manual flying performance during high workload was not improved by the presence of colour coded feedback.     

Investigating pilot performance using mixed-modality simulated data link

OBJECTIVE: General aviation (GA) pilot performance utilizing a mixed-modality simulated data link was objectively evaluated based on the time required in accessing, understanding, and executing data link commands. Additional subjective data were gathered on workload, situation awareness (SA), and preference. BACKGROUND: Research exploring mixed-modality data link integration to the single-pilot GA cockpit is lacking, especially with respect to potential effects on safety. METHODS: Sixteen visual flight rules (VFR)-rated pilots participated in an experiment using a flight simulator equipped with a mixed-modality data link. Data link modalities were text display, synthesized speech, digitized speech, and synthesized speech/text combination. Flight conditions included VFR (unlimited ceiling and visibility) or marginal VFR flight conditions (clouds 2,800 ft above ground level, 3-mile visibility). RESULTS: Statistically significant differences were found in pilot performance, mental workload, and SA across the data link modalities. Textual data link resulted in increased time and workload as compared with the three speech-type data link conditions, which did not differ. SA measures indicated higher performance with textual and digitized speech data link conditions. CONCLUSION: Textual data link can be significantly enhanced for single-pilot GA operations by the addition of a speech component. APPLICATION: Potential applications include operational safety in future GA systems that incorporate data link for use by a single pilot and guidance in the development of flight performance objectives for these systems.     

Comparison of NASA-TLX scale,modified Cooper–Harper scale and mean inter-beat interval as measures of pilot mental workload during simulated flight tasks

Abstract

The sensitivity of NASA-TLX scale, modified Cooper–Harper (MCH) scale and the mean inter-beat interval (IBI) of successive heart beats, as measures of pilot mental workload (MWL), were evaluated in a flight training device (FTD). Operational F/A-18C pilots flew instrument approaches with varying task loads. Pilots’ performance, subjective MWL ratings and IBI were measured. Based on the pilots’ performance, three performance categories were formed; high-, medium- and low-performance. Values of the subjective rating scales and IBI were compared between categories. It was found that all measures were able to differentiate most task conditions and there was a strong, positive correlation between NASA-TLX and MCH scale. An explicit link between IBI, NASA-TLX, MCH and performance was demonstrated. While NASA-TLX, MCH and IBI have all been previously used to measure MWL, this study is the first one to investigate their association in a modern FTD, using a realistic flying mission and operational pilots.

Practitioner summary: NASA-TLX scale, MCH scale and the IBI were evaluated in a flight training device. All measures were able to differentiate most task conditions and there was a positive correlation between NASA-TLX and MCH scale. An explicit link between IBI, NASA-TLX, MCH and performance was demonstrated.

Abbreviations: ANOVA: Analysis of Variance; ECG: Electrocardiograph; F/A: fighter/attack; ft: feet; FTD: flight training device; G: Gravity; km: kilometer; m: meter; m/s: meters per second; MWL: mental workload; MCH: modified cooper-harper; NASA-TLX: NASA Task Load Index; NM: Nautical Mile; NN: normal-to-normal; IBI: inter-beat interval; ILS: Instrument Landing System; RR: R-Wave to R-Wave; SD: standard deviation; TTP: tactics, techniques and procedures; WTSAT: Weapon Tactics and Situation Awareness Trainer     

ADAPT: A Predictive Cognitive Model of User Visual Attention and Action Planning

We present a computational cognitive model of novice and expert aviation pilot action planning called ADAPT that models performance in a dynamically changing simulated flight environment. We perform rigorous tests of ADAPT's predictive validity by comparing the performance of individual human pilots to that of their respective models. Individual pilots were asked to execute a series of flight maneuvers using a flight simulator, and their eye fixations and control movements were recorded in a time-synched database. Computational models of each of the 25 individual pilots were constructed, and the individual models simulated execution of the same flight maneuvers performed by the human pilots. The time-synched eye fixations and control movements of individual pilots and their respective models were compared, and rigorous tests of ADAPT's predictive validity were performed. The model explains and predicts a significant portion of pilot visual attention and control movements during flight as a function of piloting expertise. Implications for adaptive training systems are discussed.     

With flying colours: Pilot performance with colour-coded head-up flight symbology

The manipulation of colour in display symbology design has long been recognised as a method to improve operator experience and performance. Recent developments in colour head-up display (HUD) and helmet-mounted display (HMD) technology underline the necessity to understand the human factors considerations of symbology colour coding against conventional monochrome symbology formats. In this low-fidelity desktop human-in-the-loop experiment, the colour of flight symbology on an overlaid symbology set was coded as a redundant cue to indicate the accuracy of professional and non-professional pilots’ flight profile across a range of simulated flight manoeuvres. The main finding of this study was that colour coding flight symbology supported the manual flying performance of both professional and non-professional pilots. Notably, colour-coding of the bank indicator and airspeed tape minimised performance error during turning and altitude change manoeuvres, respectively. The usability of colour coded symbology was also rated higher than the monochrome symbology. We conclude that colour coded HUD/HMD symbology is preferred by the user and may improve performance during low workload manual flying tasks. A fuller understanding of performance and workload effects will require future studies to employ higher workload flying tasks and examine the utility of colour coding within higher fidelity environments.     

Pilots’ gaze strategies and manual control performance using occlusion as a measurement technique during a simulated manual flight task

The aim of this study was to analyze pilots’ visual scanning under conditions visually restricted by the occlusion paradigm. During manual flight, pilots experienced interruptions in their panel scan due to concurring tasks and monitoring of distant displays. Eleven volunteer airline pilots performed several manual instrument landing system approaches in a fixed-base flight simulator. Some of these approaches were performed using the paradigm of occlusion with shutter glasses. Under occlusion, the gaze pattern analysis revealed that pilots demonstrated reduced mean glance durations, but did not reduce their attention to lesser information displays. The results also indicated that the attitude indicator (artificial horizon) as a preview instrument was less affected by occlusion compared to other areas of interest. A subsequent scanpath analysis revealed that vertical tracking was the predominant information acquisition strategy and corresponded to larger deviations on the glideslope. These results imply the need to optimize information even for short glances, and to be very cautious with adaptive layouts of free programmable or dynamic displays, and not to overburden the pilot flying with parallel tasks.     

A method for measuring pilot workload based on task analysis

This paper presents a method for measuring pilot workload based on task analysis in order that the cockpit can be designed more reasonably. In addition, a prototype system is developed to use this method for the assessment of pilot workload. The method breaks the pilot's mission into several phases, segments, functions and tasks. And break pilot workload into 6 components using Wickens' "multi-resource theory", which are vision(V), vision goggles(G), auditory sensation(A), cognitive activity(C), psychomotor activity(P) and kinesthesis(K). All missions consist of those tasks and each task have 6 workload components. The workload components for each task have been acquired by a study on many pilots, so workload can be assessed. A mission analysis database is built, and the prototype system can simulate pilot's flight process and evaluate workload.     

Automation in future air traffic management: effects of decision aid reliability on controller performance and mental workload

Future air traffic management concepts envisage shared decision-making responsibilities between controllers and pilots, necessitating that controllers be supported by automated decision aids. Even as automation tools are being introduced, however, their impact on the air traffic controller is not well understood. The present experiments examined the effects of an aircraft-to-aircraft conflict decision aid on performance and mental workload of experienced, full-performance level controllers in a simulated Free Flight environment. Performance was examined with both reliable (Experiment 1) and inaccurate automation (Experiment 2). The aid improved controller performance and reduced mental workload when it functioned reliably. However, detection of a particular conflict was better under manual conditions than under automated conditions when the automation was imperfect. Potential or actual applications of the results include the design of automation and procedures for future air traffic control systems.     

Pilots' monitoring strategies and performance on automated flight decks: an empirical study combining behavioral and eye-tracking data

OBJECTIVE: The objective of the study was to examine pilots' automation monitoring strategies and performance on highly automated commercial flight decks. BACKGROUND: A considerable body of research and operational experience has documented breakdowns in pilot-automation coordination on modern flight decks. These breakdowns are often considered symptoms of monitoring failures even though, to date, only limited and mostly anecdotal data exist concerning pilots' monitoring strategies and performance. METHOD: Twenty experienced B-747-400 airline pilots flew a 1-hr scenario involving challenging automation-related events on a full-mission simulator. Behavioral, mental model, and eye-tracking data were collected. RESULTS: The findings from this study confirm that pilots monitor basic flight parameters to a much greater extent than visual indications of the automation configuration. More specifically, they frequently fail to verify manual mode selections or notice automatic mode changes. In other cases, they do not process mode annunciations in sufficient depth to understand their implications for aircraft behavior. Low system observability and gaps in pilots' understanding of complex automation modes were shown to contribute to these problems. CONCLUSION: Our findings describe and explain shortcomings in pilot's automation monitoring strategies and performance based on converging behavioral, eye-tracking, and mental model data. They confirm that monitoring failures are one major contributor to breakdowns in pilot-automation interaction. APPLICATION: The findings from this research can inform the design of improved training programs and automation interfaces that support more effective system monitoring.     

Air-to-ground training missions: a psychophysiological workload analysis

Psychophysiological measures are used to assess the workload of F4 Phantom aircraft pilots and weapon systems officers (WSOs) during air-to-ground training missions and during the performance of two levels of difficulty of a laboratory tracking task. The bombing range portion of the missions was associated with the highest pilot workload, while the WSO flying the aircraft was the highest workload segment for the WSOs. The pilots' data were found to have a wider range of values for the physiological measures than were found in the WSO data. The different levels of tracking task difficulty produced significant physiological effects but the range of values found for most of the flight segments were much greater. These data demonstrate that extrapolating laboratory data to the flight environment is risky at best. The various physiological measures were differentially sensitive to the different demands of the various flight segments.     

Use of Highways in the Sky and a virtual pad for landing Head Up Display symbology to enable improved helicopter pilots situation awareness and workload in degraded visual conditions

Abstract

Flight within degraded visual conditions is a great challenge to pilots of rotary-wing craft. Environmental cues typically used to guide interpretation of speed, location and approach can become obscured, forcing the pilots to rely on data available from in-cockpit instrumentation. To ease the task of flight during degraded visual conditions, pilots require easy access to flight critical information. The current study examined the effect of ‘Highways in the Sky’ symbology and a conformal virtual pad for landing presented using a Head Up Display (HUD) on pilots’ workload and situation awareness for both clear and degraded conditions across a series of simulated rotary-wing approach and landings. Results suggest that access to the HUD lead to significant improvements to pilots’ situation awareness, especially within degraded visual conditions. Importantly, access to the HUD facilitated pilot awareness in all conditions. Results are discussed in terms of future HUD development.

Practitioner Summary: This paper explores the use of a novel Heads Up Display, to facilitate rotary-wing pilots’ situation awareness and workload for simulated flights in both clear and degraded visual conditions. Results suggest that access to HUD facilitated pilots’ situation awareness, especially when flying in degraded conditions.     

A Method of Applying Flight Data to Evaluate Landing Operation Performance

Abstract

Pilots’ operation has an important effect on flight safety and performance, particularly in the final landing stage when pilots need to deal with complicated operations. This study aims to determine the potential value of flight data and develop a method of evaluating a pilot’s performance during landing phase based on flight quick access recorder (QAR) data from the perspective of risk assessment. First, a Landing Operation Performance Evaluation Model was developed based on risk evaluation principles. Three landing parameters, which are touchdown distance, touchdown vertical acceleration and touchdown pitch angle, were selected as indicators to evaluate the pilots’ landing operation performance in this model. Second, the flight landing operation performance evaluation system (FLOPES) was set up based on the evaluation model. Test results showed that FLOPES can accomplish all calculation flow of operation performance evaluation. Finally, it concluded that this method is a more accurate and effective way for evaluating the landing operation performance of a flight. It could be as a practical tool for airlines to manage landing risk quantitatively and to provide a more practical support for improving training and design in aviation.Practitioner summary: This study aims to determine the potential value of flight data and to develop a method of evaluating pilot’s landing operation performance from the risk evaluation perspective. Test results showed that this method is effective and could be as a practical tool for airlines to manage landing risk and improve training.Abbreviations:QAR: Quick Access Recorder; FLOPES: Flight Landing Operation Performance Evaluation System; ICAO: International Civil Aviation Organization; IATA: International Air Transport Association; SMS: Safety Management System; CAAC: Civil Aviation Administration of China; FOQA: Flight Operations Quality Assurance; VBA: Visual Basic for Applications     

Mission control of multiple unmanned aerial vehicles: a workload analysis

With unmanned aerial vehicles (UAVs), 36 licensed pilots flew both single-UAV and dual-UAV simulated military missions. Pilots were required to navigate each UAV through a series of mission legs in one of the following three conditions: a baseline condition, an auditory autoalert condition, and an autopilot condition. Pilots were responsible for (a) mission completion, (b) target search, and (c) systems monitoring. Results revealed that both the autoalert and the autopilot automation improved overall performance by reducing task interference and alleviating workload. The autoalert system benefited performance both in the automated task and mission completion task, whereas the autopilot system benefited performance in the automated task, the mission completion task, and the target search task. Practical implications for the study include the suggestion that reliable automation can help alleviate task interference and reduce workload, thereby allowing pilots to better handle concurrent tasks during single- and multiple-UAV flight control.     

The virtual landing pad: facilitating rotary-wing landing operations in degraded visual environments

The safety of rotary-wing operations is significantly affected by the local weather conditions, especially during key phases of flight including hover and landing. Despite the operational flexibility of rotary-wing craft, such craft accounts for a significantly greater proportion of accidents than their fixed-wing counterparts. A key period of risk when operating rotary-wing aircraft is during operations that occur in degraded visual environments, for example as a result of thick fog. During such conditions, pilots’ workload significantly increases and their situation awareness can be greatly impeded. The current study examines the extent to which providing information to pilots via the use of a head-up display (HUD) influenced perceived workload and situation awareness, when operating in both clear and degraded visual environments. Results suggest that whilst the HUD did not benefit pilots during clear conditions, workload was reduced when operating in degraded visual conditions. Overall results demonstrate that access to the HUD reduces the difficulties associated with flying in degraded visual environments.     

Examining the Effects of Conformal Terrain Features in Advanced Head‐Up Displays on Flight Performance and Pilot Situation Awareness

Synthetic vision systems (SVS) render terrain features for pilots through cockpit displays using a GPS database and three‐dimensional graphical models. Enhanced vision systems (EVS) present infrared imagery of terrain using a forward‐looking sensor in the nose of an aircraft. The ultimate goal of SVS and EVS technologies is to support pilots in achieving safety under low‐visibility and night conditions comparable to clear, day conditions. This study assessed pilot performance and situation awareness (SA) effects of SVS and EVS imagery in an advanced head‐up display (HUD) during a simulated landing approach under instrument meteorological conditions. Videos of the landing with various HUD configurations were presented to eight pilots with a superimposed tracking task. The independent variables included four HUD feature configurations (baseline [no terrain imagery], SVS, EVS, and a combination of SVS and EVS), two visibility conditions, and four legs of the flight. Results indicated that SVS increased overall SA but degraded flight path control performance because of visual confusion with other display features. EVS increased flight path control accuracy but decreased system (aircraft) awareness because of visual distractions. The combination of SVS and EVS generated offsetting effects. Display configurations did not affect pilot spatial awareness. Flight performance was not different among phases of the approach, but levels and types of pilot SA did vary from leg to leg. These results are applicable to development of adaptive HUD features to support pilot performance. They support the use of multidimensional measures of SA for insight on pilot information processing with advanced aviation displays. © 2012 Wiley Periodicals, Inc.