The impact of cognitive workload on physiological arousal in young adult drivers: a field study and simulation validation

Physiological measures provide a continuous and relatively non-invasive method of characterising workload. The extent to which such measures provide sensitivity beyond that provided by driving performance metrics is more open to question. Heart rate and skin conductance were monitored during actual highway driving in response to systematically increased levels of cognitive demand using an auditory delayed digit recall task. The protocol was consistent with an earlier simulator study, providing an opportunity to assess the validity of physiological measures recorded during driving simulation. The pattern of change in heart rate with increased cognitive demand was highly consistent between field and simulator. The findings meet statistical criteria for both relative and absolute validity, although there was a trend for absolute levels to be higher under actual driving conditions. For skin conductance level, the pattern in both environments was also quite similar and a reasonable case for overall relative validity can be made. STATEMENT OF RELEVANCE: Growing complexity and multiple demands on modern drivers' attention highlight the significance of determining whether physiological measures provide increased sensitivity in workload detection. Better understanding, including whether simulator assessments provide valid measures of real-world response patterns, has implications in evaluating and refining interface designs and for developing advanced workload managers.     

The validity of driving simulation for assessing differences between in-vehicle informational interfaces: A comparison with field testing

Data from on-road and simulation studies were compared to assess the validity of measures generated in the simulator. In the on-road study, driver interaction with three manual address entry methods (keypad, touch screen and rotational controller) was assessed in an instrumented vehicle to evaluate relative usability and safety implications. A separate group of participants drove a similar protocol in a medium fidelity, fixed-base driving simulator to assess the extent to which simulator measures mirrored those obtained in the field. Visual attention and task measures mapped very closely between the two environments. In general, however, driving performance measures did not differentiate among devices at the level of demand employed in this study. The findings obtained for visual attention and task engagement suggest that medium fidelity simulation provides a safe and effective means to evaluate the effects of in-vehicle information systems (IVIS) designs on these categories of driver behaviour.

Statement of Relevance: Realistic evaluation of the user interface of IVIS has significant implications for both user acceptance and safety. This study addresses the validity of driving simulation for accurately modelling differences between interface methodologies by comparing results from the field with those from a medium fidelity, fixed-base simulator.     

Time perception as a workload measure in simulated car driving

In experimental studies using flight simulations subjects’ duration estimates have shown to be an effective indicator of cognitive task demands. In this study we wanted to find out whether subjective time perception could serve as a measure of cognitive workload during simulated car driving. Participants drove on a round course of a driving simulator consisting of three different environments with different levels of task demands. Drivers were required to perform a time-production task while driving the vehicle. Electrodermal activity and subjective ratings of mental workload (SWAT) were recorded simultaneously. The length of produced intervals increased significantly in more complex driving situations, as did electrodermal activity and subjective ratings of mental workload. Thus, time production is a valid indicator of cognitive involvement in simulated driving and could become a valid method to measure the current mental workload of car drivers in various traffic situations.     

Comparison of driving performance on-road and in a low-cost simulator using a concurrent telephone dialling task

Driving performance in an instrumented vehicle was compared with performance in a low-cost, fixed-based driving simulator. Six men and six women drove a freeway route while periodically dialling simulated phone calls. The same subjects drove a laboratory driving simulator using two visual fidelity levels: a colour scene with relatively high detail, and a monochrome (night) scene showing only road-edge markings. Lane position, speed, steering-wheel angle and throttle position were recorded in both contexts. Lane-keeping in the simulator was less precise than on the road, but speed control performance was comparable. The SD of lane position in normal driving was about twice as large, on average, in the simulator (0.360 versus 0.165 m). Lane keeping and speed control were less precise when dialling the phone than in normal driving, both in the simulator and on the road, but the performance decrement was greater in the simulator. The addition of the phone task increased the mean lateral speed in the car by about 43%, while in the simulator the mean lateral speed increased by 158% with the addition of the phone task. Subjects >60 years of age showed larger performance decrements during a concurrent phone dialling task than did subjects 20–30 years of age both in the simulator and on-road. No important differences in driving performance were found between the high and low simulator scene fidelity levels. The simulator demonstrated good absolute validity for measures of speed control and good relative validity for the effects of the phone task and age on driving precision.     

Workload perception in drone flight training simulators

Workload perception was measured in a drone flight training Simulator computerized situation. There has been increasing research in recent years on the topic of Remotely piloted aircrafts (RPA). Eleven participants were tested for workload perception during a drone flight simulator training. Reliability, sensitivity and correlations were studied for the workload scale and its relationship with the simulator training tasks. Overall, there were clear effects of mental demand as showed in the workload perception during the training tasks. Reliability for the workload scale showed good score and sensitivity showed mental demand as the most important factor compared to the other parameters measured obtaining highest correlations with landing tasks and number of errors. In our results, we have seen how the AWT (adapted from NASA-TLX) showed good sensitivity in assessing the mental burden of participants. In our research, participants scoring higher in the mental demand subscale showed greater difficulty finishing training tasks, and also showed longer time delays in performing both training sections of the simulation. These types of tools measuring workload perception and virtual training systems can be used in future research, to see how this cognitive aspect affects piloting skills and its possible safety and training implications.     

Mental workload measurement for emergency operating procedures in digital nuclear power plants

Mental workload is a major consideration for the design of emergency operation procedures (EOPs) in nuclear power plants. Continuous and objective measures are desired. This paper compares seven mental workload measurement methods (pupil size, blink rate, blink duration, heart rate variability, parasympathetic/sympathetic ratio, total power and (Goals, Operations, Methods, and Section Rules)-(Keystroke Level Model) GOMS-KLM-based workload index) with regard to sensitivity, validity and intrusiveness. Eighteen participants performed two computerised EOPs of different complexity levels, and mental workload measures were collected during the experiment. The results show that the blink rate is sensitive to both the difference in the overall task complexity and changes in peak complexity within EOPs, that the error rate is sensitive to the level of arousal and correlate to the step error rate and that blink duration increases over the task period in both low and high complexity EOPs. Cardiac measures were able to distinguish tasks with different overall complexity. The intrusiveness of the physiological instruments is acceptable. Finally, the six physiological measures were integrated using group method of data handling to predict perceived overall mental workload.

Practitioner Summary: The study compared seven measures for evaluating the mental workload with emergency operation procedure in nuclear power plants. An experiment with simulated procedures was carried out, and the results show that eye response measures are useful for assessing temporal changes of workload whereas cardiac measures are useful for evaluating the overall workload.     

Assessment and monitoring of mental workload in subway train operations using physiological,subjective, and performance measures

Subway train operation is a complex, sociotechnical system that involves a variety of cognitively demanding tasks. The train operators are responsible for continuously monitoring the surrounding environment, maintaining awareness, processing information, and making decisions under risk. The resulting mental strain on operators can negatively affect their performance and the interaction of the human–machine system. The objective of this study was to evaluate if physiological, subjective, and performance measures could identify the level of mental workloads arising from routine and nonroutine operations in the subway system. A total of 11 subway train operators underwent different driving scenarios in a high‐fidelity simulator. The simulated tasks were divided into two categories: routine operations (preparing to drive and driving between stations without interruptions or emergencies) and nonroutine operation (responding to a tunnel fire, dealing with a high density of passengers, encountering a passenger/technician on the track, and responding to train failure). The mental workload was monitored and evaluated in these tasks using an electrocardiogram, subjective self‐rating scales, and driving performance. Both heart rate variability and performance measures (including reaction time and error rate) detected mental workload variations in the different operations. On the other hand, the subjective ratings (including NASA‐TLX) assessed the overall mental workload associated with a task, without explaining the mental demand variations within the task over time. Subway train drivers experienced different levels of mental workload during routine and nonroutine driving conditions. The findings of this study can be used to extract mental workload limits to optimize workload levels during train operations.     

What determines drivers’ speed? A replication of three behavioural adaptation experiments in a single driving simulator study

We conceptually replicated three highly cited experiments on speed adaptation, by measuring drivers’ experienced risk (galvanic skin response; GSR), experienced task difficulty (self-reported task effort; SRTE) and safety margins (time-to-line-crossing; TLC) in a single experiment. The three measures were compared using a nonparametric index that captures the criteria of constancy during self-paced driving and sensitivity during forced-paced driving. In a driving simulator, 24 participants completed two forced-paced and one self-paced run. Each run held four different lane width conditions. Results showed that participants drove faster on wider lanes, thus confirming the expected speed adaptation. None of the three measures offered persuasive evidence for speed adaptation because they failed either the sensitivity criterion (GSR) or the constancy criterion (TLC, SRTE). An additional measure, steering reversal rate, outperformed the other three measures regarding sensitivity and constancy, prompting a further evaluation of the role of control activity in speed adaptation.

Practitioner Summary: Results from a driving simulator experiment suggest that it is not experienced risk, experienced effort or safety margins that govern drivers’ choice of speed. Rather, our findings suggest that steering reversal rate has an explanatory role in speed adaptation.     

Workload measurement using physiological and activity measures for validation test: A case study for the main control room of a nuclear power plant

In design of safety-critical and social-technical systems such as a nuclear power plant, practitioners are required to conduct a performance-based Integrated System Validation (ISV) test to verify that the system design could support the safe operation of the plant. Measurement of workload should be included. However, subjective workload measurements could not provide detailed information and continuous monitoring of the changing workload. This study compared physiological (heart rate difference, heart rate variability, respiration rate and breathing wave amplitude) and activity (number of walking steps, peak acceleration, activity level, and inclination) measures with workload defined as intensity of task demand and estimated with a task complexity measure in an ISV test. The test was conducted on a full-scale simulator using a beyond design-basis accident scenario. The results show that heart rate difference and respiration rate are positively correlated with the estimated workload, while heart rate variability and breathing wave amplitude are negatively correlated with the estimated workload. For operations using traditional panels, high workload is accompanied by larger number of walking steps, higher activity level, and smaller angles of inclination. It is suggested that continuous monitoring of cardiovascular, respiration, and activity measures can detect workload change during the ISV test. Relevance to industry: This study provides recommendations for continuous monitoring of workload during an ISV test of a nuclear power plant. The identified physiological and activity measures can be applied in detecting workload change. The findings are supportive in meeting regulatory requirements and improving system design in the nuclear domain.     

Automation transparency: implications of uncertainty communication for human-automation interaction and interfaces

Operators of highly automated driving systems may exhibit behaviour characteristic for overtrust issues due to an insufficient awareness of automation fallibility. Consequently, situation awareness in critical situations is reduced and safe driving performance following emergency takeovers is impeded. A driving simulator study was used to assess the impact of dynamically communicating system uncertainties on monitoring, trust, workload, takeovers, and physiological responses. The uncertainty information was conveyed visually using a stylised heart beat combined with a numerical display and users were engaged in a visual search task. Multilevel analysis results suggest that uncertainty communication helps operators calibrate their trust and gain situation awareness prior to critical situations, resulting in safer takeovers. In addition, eye tracking data indicate that operators can adjust their gaze behaviour in correspondence with the level of uncertainty. However, conveying uncertainties using a visual display significantly increases operator workload and impedes users in the execution of non-driving related tasks.

Practitioner Summary: This article illustrates how the communication of system uncertainty information helps operators calibrate their trust in automation and, consequently, gain situation awareness. Multilevel analysis results of a driving simulator study affirm the benefits for trust calibration and highlight that operators adjust their behaviour according to multiple uncertainty levels.     

Multi-modal demands of a smartphone used to place calls and enter addresses during highway driving relative to two embedded systems

There is limited research on trade-offs in demand between manual and voice interfaces of embedded and portable technologies. Mehler et al. identified differences in driving performance, visual engagement and workload between two contrasting embedded vehicle system designs (Chevrolet MyLink and Volvo Sensus). The current study extends this work by comparing these embedded systems with a smartphone (Samsung Galaxy S4). None of the voice interfaces eliminated visual demand. Relative to placing calls manually, both embedded voice interfaces resulted in less eyes-off-road time than the smartphone. Errors were most frequent when calling contacts using the smartphone. The smartphone and MyLink allowed addresses to be entered using compound voice commands resulting in shorter eyes-off-road time compared with the menu-based Sensus but with many more errors. Driving performance and physiological measures indicated increased demand when performing secondary tasks relative to ‘just driving’, but were not significantly different between the smartphone and embedded systems.

Practitioner Summary: The findings show that embedded system and portable device voice interfaces place fewer visual demands on the driver than manual interfaces, but they also underscore how differences in system designs can significantly affect not only the demands placed on drivers, but also the successful completion of tasks.     

Assessing mental workload in virtual reality based EOT crane operations: A multi-measure approach

BackgroundEye-movement metrics and subjective workload measures are extensively used to determine mental workload of participants. The aim of this study was to assess Electric overhead travelling (EOT) crane operators’ mental workload variability based on eye movement metrics such as fixation frequency, fixation duration, saccade duration, saccade amplitude, and fixation/saccade ratio during EOT crane operations in virtual reality (VR) based EOT crane simulator.MethodsA 2^k (k = 3) factorial experiment with factors namely, hazardous scenario, activity level, and trial was designed and conducted to demonstrate the proposed assessment approach. Throughout the experiment, we recorded the eye movements of 12 EOT crane operators of a steel industry of authors’ country. Post experiment, the National Aeronautics and Space Administration task load index (NASA-TLX) was adopted as a subjective workload measure and run time of task completion was recorded. Eye-movement metrics, subjective workload measure, run time were tested with multivariate analysis of variance (MANOVA), and three way repeated measure analysis of variance (ANOVA).ResultsAt the level of α = 0.05, the experimental factors significantly influence the means of eye movement metrics, subjective ratings and run time. There was also significant influence among their interactions. A positive correlation was also found for eye movements metrics with NASA-TLX and run time.ConclusionsEye movement metrics help in understanding the mental workload of participants unobtrusively and continuously. Analysis of subjective workload measure and run time along with eye-gaze analysis provide a deeper understanding on the pattern of mental workload.     

Cardiovascular state changes during performance of a simulated ambulance dispatchers' task: Potential use for adaptive support

Adaptive support has the potential to keep the operator optimally motivated, involved, and able to perform a task. In order to use such support, the operator's state has to be determined from physiological parameters and task performance measures. In an environment where the task of an ambulance dispatcher was simulated, two studies have been carried out to evaluate the feasibility of using cardiovascular measures for adaptive support. During performance of this 2–3 h lasting planning task, a pattern of results is found that can be characterized by an initial increase of blood pressure and heart rate and a decrease of heart rate variability (defense reaction pattern) followed by an ongoing increase of blood pressure counteracted by a decrease in heart rate. This pattern can be explained by an augmented short-term blood pressure control (baroreflex), which is reflected in an increase of baroreflex sensitivity. Additionally, in this latter phase heart rate variability (HRV) increases as a function of time, while blood pressure variability decreases. In the two studies performed, the baroreflex pattern was consistent for all the relevant variables.In both studies there were periods with high and low workload. Effects of task load are mainly reflected in the variability measures, while in the second study, additionally, blood pressure level was higher during periods with high task demands.The conclusion of the studies is that consistent cardiovascular response patterns can be recognized during this semi-realistic planning task, where variability measures are most sensitive to task demand changes, while blood pressure and baroreflex sensitivity are most informative with respect to cardiovascular state changes. These findings can be seen as a great potential benefit for future use in adaptive support applications.     

Psychophysiological evaluation of simulator sickness evoked by a graphic simulator

The present study investigated the effects of simulator sickness, as an important bias factor on evaluation of emotional changes under the controlled condition of driving a car for 60 min at a constant speed (60 km/h) in a graphic simulator. Simulator sickness was measured and analyzed every 5 min using both subjective evaluation and physiological signals. Results of the subjective evaluation showed there was a significant difference between the rest and the driving conditions 10 min after the main experiment started and that the level of difference increased linearly with time. Analysis of the central and the autonomic nervous systems showed the significant differences in delta, theta, alpha and beta bands of an electroencephalogram (EEG), skin temperature, and the R-R interval between the rest and the driving conditions after about 5 min from the start of driving. In particular, there was the highest correlation between parameter of theta and subjective evaluation, and thus theta was considered an effective physiological parameter for numerically evaluating simulator sickness. The results indicate that physiological changes due to simulator sickness can be a bias factor in evaluation of human sensibility.     

Speech Recognition and In-Vehicle Telematics Devices: Potential Reductions in Driver Distraction

Speech Recognition is frequently cited as a potential remedy to distraction resulting from drivers' operation of in-vehicle devices. This position typically assumes that the introduction of speech recognition will result in reduced cognitive workload and improved driving performance. Past research neither fully supports nor fully discounts this assumption. However, it is difficult to compare many of these studies, due to differences in device operation tasks, the pacing of those tasks, speech recognition system performance, and system interface designs. In an effort to directly address the effect of voice recognition on driver distraction, the present authors developed a capability to manipulate the performance characteristics of a speech recognition system through a Wizard of Oz speech recognition system and installed this system in a simulated driving environment. The sensitivity of the simulated driving environment and speech recognition accuracy manipulation were evaluated in an initial study comparing driver cognitive workload and driving performance during self-paced simulated operation of a personal digital assistant (PDA) during no PDA use, manual control of the PDA, and speech control of the PDA. In the Speech PDA condition, speech recognition accuracy was varied between drivers. Analysis of drivers' emergency braking response times and rated cognitive workload revealed significantly lower cognitive demand and better performance in the No PDA condition when compared to the Manual PDA condition. The Speech PDA condition resulted in response times and rated cognitive workload levels that were between the No PDA and Manual PDA conditions, but not significantly different from either of these conditions. Further analysis of emergency braking performance revealed a non-significant trend towards better performance in conjunction with higher speech recognition accuracy levels. The potential for reducing driver distraction through the careful development and evaluation of speech recognition systems is discussed.     

Human interaction with robotic systems: performance and workload evaluations

Abstract

We first tested the effect of differing tactile informational forms (i.e. directional cues vs. static cues vs. dynamic cues) on objective performance and perceived workload in a collaborative human–robot task. A second experiment evaluated the influence of task load and informational message type (i.e. single words vs. grouped phrases) on that same collaborative task. In both experiments, the relationship of personal characteristics (attentional control and spatial ability) to performance and workload was also measured. In addition to objective performance and self-report of cognitive load, we evaluated different physiological responses in each experiment. Results showed a performance–workload association for directional cues, message type and task load. EEG measures however, proved generally insensitive to such task load manipulations. Where significant EEG effects were observed, right hemisphere amplitude differences predominated, although unexpectedly these latter relationships were negative. Although EEG measures were partially associated with performance, they appear to possess limited utility as measures of workload in association with tactile displays.

Practitioner Summary: As practitioners look to take advantage of innovative tactile displays in complex operational realms like human–robotic interaction, associated performance effects are mediated by cognitive workload. Despite some patterns of association, reliable reflections of operator state can be difficult to discern and employ as the number, complexity and sophistication of these respective measures themselves increase.     

Measuring mental workload with the NASA-TLX needs to examine each dimension rather than relying on the global score: an example with driving

The distinction between several components of mental workload is often made in the ergonomics literature. However, measurements used are often established from a global score, notably with several questionnaires that originally reflect several dimensions. The present study tested the effect of driving situation complexity, experience and subjective levels of tension and alertness on each dimension of the NASA-TLX questionnaire of workload, in order to highlight the potential influence of intrinsic, extraneous and germane load factors. The results showed that, in complex situation, mental, temporal and physical demand (load dimensions) increased, and that novice drivers presented high physical demand when subjective tension was low on performance. Moreover, increase of mental and physical demand increased effort. It thus, appears essential to distinguish the different components of mental workload used in the NASA-TLX questionnaire.

Practitioner Summary: Currently, global score of NASA-TLX questionnaire is used to measure mental workload. Here, we considered independently each dimension of NASA-TLX, and results showed that mental load factors (driving situation complexity, experience, subjective tension and alertness) had a different effect on dimensions, questioning global score use to evaluate workload.     

Development and validation of a behavioural video coding scheme for detecting mental workload in manual assembly

Abstract

Manual assembly in the future Industry 4.0 workplace will put high demands on operators’ cognitive processing. The development of mental workload (MWL) measures therefore looms large. Physiological gauges such as electroencephalography (EEG) show promising possibilities, but still lack sufficient reliability when applied in the field. This study presents an alternative measure with a substantial ecological validity. First, we developed a behavioural video coding scheme identifying 11 assembly behaviours potentially revealing MWL being too high. Subsequently, we explored its validity by analysing videos of 24 participants performing a high and a low complexity assembly. Results showed that five of the behaviours identified, such as freezing and the amount of part rotations, significantly differed in occurrence and/or duration between the two conditions. The study hereby proposes a novel and naturalistic method that could help practitioners to map and redesign critical assembly phases, and researchers to enrich validation of MWL-measures through measurement triangulation.

Practitioner summary: Current physiological mental workload (MWL) measures still lack sufficient reliability when applied in the field. Therefore, we identified several observable assembly behaviours that could reveal MWL being too high. The results propose a method to map MWL by observing specific assembly behaviours such as freezing and rotating parts.

Abbreviations: MWL: mental workload; EEG: electroencephalography; fNIRS: functional near infrared spectroscopy; AOI: area of interest; SMI: SensoMotoric Instruments, ETG: Eye-Tracking Glasses; FPS: frames per second; BORIS: Behavioral Observation Research Interactive Software; IRR: inter-rater reliability; SWAT: Subjective Workload Assessment Technique; NASA-TLX: National Aeronautics and Space Administration Task Load Index; EL: emotional load; DSSQ: Dundee Stress State Questionnaire; PHL: physical load; SBO: Strategisch Basis Onderzoek     

Physiological workload reactions to increasing levels of task difficulty

The sensitivity of physiological measures to mental workload was investigated in a flight simulator. Twelve pilots had to fly through a tunnel with varying levels of difficulty. Additionally, they had to perform a memory task with four levels of difficulty. The easiest memory task was combined with the easiest tunnel task and the most difficult memory task with the most difficult tunnel task. Between the tunnel tasks, subjects had to fly a pursuit task in which a target jet had to be followed. Rest periods before and after the experiment were used as a baseline for the physiological measures. Mental workload was measured with heart period, continuous blood pressure, respiration and eye blinks. Several respiratory parameters, heart rate variability, blood pressure variability and the gain between systolic blood pressure and heart period (modulus) were scored. All measures showed differences between rest and flight, and between the pursuit and the tunnel task. Only heart period was sensitive to difficulty levels in the tunnel task. Heart rate variability increased when respiratory activity around 0.10 Hz increased, which occurred often. The modulus was hardly influenced by respiration and therefore appears to be a better measure than heart rate variability. Among the respiratory parameters, the duration of a respiratory cycle was the most sensitive to changes in workload. The time in between two successive eye blinks (blink interval) increased and the blink duration decreased as more visual information had to be processed. Increasing the difficulty of the memory task led to a decrement in blink interval, probably caused by subvocal activity during rehearsal of target letters. The data show that physiological measures are sensitive to mental effort, whereas rating scales are sensitive to both mental effort and task difficulty.