Inter-subject transfer learning for EEG-based mental fatigue recognition

Mental fatigue is one of the major factors leading to human errors. To avoid failures caused by mental fatigue, researchers are working on ways to detect/monitor fatigue using different types of signals. Electroencephalography (EEG) signal is one of the most popular methods to recognize mental fatigue since it directly measures the neurophysiological activities in the brain. Current EEG-based fatigue recognition algorithms are usually subject-specific, which means a classifier needs to be trained per subject. However, as fatigue may need a relatively long period to induce, collecting training data from each new user could be time-consuming and troublesome. Calibration-free methods are desired but also challenging since significant variability of physiological signals exists among different subjects. In this paper, we proposed algorithms using inter-subject transfer learning for EEG-based mental fatigue recognition, which did not need a calibration. To explore the influence of the number of EEG channels on the algorithms’ accuracy, we also compared the cases of using one channel only and multiple channels. Random forest was applied to choose the channel that has the most distinguishable features. A public EEG fatigue dataset recorded during driving was used to validate the algorithms. EEG data from 11 subjects were selected from the dataset and leave-one-subject-out cross-validation was employed. The channel from the occipital lobe is selected when only one channel is desired. The proposed transfer learning-based algorithms using Maximum Independence Domain Adaptation (MIDA) achieved an accuracy of 73.01% with all thirty channels, and using Transfer Component Analysis (TCA) achieved 68.00% with the one selected channel.     

Effect of local wrist muscle fatigue on multiple assembly‐related psychomotor skills

Human operators are still a vital resource in modern manufacturing systems, especially for fine assembly tasks. Declining muscle strength due to physical tasks can impair psychomotor skills and can cause adverse effects on assembly quality. To understand how local muscle fatigue can influence multiple psychomotor skills, 20 subjects participated in an experiment in which four psychomotor tests were performed to mimic fine assembly tasks, and their performances under nonfatigued wrist and fatigued wrist conditions were compared. Results showed that wrist fatigue caused significant decrements in the dexterity of fingers and fingertips, but there was no significant influence on arm‐hand steadiness. Local muscle fatigue impairs psychomotor skills to different extents but does not necessarily lead to lower performance. Humans are capable of coordinating motion units to compensate for the influence of muscle fatigue, but this is rather limited. It is suggested that in assembly operations, tasks should be arranged according to different task demands on diverse psychomotor skills.     

An optimization algorithm inspired by propagation of yeast for fleet maintenance decision making problem involving fatigue structures

To minimize airline maintenance costs and maximize fleet availability, we developed a fleet maintenance decision-making model based on CBM with collaborative optimization (CO) for fatigue structures. The model is divided into two levels: a system level and a subsystem level. Different optimization routines are used at these two levels. The system level focuses on maximizing fleet availability and the subsystem level focuses on minimizing aircraft maintenance costs. Moreover, we proposed an optimization algorithm inspired by the propagation of yeast (OA/PY) to handle the situation where optimal solution is not unique. Finally, a case study regarding a fleet of 10 aircrafts is conducted, and the results demonstrated the effectiveness of the proposed algorithm. In the case study, aircraft maintenance planning (subsystem level) was obtained, and then it was adjusted with OA/PY to obtain optimal fleet maintenance plan (system level). Total incremental maintenance cost caused by the adjustment in the proposed method was reduced by 70.65%.     

Vocal fatigue induced by prolonged oral reading: Analysis and detection

This article uses prolonged oral reading corpora for various experiments to analyze and detect vocal fatigue. Vocal fatigue particularly concerns voice professionals, including teachers, telemarketing operators, users of automatic speech recognition technology and actors. In analyzing and detecting vocal fatigue, we focused our investigations on three main experiments: a prosodic analysis that can be compared to the results found in related work, a two-class Support Vector Machines (SVM) classifier into Fatigue and Non-Fatigue states using a large set of audio features and a comparison function that estimates the difference in fatigue level between two speech segments using a combination of multiple phoneme-based comparison functions. The experiments on prosodic analysis showed that vocal fatigue was not associated with an increase in fundamental frequency and voice intensity. A two-class SVM classifier using the Paralinguistic Challenge 2010 audio feature set gave an unweighted accuracy of 94.1% for the training set (10-fold cross-validation) and 68.2% for the test set. These results show that the phenomenon of vocal fatigue can be modeled and detected. The comparison function was assessed by detecting increased fatigue levels between two speech segments. The fatigue level detection performance in Equal Error Rate (EER) was 31% using all phonetic segments and yielded EER of 21% after filtering phonetic segments and 19% after filtering phonetic segments and cepstral features. These results show that some phonemes are more sensitive than others to vocal fatigue. These experiments show that the fatigued voice has specific characteristics for prolonged oral reading and suggest the feasibility of vocal fatigue detection.     

Subjective symptoms and physiological measures of fatigue in air traffic controllers

This study measured the correlation between fatigue and physiological stress symptoms in 102 air traffic controllers (ATCs) in Taiwan. The subjects were asked to complete a fatigue questionnaire and physiological measurement before and after work. The study results showed that nearly 50% of the subjects felt tired and weary after work. However, according to the results of the critical flicker frequency test, all ATCs performed better after work, and the strength of masculine index fingers improved. The systolic and diastolic blood pressure decreased after work. In addition, the levels of urinary 17-hydroxycorticosteroids (17-OHCS) at the pre-shift and post-shift stages were 1.91 ± 1.01 mg/g cr. and 1.50 ± 0.95 mg/g cr., respectively. This study indicated that, while the ATCs' subjective ratings showed possible work-induced fatigue, it did not affect their physiological response. The findings suggested that ATC is a stressful job, and that complaints regarding excess work stress should be taken seriously. Subsequently the study proposed appropriate intervention strategies to address the identified problems.Relevance to industryThis paper presents the subjective and objective measures of fatigue in ATCs. Recommendations are proposed with respect to potentially viable countermeasures to reduce the impact of fatigue in ATC operations, and achieve a broader understanding of the causal factors for fatigue in ATC.     

Determining fatigue allowances for grocery order selectors

This paper compares and contrasts four fatigue allowance worksheets commonly used to establish fatigue (relaxation) allowances for production standards; the factors, weights, degree of documentation and other operational characteristics of these worksheets are also examined. We briefly review the fatigue literature including objective physical measures of fatigue. Results of an experiment in which 11 industrial engineers independently applied the four worksheets to a standardized job analysis and video tape of a grocery order selector are reported. We conclude that inter-rater reliability and cross-validation are very low for the four worksheets and suggest validation studies using objective physiological measures of fatigue would be appropriate.     

The effects of anxiety and exercise-induced fatigue on shooting accuracy and cognitive performance in infantry soldiers

Operational performance in military settings involves physical and mental skills that are generally investigated separately in lab settings, leading to reduced ecological validity. Therefore, we investigated the effects of anxiety and exercise-induced fatigue, separately and in combination, on cognitive and shooting performance of 22 soldiers in a real-world setting. Findings indicated that soldiers' shooting accuracy and decision-making and mathematical skills decreased significantly under anxiety. Whether exercise-induced fatigue was beneficial or detrimental to task performance depended on the task at hand. The increased arousal levels through exercise prevented shooting accuracy from deteriorating in the decision task. In contrast, cognitive performance suffered from the increased arousal: participants more often failed to shoot when being fired at by an opponent and also math performance seemed to decrease. We conclude that anxiety can deteriorate soldier performance and that exercise-induced fatigue may improve or deteriorate performance in combination with anxiety depending on the nature of the task.

Practitioner Summary: Soldiers encounter anxiety and exercise-induced fatigue. We investigated to what degree these factors influence soldiers' shooting and cognitive performance. Experimental manipulation of anxiety and exercise during a representative field course indicated decreased performance under anxiety. Exercise prevented shooting accuracy from deteriorating under anxiety, although cognitive performance was negatively affected after exercise.     

飞机多疲劳结构动态成组维修决策优化方法

以多个疲劳结构组成的飞机为对象,针对此类系统存在的维护困难、停机成本大等特点,研究多疲劳结构动态成组维修优化问题.在充分考虑结构停机维修用时以及多个疲劳结构之间维修相关性对于飞机使用率和维修费用的影响下,以维修费用和使用率为优化目标,以可靠度为约束,建立多疲劳结构动态成组维修决策优化模型.考虑飞机工作环境的严酷性和动态性,基于滚动时间轴模型,提出多疲劳结构动态成组维修决策优化方法.为了充分利用传感器获得的实时状态信息降低结构服役过程中损伤不确定性对维修计划制定的影响,并使维修计划能够适应复杂多变的动态环境,当每执行完一次停机维修活动或出现新的结构状态信息时,通过将维修决策时间窗口进行不断的滚动,使维修计划能够自适应地动态调整,达到无限规划周期的效果.     

Driver fatigue detection using a genetic algorithm

Nowadays, many traffic accidents occur due to driver fatigue. Driver fatigue detection based on computer vision is one of the most hopeful applications of image recognition technology. There are several factors that reflect driver's fatigue. Many efforts have been made to develop fatigue monitoring, but most of them focus on only a single behavior, a feature of the eyes, or a head motion, or mouth motion, etc. When fatigue monitoring is implemented on a real model, it is difficult to predict the driver fatigue accurately or reliably based only on a single driver behavior. Additionally, the changes in a driver's performance are more complicated and not reliable. In this article, we represent a model that simulates a space in a real car. A web camera as a vision sensor is located to acquire video-images of the driver. Three typical characteristics of driver fatigue are involved, pupil shape, eye blinking frequency, and yawn frequency. As the influences of these characteristics on driver fatigue are quite different from each other, we propose a genetic algorithm (GA)-based neural network (NN) system to fuse these three parameters. We use the GA to determine the structure of the neural network system. Finally, simulation results show that the proposed fatigue monitoring system detects driver fatigue probability more exactly and robustly. This work was presented in part at the 11th International Symposium on Artificial Life and Robotics, Oita, Japan, January 23–25, 2006     

Development and initial assessment of objective fatigue measures for apple harvest work

Previous research has shown that neck, back and shoulder musculoskeletal strain is a major occupational health problem affecting migrant orchard harvest workers. Researchers seek to measure the effect of an ergonomic modification to the apple picking bucket on muscle fatigue, however objective measures for use in the orchard are not yet available.

The purpose of this study is to develop simple back, shoulder or arm strength measures, which detect statistically significant drops in strength over one workday. Candidate muscle strength measures were piloted in the laboratory, adapted for the orchard and evaluated (n=102). Data were analyzed for morning to afternoon fatigue, and for correlation between fatigue score and hours worked.

In the laboratory, the timed arm hold (35.7% time reduction, 95% CI: 21.81–49.61), and the timed spinal extension (31.8% time reduction, 95% CI: 23.54–39.96) showed significant fatigue. In the orchard (n=102), only the timed arm hold showed significant (11.4%, p<.0001) fatigue. The potential effect of field conditions and subject motivation on these results needs further exploration.     

Multivariate autoregressive models and kernel learning algorithms for classifying driving mental fatigue based on electroencephalographic

Long-term driving is a significant cause of fatigue-related accidents. Driving mental fatigue has major implications for transportation system safety. Monitoring physiological signal while driving can provide the possibility to detect the mental fatigue and give the necessary warning. In this paper an EEG-based fatigue countermeasure algorithm is presented to classify the driving mental fatigue. The features of multichannel electroencephalographic (EEG) signals of frontal, central and occipital are extracted by multivariate autoregressive (MVAR) model. Then kernel principal component analysis (KPCA) and support vector machines (SVM) are employed to identify three-class EEG-based driving mental fatigue. The results show that KPCA–SVM method is able to effectively reduce the dimensionality of the feature vectors, speed up the convergence in the training of SVM and achieve higher recognition accuracy (81.64%) of three driving mental fatigue states in 10 subjects. The KPCA–SVM method could be a potential tool for classification of driving mental fatigue.     

Organisational factors and scheduling in locomotive engineers and conductors: Effects on fatigue, health and social well-being

This study examines critical organisational factors and work scheduling in railway freight operators to understand how job-related factors are related to fatigue, health and social well-being. A 148-item questionnaire was developed and distributed to a sample of 276 locomotive engineers and conductors working for the U.S. Operations of a North American Railway. One hundred and twenty-five questionnaires were returned, which was a response rate of 45.3%. Structural equation modeling was performed to identify the relationships between the examined factors. The analytical results of this study indicate that organisational factors and the scheduling system could not be distinguished as two separate factors. The scheduling system is not just the practice of assigning locomotive crewmen to trains, but it is a function tightly connected with other organisational factors. Social Well-Being is an important mediator between Scheduling and Fatigue. Finally, the study revealed a strong relationship between fatigue and health complaints.     

Work and fatigue characteristics of unsuited and suited humans during isolated isokinetic joint motions

The effects of a pressurized suit on human performance were investigated. The suit is known as an Extra-Vehicular Mobility Unit (EMU) and is worn by astronauts while working outside their spacecraft in a low earth orbit. Isolated isokinetic joint torques of three female and three male subjects (all experienced users of the suit in 1G gravity) were measured while working at 100% and 80% of their maximum voluntary torque (MVT, which is synonymous with maximum voluntary contraction (MVC)). It was found that the average decrease in the total amount of work (the sum of the work in each repetition until fatigue) done when the subjects were wearing the EMU were 48% and 41% while working at 100% and 80% MVT, respectively. There is a clear relationship between the MVT and the time and amount of work done until fatigue. Here, the time to fatigue is defined as the ending time of the repetition for which the computed work done during that repetition dropped below 50% of the work done during the first repetition. In general the stronger joints took longer to fatigue and did more work than the weaker joints. It was found that the EMU decreases the work output at the wrist and shoulder joints the most, due to the EMU joint geometry. The EMU also decreased the joint range of motion. The average total amount of work done by the test subjects increased by 5.2% (20.4%) for the unsuited (suited) case, when the test subjects decreased the level of effort from 100% to 80% MVT. Also, the average time to fatigue increased by 9.2% (25.6%) for the unsuited (suited) case, when the test subjects decreased the level of effort from 100% to 80% MVT. It was also found that the experimentally measured torque decay could be predicted by a logarithmic equation. The absolute average errors in the predictions were found to be 18.3% and 18.9% for the unsuited and suited subjects, respectively, when working at 100% MVT, and 22.5% and 18.8% for the unsuited and suited subjects, respectively, when working at 80% MVT. These results could be very useful in the design of future EMU suits and the planning of Extra-Vehicular Activity (EVA) for the future International Space Station assembly operations.     

Work and fatigue characteristics of unsuited and suited humans during isolated isokinetic joint motions

The effects of a pressurized suit on human performance were investigated. The suit is known as an Extra-Vehicular Mobility Unit (EMU) and is worn by astronauts while working outside their spacecraft in a low earth orbit. Isolated isokinetic joint torques of three female and three male subjects (all experienced users of the suit in 1G gravity) were measured while working at 100% and 80% of their maximum voluntary torque (MVT, which is synonymous with maximum voluntary contraction (MVC)). It was found that the average decrease in the total amount of work (the sum of the work in each repetition until fatigue) done when the subjects were wearing the EMU were 48% and 41% while working at 100% and 80% MVT, respectively. There is a clear relationship between the MVT and the time and amount of work done until fatigue. Here, the time to fatigue is defined as the ending time of the repetition for which the computed work done during that repetition dropped below 50% of the work done during the first repetition. In general the stronger joints took longer to fatigue and did more work than the weaker joints. It was found that the EMU decreases the work output at the wrist and shoulder joints the most, due to the EMU joint geometry. The EMU also decreased the joint range of motion. The average total amount of work done by the test subjects increased by 5.2% (20.4%) for the unsuited (suited) case, when the test subjects decreased the level of effort from 100% to 80% MVT. Also, the average time to fatigue increased by 9.2% (25.6%) for the unsuited (suited) case, when the test subjects decreased the level of effort from 100% to 80% MVT. It was also found that the experimentally measured torque decay could be predicted by a logarithmic equation. The absolute average errors in the predictions were found to be 18.3% and 18.9% for the unsuited and suited subjects, respectively, when working at 100% MVT, and 22.5% and 18.8% for the unsuited and suited subjects, respectively, when working at 80% MVT. These results could be very useful in the design of future EMU suits and the planning of Extra-Vehicular Activity (EVA) for the future International Space Station assembly operations.     

Relationship between phoria and visual fatigue in autostereoscopic 3D displays

Many people complain about visual fatigue arising from viewing three‐dimensional (3D) displays. This paper investigates relationship between visual fatigue and viewers' phoria for viewing autostereoscopic 3D displays. Visual fatigue is evaluated through subjective symptoms with a questionnaire and optometric indicators comprising fusion range as well as accommodation convergence/accommodation (AC/A) ratio to measure the variation in visual functions. A screening test is adopted to divide the subjects into two groups based on whether they suffer from phoria. Then a 2 × 2 × 2 mixed design experiment is conducted with display type, viewing stage, and visual state as factors to examine visual fatigue during viewing session. The results show that phoria subjects obtain more severe visual fatigue than normal on subjective evaluation. The normal subjects reveal a more marked difference with phoria in fusion range and AC/A ratio after viewing 3D video clip. Fusion range can significantly distinguish between the two‐dimensional (2D) and 3D condition as well as between the pre‐ and post‐viewing stages. The sensitivity and specificity of fusion range is higher than AC/A ratio with respect to viewing of 3D contents, so it is more appropriate as an optometric indicator of visual fatigue for autostereoscopic 3D displays.     

Effects of intermittent odours on cognitive-motor performance and brain functioning during mental fatigue

Effects of intermittent presentation of odours on cognitive-motor performance and brain activity during mental fatigue were examined using event-related brain potentials. Participants performed a Go/NoGo task for 60 min, in both odour and air control conditions. The time-on-task reaction time increase was significantly smaller in the odour condition than in the air control condition. Go- and NoGo-P3 amplitudes were larger in the presence of odours than during the air control, during mental fatigue. There were no effects of odours on error negativity (Ne)/error-related negativity (ERN) amplitude and latency. These results suggest that the presence of intermittent odours improves attentional/effortful control of response selection, and that this effect mitigates the deterioration of cognitive-motor performance during mental fatigue. PRACTITIONER SUMMARY: The present study provides evidence for a potentially effective strategy, the use of odours, to mitigate deficits in cognitive-motor performance during time-on-task. The results show that the presence of intermittent odours is an efficient tool for maintenance of attention and reaction time during a prolonged task.     

Measures of localized spinal muscle fatigue

The objective of this study was to determine the best variable, if any, to indicate the level of localized muscle fatigue. Six male and six female subjects were studied while they exerted their maximal voluntary contraction and 40% of maximal voluntary contraction of spinal extensors in an isometric lifting activity. The electromyography (EMG) of erectores spinae at thoracic and lumbar levels was measured bilaterally. Also, the muscle bed blood volume, level of blood oxygenation to erector spinae at L3 level and heart rate were measured. The initial and final values of subjective feelings of fatigue through visual analogue score, rate of perceived exertions and body part discomfort rating were recorded. The mean maximal voluntary contraction forces for males and females were 899 (238) N and 431 (135) N respectively. The mean durations of hold in maximal voluntary fatiguing contraction were 68.1 (39.9) s and 72.3 (37.0) s for men and women respectively. By the end of the hold the force declined to 52% for males and 62% for females. The EMG amplitudes and median frequencies also progressively declined (p < 0.01). ANOVA revealed that the task percentile values of all variables were significantly different (p < 0.01). Gender had a significant main effect (p < 0.01). The correlation coefficients between force and other individual variables were weak to modest, but significant (p < 0.001). None of the single variables predicted fatigue reliably for either gender and levels of contraction. The regression equations developed were highly significant (p < 0.01) and they explained 96 to 98% of variance in both genders and contractions.     

On the relationship between subjective and objective measures of fatigue

Objective: Objective and subjective methods have been used in the past to assess workplace fatigue, but little is known about correlations between them. We examine correlations between subjective and objective measures, including measures collected in a workplace scenario. Methods: 15 young and 17 older participants were assessed before and after work with four types of fatigue measure: objective physical (posturography), objective mental (psychomotor vigilance task), subjective physical and mental (self-assessment), objective and subjective realistic (oculomotor behaviour, observer-rated facial expression, typing performance). Results: Pre- and post-test scores were analysed with an ANOVA, significant differences were submitted to a factor analysis. It yielded three factors: one representing posturography, the second self-rated mental and physical fatigue and the third observer-rated facial expression. Conclusions: Results advocate the existence of three independent fatigue components: Objective physical fatigue, introspective and extrospective fatigue.

Practitioner Summary: This study analyses correlations between different subjective and objective fatigue markers to better understand the complex nature of workplace fatigue. Measurements were conducted directly at the workplace. Results reveal that fatigue comprises three independent fatigue components: Objective physical fatigue, introspective and extrospective fatigue.