Physiological workload reactions to increasing levels of task difficulty

The sensitivity of physiological measures to mental workload was investigated in a flight simulator. Twelve pilots had to fly through a tunnel with varying levels of difficulty. Additionally, they had to perform a memory task with four levels of difficulty. The easiest memory task was combined with the easiest tunnel task and the most difficult memory task with the most difficult tunnel task. Between the tunnel tasks, subjects had to fly a pursuit task in which a target jet had to be followed. Rest periods before and after the experiment were used as a baseline for the physiological measures. Mental workload was measured with heart period, continuous blood pressure, respiration and eye blinks. Several respiratory parameters, heart rate variability, blood pressure variability and the gain between systolic blood pressure and heart period (modulus) were scored. All measures showed differences between rest and flight, and between the pursuit and the tunnel task. Only heart period was sensitive to difficulty levels in the tunnel task. Heart rate variability increased when respiratory activity around 0.10 Hz increased, which occurred often. The modulus was hardly influenced by respiration and therefore appears to be a better measure than heart rate variability. Among the respiratory parameters, the duration of a respiratory cycle was the most sensitive to changes in workload. The time in between two successive eye blinks (blink interval) increased and the blink duration decreased as more visual information had to be processed. Increasing the difficulty of the memory task led to a decrement in blink interval, probably caused by subvocal activity during rehearsal of target letters. The data show that physiological measures are sensitive to mental effort, whereas rating scales are sensitive to both mental effort and task difficulty.     

Psychophysiology of the passive user: Exploring the effect of technological conditions and personality traits

This purpose of this study was to investigate physiological response patterns of passive users versus active users under varying technological conditions. The relationship between user personality traits and physiological responses was also explored. In work systems, different types of users interact with technologies in different ways. Active users may control technologies, while passive users may watch the active user and the technology. While they are both important stakeholders in the system, little empirical research has been conducted to understand the passive users to date. A mixed design experiment was conducted to investigate passive user psychophysiological state, while active and passive users operated a technology under varying technological states (normal, unreliable, and difficult). Physiological measures were collected from passive users while technologies were being operated, these included electrodermal and cardiovascular (heart rate and heart rate variability). Results show that individual active users and passive users have distinct physiological response patterns in heart rate and heart rate variability while using the technology. Exploratory analyses indicated that conscientiousness was positively related to individual active users' task performance; openness, agreeableness, and extraversion were associated with passive users' electrodermal and cardiovascular responses. In addition, social interaction is more influential for passive users' physiological responses, compared to task performing. The effects of active and passive user personality traits on technology interactions are illustrated through distinctive physiological responses.Relevance to industryFindings from this study suggest that to improve passive user experience, systems that include both active and passive users should be designed to facilitate communication and information sharing between users.     

Dual-task performance in multimodal human-computer interaction: a psychophysiological perspective

This paper examines the psychophysiological effects of mental workload in single-task and dual-task human-computer interaction. A mental arithmetic task and a manual error correction task were performed both separately and concurrently on a computer using verbal and haptic input devices. Heart rate, skin conductance, respiration and peripheral skin temperature were recorded in addition to objective performance measures and self-report questionnaires. Analysis of psychophysiological responses found significant changes from baseline for both single-task and dual-task conditions. There were also significant psychophysiological differences between the mental arithmetic task and the manual error correction task, but no differences in questionnaire results. Additionally, there was no significant psychophysiological difference between performing only the mental arithmetic task and performing both tasks at once. These findings suggest that psychophysiological measures respond differently to different types of tasks and that they do not always agree with performance or with participants’ subjective feelings.     

A tool to assess the comfort of wearable computers

Wearable computer comfort can be affected by numerous factors, making its assessment based on one value with one scale inappropriate. This paper presents a tool that measures wearable comfort across six dimensions: emotion, attachment, harm, perceived change, movement, and anxiety. The dimensions for these comfort rating scales were specifically developed for wearable equipment assessment by applying multidimensional scaling to a comfort term association matrix developed using the results of groupings of wearable computer comfort terms. Testing the scales on four different types of wearable computer showed that the scales can be used to highlight differences in comfort between different types of technology for different aspects of comfort. An intraclass correlation of .872 suggested that the scales were used with a high level of reliability. A second study showed that modifications made to a wearable computer resulted in improvements in comfort, although they were not significant (p > .05). A potential application for this research is as an aid to designers and researchers for assessing the wearability, in terms of comfort, of wearable computer devices and to determine the effectiveness of any modifications made to the design of a wearable device.     

Heart period variability and respiratory changes associated with physical and mental load: non-linear analysis

In the investigation of heart rate and heart rate variability, the discrimination between mental workload, physical activity and respiration is known to be methodologically difficult. At most, heart rate variability measures are more likely to be coarse-grained measures with variability confounded by heart rate. Moreover, the spectral analysis of heart rate variability shows broad-band frequency characteristics, pointing towards non-stationarity or non-linearity. From this it is suggested to focus on non-linear dynamic analyses that are variance-insensitive. The experimental section of the paper focuses on the estimation of two non-linear measures for both heartbeat dynamics and respiration, the correlation dimension indicating complexity and the Lyapunov exponents indicating predictability. The results indicate that the complexity of heart dynamics is related to the type of task and that the predictability of heart dynamics is related to the amount of load.     

Workload measurement using physiological and activity measures for validation test: A case study for the main control room of a nuclear power plant

In design of safety-critical and social-technical systems such as a nuclear power plant, practitioners are required to conduct a performance-based Integrated System Validation (ISV) test to verify that the system design could support the safe operation of the plant. Measurement of workload should be included. However, subjective workload measurements could not provide detailed information and continuous monitoring of the changing workload. This study compared physiological (heart rate difference, heart rate variability, respiration rate and breathing wave amplitude) and activity (number of walking steps, peak acceleration, activity level, and inclination) measures with workload defined as intensity of task demand and estimated with a task complexity measure in an ISV test. The test was conducted on a full-scale simulator using a beyond design-basis accident scenario. The results show that heart rate difference and respiration rate are positively correlated with the estimated workload, while heart rate variability and breathing wave amplitude are negatively correlated with the estimated workload. For operations using traditional panels, high workload is accompanied by larger number of walking steps, higher activity level, and smaller angles of inclination. It is suggested that continuous monitoring of cardiovascular, respiration, and activity measures can detect workload change during the ISV test. Relevance to industry: This study provides recommendations for continuous monitoring of workload during an ISV test of a nuclear power plant. The identified physiological and activity measures can be applied in detecting workload change. The findings are supportive in meeting regulatory requirements and improving system design in the nuclear domain.     

Development of a wrist-worn calorie monitoring system using bluetooth

Methods and tools for monitoring real-time human body information in daily life are required for advanced healthcare. In this study, a method for estimating energy expenditure during health exercises was evaluated and a wrist-worn sensing system based on the method was developed. Pulse monitoring was used to calculate energy expenditure by estimating oxygen uptake from a correlation between heart rate and oxygen uptake. Bluetooth technology was utilized for sending data by wireless communication. By the newly developed system, energy expenditure during exercise can be estimated considering individual difference and distinguishing changes in grade or load. Our goal is to construct a miniaturized wearable system that monitors vital signs and has many applications for healthcare. The study suggests that a wearable pulse sensing system proposed could provide useful information for healthcare.     

Respiratory control of heart rate†

A new model for the respiratory Control of the heart rate is suggested. The model includes the effect of pressoreceptor feedback in the production of the heart rate transient during respiration. The reflex is assumed to be initiated by a rate sensitive stretch receptor with varying compliances for inspiration and expiration. The effect of posture in altering the transient envelope is also considered.     

Data fusion and multiple classifier systems for human activity detection and health monitoring: Review and open research directions

Activity detection and classification using different sensor modalities have emerged as revolutionary technology for real-time and autonomous monitoring in behaviour analysis, ambient assisted living, activity of daily living (ADL), elderly care, rehabilitations, entertainments and surveillance in smart home environments. Wearable devices, smart-phones and ambient environments devices are equipped with variety of sensors such as accelerometers, gyroscopes, magnetometer, heart rate, pressure and wearable camera for activity detection and monitoring. These sensors are pre-processed and different feature sets such as time domain, frequency domain, wavelet transform are extracted and transform using machine learning algorithm for human activity classification and monitoring. Recently, deep learning algorithms for automatic feature representation have also been proposed to lessen the burden of reliance on handcrafted features and to increase performance accuracy. Initially, one set of sensor data, features or classifiers were used for activity recognition applications. However, there are new trends on the implementation of fusion strategies to combine sensors data, features and classifiers to provide diversity, offer higher generalization, and tackle challenging issues. For instances, combination of inertial sensors provide mechanism to differentiate activity of similar patterns and accurate posture identification while other multimodal sensor data are used for energy expenditure estimations, object localizations in smart homes and health status monitoring. Hence, the focus of this review is to provide in-depth and comprehensive analysis of data fusion and multiple classifier systems techniques for human activity recognition with emphasis on mobile and wearable devices. First, data fusion methods and modalities were presented and also feature fusion, including deep learning fusion for human activity recognition were critically analysed, and their applications, strengths and issues were identified. Furthermore, the review presents different multiple classifier system design and fusion methods that were recently proposed in literature. Finally, open research problems that require further research and improvements are identified and discussed.     

New goals for HCI training: How to mix old and new skills in the trainee

The development of human‐computer interaction systems and the acquisition of skills associated with such systems typically occur in the context of previous experience. What is learned in one situation may facilitate or impede learning in another situation. The aim of this article is to discuss the role of experience in human‐computer interaction. The ACT? theory of skill acquisition and transfer is extended to account for the effects of old skills on the learning of new tasks. The extended model predicts a number of changes in performance that will occur when a new task involves the combination of old and new skills, including the suggestion that the learning rate of the new task will be slower than the rate at which the old skills were originally acquired. Two experiments are reported, the results of which support most of the model's predictions. The results also suggest that the minimum performance time of a task may be increased if performance of the task involves combining old and new skills. Implications of the effects of such combinations are considered with respect to the best methods of training for human‐computer interaction systems and the development of such systems.     

Human interaction with robotic systems: performance and workload evaluations

Abstract

We first tested the effect of differing tactile informational forms (i.e. directional cues vs. static cues vs. dynamic cues) on objective performance and perceived workload in a collaborative human–robot task. A second experiment evaluated the influence of task load and informational message type (i.e. single words vs. grouped phrases) on that same collaborative task. In both experiments, the relationship of personal characteristics (attentional control and spatial ability) to performance and workload was also measured. In addition to objective performance and self-report of cognitive load, we evaluated different physiological responses in each experiment. Results showed a performance–workload association for directional cues, message type and task load. EEG measures however, proved generally insensitive to such task load manipulations. Where significant EEG effects were observed, right hemisphere amplitude differences predominated, although unexpectedly these latter relationships were negative. Although EEG measures were partially associated with performance, they appear to possess limited utility as measures of workload in association with tactile displays.

Practitioner Summary: As practitioners look to take advantage of innovative tactile displays in complex operational realms like human–robotic interaction, associated performance effects are mediated by cognitive workload. Despite some patterns of association, reliable reflections of operator state can be difficult to discern and employ as the number, complexity and sophistication of these respective measures themselves increase.     

The design of the SensVest

The SensVest is an item of wearable technology that measures, records and transmits aspects of human physical performance such as heart rate, temperature and movement. The SensVest has been designed for use by science teachers and students to meet their requirements. This paper reports the stages undertaken to design the SensVest, from determining appropriate methods of assessing human performance, to considerations of mounting the technology on the body. Trials have shown that concessions need to be made with ease of use and cost to ensure that the data collected is reliable and usable, with an awareness of the sensors limitations. By designing the SensVest with the wearer in mind a system has been developed that is comfortable, does not inhibit normal performance and is wearable. User trials have shown that meaningful, reliable and useful data can be collected using the SensVest.

Factors influencing the perceived usability of wearable chair exoskeleton with market segmentation: A structural equation modeling and K-Means Clustering approach

The sudden implementation of work-from-home setup has caused the rise of sedentary position among employees which led to an increase in development or worsening of musculoskeletal disorders. To help mitigate this problem, wearable assistive devices such as wearable chairs have been argued to be beneficial. Several studies have expounded on the benefit of using this, however little to no available device is yet present in the Philippines. To which, businesses and developers could capitalize on this aspect. This study examined the perceived usability of wearable chair exoskeleton in the Philippines by integrating the Unified Theory of Acceptance and Use of Technology (UTAUT2) and System Usability Scale (SUS). A questionnaire was developed and distributed to 365 residents in the Philippines to measure the factors and their relationship to perceived usability accurately. Indicators and measures used in the questionnaire were derived from existing literature on technology acceptance and usability evaluation. Partial least squares structural equation modeling (PLS-SEM) was used for this study. Results revealed that perceived usability was significantly influenced by usage behavior. At the same time, habit, hedonic motivation, and performance expectancy significantly influenced the behavioral intention of the wearable chair exoskeleton. K-Means clustering was also utilized to identify clusters of target users of the wearable chair in the country, such as high-value customers, core customers, and low-value customers. Results indicated that demographic factors such as females, 30 years old and below, earning Php45,000 and above monthly, and living in a city are highly likely to utilize the product. The findings of this study can be applied and extended as a theoretical framework for future researchers of consumer behavior and exoskeleton developers for enhancing the innovation and usability of this new technology. This study may also be used and capitalized by investors to strategize the development and marketing of the exoskeletons among industrial and teleworkers worldwide.     

The impact of general and specific performance and self-efficacy on learning with computer-based concept mapping

When students learn a new and challenging task, for which they have very limited corresponding personal or vicarious experiences to refer, it is important to understand how their self-efficacy beliefs evolve during the course of sequential lessons; how they differ in the way their general learning performance and self-efficacy influence their ratings of task-specific self-efficacy and performance; and, how such differences may result in different learning outcomes and motivations for learning. By examining a group of 66 students engaged in learning to construct good concept maps with computer software, this study revealed several important findings on these questions. Students generally tended to initially overestimate their ability to successfully carry out the required task. They also varied in the way their general self-efficacy and performance were associated with their task-specific self-efficacy and performance. And, these differences did appear to be associated with different learning outcomes and motivations for learning. These results indicate that investigating individual differences in students’ patterns of association between general and specific performance and their self-efficacy may lead to a better understanding of how students differ in their levels of motivation and outcomes when learning a new and challenging task.     

Real-time assessment of mental workload using psychophysiological measures and artificial neural networks

The functional state of the human operator is critical to optimal system performance. Degraded states of operator functioning can lead to errors and overall suboptimal system performance. Accurate assessment of operator functional state is crucial to the successful implementation of an adaptive aiding system. One method of determining operators' functional state is by monitoring their physiology. In the present study, artificial neural networks using physiological signals were used to continuously monitor, in real time, the functional state of 7 participants while they performed the Multi-Attribute Task Battery with two levels of task difficulty. Six channels of brain electrical activity and eye, heart and respiration measures were evaluated on line. The accuracy of the classifier was determined to test its utility as an on-line measure of operator state. The mean classification accuracies were 85%, 82%, and 86% for the baseline, low task difficulty, and high task difficulty conditions, respectively. The high levels of accuracy suggest that these procedures can be used to provide accurate estimates of operator functional state that can be used to provide adaptive aiding. The relative contribution of each of the 43 psychophysiological features was also determined. Actual or potential applications of this research include test and evaluation and adaptive aiding implementation.     

Deep learning-based smart task assistance in wearable augmented reality

Wearable augmented reality (AR) smart glasses have been utilized in various applications such as training, maintenance, and collaboration. However, most previous research on wearable AR technology did not effectively supported situation-aware task assistance because of AR marker-based static visualization and registration. In this study, a smart and user-centric task assistance method is proposed, which combines deep learning-based object detection and instance segmentation with wearable AR technology to provide more effective visual guidance with less cognitive load. In particular, instance segmentation using the Mask R-CNN and markerless AR are combined to overlay the 3D spatial mapping of an actual object onto its surrounding real environment. In addition, 3D spatial information with instance segmentation is used to provide 3D task guidance and navigation, which helps the user to more easily identify and understand physical objects while moving around in the physical environment. Furthermore, 2.5D or 3D replicas support the 3D annotation and collaboration between different workers without predefined 3D models. Therefore, the user can perform more realistic manufacturing tasks in dynamic environments. To verify the usability and usefulness of the proposed method, we performed quantitative and qualitative analyses by conducting two user studies: 1) matching a virtual object to a real object in a real environment, and 2) performing a realistic task, that is, the maintenance and inspection of a 3D printer. We also implemented several viable applications supporting task assistance using the proposed deep learning-based task assistance in wearable AR.     

The role of spatial abilities and age in performance in an auditory computer navigation task

Age-related differences in spatial ability have been suggested as a mediator of age-related differences in computer-based task performance. However, the vast majority of tasks studied have primarily used a visual display (e.g., graphical user interfaces). In the current study, the relationship between spatial ability and performance in a non-visual computer-based navigation task was examined in a sample of 196 participants ranging in age from 18 to 91. Participants called into a simulated interactive voice response system and carried out a variety of transactions. They also completed measures of attention, working memory, and spatial abilities. The results showed that age-related differences in spatial ability predicted a significant amount of variance in performance in the non-visual computer task, even after controlling for other abilities. Understanding the abilities that influence performance with technology may provide insight into the source of age-related performance differences in the successful use of technology.     

The role of spatial abilities and age in performance in an auditory computer navigation task

Age-related differences in spatial ability have been suggested as a mediator of age-related differences in computer-based task performance. However, the vast majority of tasks studied have primarily used a visual display (e.g., graphical user interfaces). In the current study, the relationship between spatial ability and performance in a non-visual computer-based navigation task was examined in a sample of 196 participants ranging in age from 18 to 91. Participants called into a simulated interactive voice response system and carried out a variety of transactions. They also completed measures of attention, working memory, and spatial abilities. The results showed that age-related differences in spatial ability predicted a significant amount of variance in performance in the non-visual computer task, even after controlling for other abilities. Understanding the abilities that influence performance with technology may provide insight into the source of age-related performance differences in the successful use of technology.     

Personality and inter-subject differences in performance and physiological cost during whole-body vibration

When subjects are exposed to whole-body vibration, extra effort is required to maintain pre-vibration standards of performance. Therefore the willingness of subjects to expend this effort might influence both performance and physiological cost. Willingness may be related to a personality variable-score on the locus of control scale. This hypothesis was tested in 12 subjects who performed a simulated driving task during 10 min of vertical (±Gz) whole-body vibration at energy levels of 021,0-28 and 0-35 r.m.s.g using a sinusoidal and a random waveform. Accuracy at a foot-controlled, compensatory tracking task, reaction time, oxygen uptake and heart rate were measured. Subjects with an ‘internal’ locus of control had less tracking error(p<0.001)and higher heart rates (p<0.05) than did subjects with an ‘external’ locus of control. Furthermore, both variables were significantly correlated with the locus of control scores (r= +0.73 and —0.66) respectively. These findings suggest that the inter-subject differences found in investigations using human subjects may be explained in part by personality differences related to locus of control.     

The role of respiratory measures to assess mental load in pilot selection

While cardiovascular measures have a long tradition of being used to determine operator load, responsiveness of the respiratory system to mental load has rarely been investigated. In this study, we assessed basic and variability measures of respiration rate (RR), partial pressure of end-tidal carbon dioxide (petCO₂) as well as performance measures in 63 male pilot candidates during completion of a complex cognitive task and subsequent recovery. Mental load was associated with an increase in RR and a decrease in respiratory variability. A significant decrease was also found for petCO₂. RR and respiratory variability showed partial and complete effects of recovery, respectively, whereas petCO₂ did not return to baseline level. Overall, a good performance was related to a stronger reactivity in RR. Our findings suggest that respiratory parameters would be a useful supplement to common measures for the assessment of mental load in pilot selection.

Practitioner Summary: Respiratory measures are a promising yet poorly investigated approach to monitor operator load. For pilot selection, we assessed respiration in response to multitasking in 63 candidates. Task-related changes as well as covariation with performance strongly support the consideration of respiratory parameters when evaluating reactivity to mental load.