Effects of three‐dimensional virtual reality and traditional training methods on mental workload and training performance

Industries will implement effective training programs to improve training performance, and an ideal training performance occurs under proper mental workload (MWL). Virtual reality (VR) has recently been widely utilized in training; however, only a few studies have investigated its effects on MWL and training performance simultaneously. The purpose of this study is to investigate the effects of VR training and traditional training methods, such as technical manuals (TM) and multimedia films (MF), on training performance and MWL. The results of the performance measurement show that VR training is considered the best training method compared to TM and MF, particularly in the case of complex tasks. The results of physiological measurements (GSR [galvanic skin response], LF% [low frequency], and LF/HF [high frequency] ratio) show a significant difference between reading TM and using computer (MF and VR), wherein the latter has a lower MWL. However, no significant difference in subjective MWL assessment (NASA‐TLX [task load index]) and HF% measurement is found.     

Cybersickness during VR gaming undermines game enjoyment: A mediation model

Recent advances in virtual reality (VR) technology have ushered in a new era of VR gaming. While VR gaming experience represents a burgeoning area of research within human-computer interaction circles, the role of cybersickness, physiological repercussions of VR exposure to users characterized by a multitude of symptoms, such as nausea, lightheadedness, and dizziness, in the effect of VR gaming on game enjoyment remains understudied. In two experiments, the current study proposed and tested a causal mediation model in which the effect of VR gaming on game enjoyment was mediated by the level of cybersickness experienced during the gameplay. Results from both experiments supported the proposed model and showed that increased cybersickness levels during VR gaming led to decreases in game enjoyment, indicating that cybersickness experienced during VR gaming undermines the enjoyment of the gaming experience. Results also revealed that compared to traditional desktop gaming, VR gaming invoked greater levels of cybersickness, but VR gaming did not lead to greater levels of game enjoyment. When the effect of cybersickness on game enjoyment was statistically removed, however, VR gaming was found to be more enjoyable. The current experiments provide preliminary evidence that cybersickness may hinder the enjoyment of VR gaming experience.     

A mobile VR-based respiratory biofeedback game to foster diaphragmatic breathing

Virtual reality (VR) has become popular in mental health research. Several studies have explored the use of VR in the context of biofeedback protocols. In the present paper, we report on the development and evaluation of a VR-based respiratory biofeedback game to foster diaphragmatic breathing. The game integrates respiratory biofeedback, restorative VR and gamification. The game is designed to run on a mobile, all-in-one VR headset. Notably, an integrated VR hand controller is utilized as a sensor to detect respiration-induced movements of the diaphragm. In a longitudinal within-subjects study, we explored the feasibility of the game and tested the effectiveness of six training sessions. Participants reported a pleasant user experience. Moreover, the results show that the brief VR-based breathing training increased perceived breath awareness, improved diaphragmatic breathing, increased relaxation, decreased perceived stress, reduced symptoms of burnout and boosted relaxation-related self-efficacy. Future studies need to address the generalizability and long-term stability of the results, compare the approach with existing treatments and fine-tune the training components.

     

Evaluation and prediction mental workload in user interface of maritime operations using eye response

Eye response measurement is one of the objective measure methods and useful for assessing of operators' mental workload (MWL). The main objectives of this paper are to consider the relationship between operators' MWL and eye responses in the task of operating marine engine interface. Also, an artificial neural network (ANN) model was developed to predict the operators' MWL based on integrating eye response data. Eye response indices (pupil dilation, blink rate, fixation rate, and saccadic rate) were recorded, and two subjective rating methods (The National Aeronautics and Space Administration's Task Load Index [NASA-TLX] and subjective workload assessment technique [SWAT]) were used for 27 participants. The results again confirm that the eye response is sensitive to MWL in workload levels of the task when using the interface control. The ANN model developed by measuring these indices can predict the operators' MWL with the determination coefficient (R²) of 0.971, 0.912 and 0.918 for training, validation, and testing, respectively. These results indicated that the ANN approach is quite accurate for the prediction of operators' MWL based on eye response indices.Relevance to industryThe developed model is expected to provide the operator with a reference value of their MWL by evaluating their physiological indices. This result might be applied for developing an intelligent prediction model in the actual work environment to inform or support the operator in a variety of ways. From this, the manager can organize the human resources for each task to sustain the appropriate MWL as well as to improve the work performance.     

Comparison of NASA-TLX scale,modified Cooper–Harper scale and mean inter-beat interval as measures of pilot mental workload during simulated flight tasks

Abstract

The sensitivity of NASA-TLX scale, modified Cooper–Harper (MCH) scale and the mean inter-beat interval (IBI) of successive heart beats, as measures of pilot mental workload (MWL), were evaluated in a flight training device (FTD). Operational F/A-18C pilots flew instrument approaches with varying task loads. Pilots’ performance, subjective MWL ratings and IBI were measured. Based on the pilots’ performance, three performance categories were formed; high-, medium- and low-performance. Values of the subjective rating scales and IBI were compared between categories. It was found that all measures were able to differentiate most task conditions and there was a strong, positive correlation between NASA-TLX and MCH scale. An explicit link between IBI, NASA-TLX, MCH and performance was demonstrated. While NASA-TLX, MCH and IBI have all been previously used to measure MWL, this study is the first one to investigate their association in a modern FTD, using a realistic flying mission and operational pilots.

Practitioner summary: NASA-TLX scale, MCH scale and the IBI were evaluated in a flight training device. All measures were able to differentiate most task conditions and there was a positive correlation between NASA-TLX and MCH scale. An explicit link between IBI, NASA-TLX, MCH and performance was demonstrated.

Abbreviations: ANOVA: Analysis of Variance; ECG: Electrocardiograph; F/A: fighter/attack; ft: feet; FTD: flight training device; G: Gravity; km: kilometer; m: meter; m/s: meters per second; MWL: mental workload; MCH: modified cooper-harper; NASA-TLX: NASA Task Load Index; NM: Nautical Mile; NN: normal-to-normal; IBI: inter-beat interval; ILS: Instrument Landing System; RR: R-Wave to R-Wave; SD: standard deviation; TTP: tactics, techniques and procedures; WTSAT: Weapon Tactics and Situation Awareness Trainer     

Development and validation of a behavioural video coding scheme for detecting mental workload in manual assembly

Abstract

Manual assembly in the future Industry 4.0 workplace will put high demands on operators’ cognitive processing. The development of mental workload (MWL) measures therefore looms large. Physiological gauges such as electroencephalography (EEG) show promising possibilities, but still lack sufficient reliability when applied in the field. This study presents an alternative measure with a substantial ecological validity. First, we developed a behavioural video coding scheme identifying 11 assembly behaviours potentially revealing MWL being too high. Subsequently, we explored its validity by analysing videos of 24 participants performing a high and a low complexity assembly. Results showed that five of the behaviours identified, such as freezing and the amount of part rotations, significantly differed in occurrence and/or duration between the two conditions. The study hereby proposes a novel and naturalistic method that could help practitioners to map and redesign critical assembly phases, and researchers to enrich validation of MWL-measures through measurement triangulation.

Practitioner summary: Current physiological mental workload (MWL) measures still lack sufficient reliability when applied in the field. Therefore, we identified several observable assembly behaviours that could reveal MWL being too high. The results propose a method to map MWL by observing specific assembly behaviours such as freezing and rotating parts.

Abbreviations: MWL: mental workload; EEG: electroencephalography; fNIRS: functional near infrared spectroscopy; AOI: area of interest; SMI: SensoMotoric Instruments, ETG: Eye-Tracking Glasses; FPS: frames per second; BORIS: Behavioral Observation Research Interactive Software; IRR: inter-rater reliability; SWAT: Subjective Workload Assessment Technique; NASA-TLX: National Aeronautics and Space Administration Task Load Index; EL: emotional load; DSSQ: Dundee Stress State Questionnaire; PHL: physical load; SBO: Strategisch Basis Onderzoek     

Using immersive game-based virtual reality to teach fire-safety skills to children

Virtual reality (VR) has been used both to simulate situations that are too dangerous to practice in real life and as a tool to help children learn. This study was conducted as part of a larger more comprehensive long-term research project which aims to combine the two techniques and demonstrate a novel application of the result, using immersive VR to help children learn about fire hazards and practice escape techniques. In the current study, a CAVE was used to immerse participants in a fire scene. To improve the children’s motivation for learning over prior VR fire-safety training methods, game-like interface interaction techniques were used and students were encouraged to explore the virtual world. Rather than being passive viewers, as in prior related studies, the children were given full control to navigate through the virtual environment and to interact with virtual objects using a game pad and a 6DOF wand. Students identified home fire hazards with a partner and then practiced escaping from a simulated fire in the virtual environment. To test for improved motivation, a user study was completed. Results indicate that students were more engaged by the new game-like learning environment and that they reported that they found the experience fun and intriguing. Their enhanced enthusiasm for what is relatively standard fire-safety information demonstrates the promise of using game-based virtual environments for vital but otherwise tedious fire-safety skills training for children.     

Exploring educational immersive videogames: an empirical study with a 3D multimodal interaction prototype

ABSTRACT

Gestural interaction devices emerged and originated various studies on multimodal human–computer interaction to improve user experience (UX). However, there is a knowledge gap regarding the use of these devices to enhance learning. We present an exploratory study which analysed the UX with a multimodal immersive videogame prototype, based on a Portuguese historical/cultural episode. Evaluation tests took place in high school environments and public videogaming events. Two users would be present simultaneously in the same virtual reality (VR) environment: one as the helmsman aboard Vasco da Gama’s fifteenth-century Portuguese ship and the other as the mythical Adamastor stone giant at the Cape of Good Hope. The helmsman player wore a VR headset to explore the environment, whereas the giant player used body motion to control the giant, and observed results on a screen, with no headset. This allowed a preliminary characterisation of UX, identifying challenges and potential use of these devices in multi-user virtual learning contexts. We also discuss the combined use of such devices, towards future development of similar systems, and its implications on learning improvement through multimodal human–computer interaction.     

A defeasible reasoning framework for human mental workload representation and assessment

Human mental workload (MWL) has gained importance in the last few decades as an important design concept. It is a multifaceted complex construct mainly applied in cognitive sciences and has been defined in many different ways. Although measuring MWL has potential advantages in interaction and interface design, its formalisation as an operational and computational construct has not sufficiently been addressed. This research contributes to the body of knowledge by providing an extensible framework built upon defeasible reasoning, and implemented with argumentation theory (AT), in which MWL can be better defined, measured, analysed, explained and applied in different human–computer interactive contexts. User studies have demonstrated how a particular instance of this framework outperformed state-of-the-art subjective MWL assessment techniques in terms of sensitivity, diagnosticity and validity. This in turn encourages further application of defeasible AT for enhancing the representation of MWL and improving the quality of its assessment.     

基于VR技术的三维射击游戏开发研究

目前三维游戏中利用了虚拟场景,以便捷的人机交互,为游戏者提供最佳的游戏体验。三维游戏的衍生在很大程度上推动了虚拟现实(VR)技术的更新优化,其涵盖了虚拟现实所有技术。本文基于VR技术设计开发三维射击游戏。首先分析游戏开发需求,其次设计三维VR系统结构与游戏开发流程,再次详细规划三维射击游戏功能模块,最后进行游戏人物、场景、道具开发与实现。得出结论,基于VR技术的三维射击游戏,可增强游戏交互性,为游戏者提供身临其境的游戏场景,以此带来全新的游戏刺激与体验。     

Emotion spectrum analysis for a daily repetitive mental workload

In this study, a mental workload of daily mental arithmetic training was objectively evaluated by physiological indices using the emotion spectrum analysis for a group with such training. Physiological measurements of the mental arithmetic task were conducted once a week for each subject. The physiological indices measured were the α-wave power spectrum, the β-wave power spectrum, the θ-wave power spectrum, variations in nasal skin temperature, and the high-frequency component of heart rate variability. Any depressive tendency during the mental arithmetic task was measured as a psychophysical index by using the Profile of Mood States (POMS) (brief form), and the time taken for the mental arithmetic task was also measured as a performance index. Correlations among physiological indices, the psychological index, the performance index, and the emotion spectrum were analyzed in relation to the mental workload.     

The Effect of Head Tracking on the Degree of Presence in Virtual Reality

ABSTRACT

The performance of a virtual reality (VR) system can be assessed from two aspects in the human-VR interaction loop. One aspect is the degree of immersion, which objectively quantifies the performance of the VR system using metrics such as the display field of view or the refresh rate. The other aspect is presence, which measures the user response to the VR system. This article presents a study that compares the impact on presence by changing immersion through enabling and disabling of the head tracking ability on a VR headset. The study quantitatively assesses this change by taking objective measurements of posture and subjective ratings of the VR experience, in terms of presence and motion sickness, after participants have gone through two versions of a roller coaster simulation; one with head tracking on and the other with head tracking off. The results indicate that a loss of immersion, caused by turning the head tracking feature off, results in a significant reduction in postural sway. This loss of immersion also affected presence, as shown through the user surveys. The survey responses indicate that the simulation with head tracking off was less enjoyable and caused more motion sickness compared to when head tracking is kept on.     

Examining virtual reality as a platform for developing mental models of industrial systems

Industrial systems can be complex and not intuitive to perceive. Therefore, students in technology and engineering programs can benefit from developing mental models of industrial systems during their journey in college. However, more often than not, these students do not have access to industrial facilities; thus, developing mental models for systems is a challenge. This paper examines the merit of an Immersive Virtual Reality (IVR) framework application in creating proper mental models for industrial systems in technology and engineering students. Two IVR applications were developed. One IVR application afforded interaction with components of a prefabricated industrial cooling water system (CWVR). In the other application, students designed and built industrial systems with IVR (system designer VR [SDVR]). SDVR facilitated constructive-generative engagement. A group of 33 students was divided into two; one group (the Design, experimental group) was tasked with building a system with SDVR and interacting with the cooling water system in CWVR. The other group was tasked with directly interacting with the CWVR without building a system with SDVR (the Interaction, comparison group). Students' mental models of the cooling water system in CWVR were evaluated following completing the interaction experience with CWVR. The results demonstrate that the causal model notion of the mental model of the cooling water system was significantly higher in the Design, experimental group. The results suggest that designing a rich IVR application that facilitates constructive-generative engagements may carry merit in informing student mental models of complex technical concepts.     

Experiencing immersive virtual reality in museums

Virtual Reality (VR) has been regarded as a highly effective technology that enables people to gain enjoyable and immersive information about museum collections. Drawing from the four realms of the experience economy, we assume absorptive experiences influence immersive experiences, overall museum VR tour experience, and intention to visit a museum. The results show that all the hypotheses are supported. Furthermore, we compared and tested the proposed model and its rival model (postulating the direct influence of the four realms of the experience economy on museum VR experience) and found that the proposed model is better than the rival model.     

Taking the load off: investigations of how adaptive cruise control affects mental workload

It has been posited that Adaptive Cruise Control (ACC) represents a new generation of vehicle automation, in that it has the potential to relieve drivers of mental as well as physical workload. The results of previous research however, have raised some confusing issues about the specific effects of Adaptive Cruise Control (ACC) on driver mental workload (MWL)--some studies report reduced MWL compared to manual driving, while others find no effect. Two hypotheses are proposed in an attempt to explain these discrepancies: (a) that any potential MWL reductions due to ACC could be masked by the overriding influence of steering demand; or (b) that the tasks designed in some experiments do not exploit the adaptive nature of the ACC system, therefore precluding any potential benefits. Two related experiments were designed to test these hypotheses. It was found that the main reason for the discrepant findings was the nature of the driving task chosen--constant-speed tasks do not realise the mental workload benefits of ACC. Future researchers using ACC devices are advised to use variable-speed tasks to ensure that all aspects of device functionality are covered.     

Developing spatial visualization and mental rotation with a digital puzzle game at primary school level

Spatial ability has been recognized as a significant human skill involving the retrieval, retention, and transformation of visual information in a special context. The enhancement of the spatial ability and the effective method of training are rarely considered as an area of study in the education field. This study focuses on the enhancement of spatial visualization and mental rotation, which are two major components of spatial ability, through the practice of puzzle games. This study adopts an experimental approach to test whether the game is effective in facilitating student's development of spatial visualization and mental rotation. 79 primary school students in Taiwan are included to be volunteer participants. Two instruments are adopted to measure the participants' performance of spatial visualization and mental rotation. The research findings show that the designed puzzle games effectively improve the participants' abilities in spatial visualization and mental rotation and that the traditional puzzle games can only enhance participants' mental rotation. This study suggests that the theory-based design of multimedia games can offer a more effective learning environment for developing and improving cognitive skills.     

Learning rate and subjective mental workload in five truck driving tasks

Both learning curve models and subjective mental workload are useful tools for determining the length of training for new workers and predicting future task performance. An experiment was designed to collect the task completion times and subjective mental workload of five driving tasks including (a) reverse into garage, (b) 3-point turn, (c) parallel parking, (d) S-curve and (e) up-down-hill. The results indicated that task completion times of truck driving can be predicted with a learning curve. Practice significantly reduced the mental workload rating. However, the novice trainees tended to have a more significant reduction because, compared to experienced trainees, they tended to give greater or lower workload scores than the experienced trainees before and after practice, respectively. The current study may not be complete enough to provide guidelines for a training programme, but it is adequate to suggest that learning rate and workload measure can serve as indexes for factoring in the individual differences.

Practitioner summary: Learning curves can be used to determine the length of training for new workers and performance standards for a particular task. Learning rate and mental workload were found to be important measures for comparing individual differences in order to better design a training programme. However, mental workload must be evaluated by experienced participants.     

Progression of mental models throughout the phases of a computer-based instructional simulation: Supportive information,practice, and performance

Students of chemical engineering (n = 26) participated in an experiment using a computer-based simulation of a chemical plant. The progression of participants’ mental models was examined throughout a computer-based instructional experience as they acquired the complex cognitive skills of troubleshooting. Participants’ mental models of the complex learning task were matched against an expert mental model at five observation points through the instruction. Progressions of learners’ mental models were examined before and after three phases of the instructional process: supportive information presentation, problem solving practice, and performance test. The results indicated a significant change in participants’ mental models after receiving the supportive information and little change after practice or performance. This paper presents the results of this investigation and discusses the findings and their implications for computer-based instruction and training.     

Virtual Reality Interfaces for Interacting with Three-Dimensional Graphs

Today, virtual reality (VR) systems are widely available through low-cost devices such as Oculus Rift and HTC Vive. Although VR technology has so far been centered on entertainment, there is a growing interest from developers, technology companies, and consumers to evaluate it in a wider variety of contexts. This paper explores the effectiveness of visualizing and interacting with three-dimensional graphs in VR in comparison with the traditional approach. In particular, we present an empirical evaluation study for exploring and interacting with three-dimensional graphs using Oculus Rift and Leap Motion. We designed several interfaces exploiting the natural user interface in a VR environment and compared them with traditional mouse–keyboard and joypad configurations. Our evaluation suggests that, although these upcoming VR technologies are more challenging than more traditional ones, they facilitate user involvement during graph interaction and visualization tasks, given the enjoyable experience elicited when combining gesture-based interfaces and VR.