Reality-based interaction affecting mental workload in virtual reality mental arithmetic training

ABSTRACT

The concept of digital game-based learning (DGBL) evolves rapidly together with technological enhancements of virtual reality (VR) and smart phones. However, the mental workload (MWL) that VR-training applications demand and motivational qualities originating from user experience (UX) should be identified in order to create effective and enjoyable training/learning challenges that fit with individual users’ capabilities. This study examined the effects of reality-based interaction (RBI) and VR on measures of student motivation and MWL, in a mental arithmetic game for secondary school pupils. In a randomised controlled trial with sixty school children, a mental arithmetic game was tested with three different interaction and two different presentation methods – VR RBI, VR head-mounted-display tapping and tablet flick-gesture. Results found a significant effect of RBI on MWL but no differences in enjoyment of training were found between VR-experience and tablet training-experience. In fact, adding the gaming-context to the mental arithmetic task created an enjoyable, motivating experience regardless of presentation or interaction-style.     

A Web-based cost-effective training tool with possible application to brain injury rehabilitation

Virtual reality (VR) has provoked enormous interest in the medical community. In particular, VR offers therapists new approaches for improving rehabilitation effects. However, most of these VR assistant tools are not very portable, extensible or economical. Due to the vast amount of 3D data, they are not suitable for Internet transfer. Furthermore, in order to run these VR systems smoothly, special hardware devices are needed. As a result, existing VR assistant tools tend to be available in hospitals but not in patients' homes. To overcome these disadvantages, as a case study, this paper proposes a Web-based Virtual Ticket Machine, called WBVTM, using VRML [VRML Consortium, The Virtual Reality Modeling Language: International Standard ISO/IEC DIS 14772-1, 1997, available at ], Java and EAI (External Authoring Interface) [Silicon Graphics, Inc., The External Authoring Interface (EAI), available at ], to help people with acquired brain injury (ABI) to relearn basic living skills at home at a low cost. As these technologies are open standard and feature usability on the Internet, WBVTM achieves the goals of portability, easy accessibility and cost-effectiveness.     

Evaluation and prediction mental workload in user interface of maritime operations using eye response

Eye response measurement is one of the objective measure methods and useful for assessing of operators' mental workload (MWL). The main objectives of this paper are to consider the relationship between operators' MWL and eye responses in the task of operating marine engine interface. Also, an artificial neural network (ANN) model was developed to predict the operators' MWL based on integrating eye response data. Eye response indices (pupil dilation, blink rate, fixation rate, and saccadic rate) were recorded, and two subjective rating methods (The National Aeronautics and Space Administration's Task Load Index [NASA-TLX] and subjective workload assessment technique [SWAT]) were used for 27 participants. The results again confirm that the eye response is sensitive to MWL in workload levels of the task when using the interface control. The ANN model developed by measuring these indices can predict the operators' MWL with the determination coefficient (R²) of 0.971, 0.912 and 0.918 for training, validation, and testing, respectively. These results indicated that the ANN approach is quite accurate for the prediction of operators' MWL based on eye response indices.Relevance to industryThe developed model is expected to provide the operator with a reference value of their MWL by evaluating their physiological indices. This result might be applied for developing an intelligent prediction model in the actual work environment to inform or support the operator in a variety of ways. From this, the manager can organize the human resources for each task to sustain the appropriate MWL as well as to improve the work performance.     

Using virtual reality technology for aircraft visual inspection training: presence and comparison studies

The aircraft maintenance industry is a complex system consisting of several interrelated human and machine components. Recognizing this, the Federal Aviation Administration (FAA) has pursued human factors related research. In the maintenance arena the research has focused on the aircraft inspection process and the aircraft inspector. Training has been identified as the primary intervention strategy to improve the quality and reliability of aircraft inspection. If training is to be successful, it is critical that we provide aircraft inspectors with appropriate training tools and environment. In response to this need, the paper outlines the development of a virtual reality (VR) system for aircraft inspection training.

VR has generated much excitement but little formal proof that it is useful. However, since VR interfaces are difficult and expensive to build, the computer graphics community needs to be able to predict which applications will benefit from VR. To address this important issue, this research measured the degree of immersion and presence felt by subjects in a virtual environment simulator. Specifically, it conducted two controlled studies using the VR system developed for visual inspection task of an aft-cargo bay at the VR Lab of Clemson University. Beyond assembling the visual inspection virtual environment, a significant goal of this project was to explore subjective presence as it affects task performance. The results of this study indicated that the system scored high on the issues related to the degree of presence felt by the subjects. As a next logical step, this study, then, compared VR to an existing PC-based aircraft inspection simulator. The results showed that the VR system was better and preferred over the PC-based training tool.     

Application of virtual reality (VR) technology for medical practitioners in type and screen (T&S) training

Nowadays, patients' safety is the top priority for medical services around the world. However, it is believed that many of the adverse events in hospitals are preventable. Type and screen (T&S) procedures require intense practical training by each medical practitioner in each hospital. This study applied an interactive Virtual Reality (VR) technology to supplement the traditional approach to facilitate procedural training. The VR system made use of the Unity3D for application development. To investigate the reliability and validity of the conceptual medical training model, a survey was conducted to measure the content, motivation and enhanced readiness of practitioners. The partial least squares (PLS) modelling was carried out to investigate the correlation between each pair of measured variables. The study results indicated that the learning model has good reliability for each measurement factor and validates the survey study. The PLS modelling also indicated a significant correlation between each pair of measured variables. The project developed a VR training program for training in T&S procedures. The study provides important implications on the development of a practical VR training program for medical practitioners, as well as valuable insights for the development of similar VR training programs in the future.     

基于三维视觉指导的运动想象训练性能分析

为提高视觉指导下运动想象（MI）的训练效率和脑机接口（BCI）的分类准确率,研究了虚拟现实（VR）环境对MI训练的影响以及不同视觉指导下脑电（EEG）分类模型的差异。首先,设计了三种三维手部交互动画及其EEG采集程序;然后,分别在头戴式头盔（HMD）和平面液晶屏（LCD）的呈现环境下,对5名健康被试进行了标准（单次实验5min）和长测（单次实验15min）两种实验方案的左右手MI训练;最后,通过对EEG数据的模式分类,分析了呈现环境和内容形式对分类准确率的影响。实验结果表明,在视觉指导的MI训练中,HMD与LCD的呈现方式存在显著性差异。HMD所呈现的VR环境能够提高MI分类准确率,延长单次训练时长;此外,不同视觉指导内容下的分类模型存在较大差别,当测试样本与训练样本为同一视觉指导内容时,其平均分类准确率较之不同情况高出16.34%。     

The stress and workload of virtual reality training: the effects of presence,immersion and flow

The present investigation evaluated the effects of virtual reality (VR) training on the performance, perceived workload and stress response to a live training exercise in a sample of Soldiers. We also examined the relationship between the perceptions of that same VR as measured by engagement, immersion, presence, flow, perceived utility and ease of use with the performance, workload and stress reported on the live training task. To a degree, these latter relationships were moderated by task performance, as measured by binary (Go/No-Go) ratings. Participants who reported positive VR experiences also tended to experience lower stress and lower workload when performing the live version of the task. Thus, VR training regimens may be efficacious for mitigating the stress and workload associated with criterion tasks, thereby reducing the ultimate likelihood of real-world performance failure.

Practitioner Summary: VR provides opportunities for training in artificial worlds comprised of highly realistic features. Our virtual room clearing scenario facilitated the integration of Training and Readiness objectives and satisfied training doctrine obligations in a compelling engaging experience for both novice and experienced trainees.     

On Weight Design of Maximum Weighted Likelihood and an Extended EM Algorithm

The recent Maximum Weighted Likelihood (MWL) [18], [19] has provided a general learning paradigm for density-mixture model selection and learning, in which weight design, however, is a key issue. This paper will therefore explore such a design, and through which a heuristic extended Expectation-Maximization (X-EM) algorithm is presented accordingly. Unlike the EM algorithm [1], the X-EM algorithm is able to perform model selection by fading the redundant components out from a density mixture, meanwhile estimating the model parameters appropriately. The numerical simulations demonstrate the efficacy of our algorithm.     

Empirical evidence,evaluation criteria and challenges for the effectiveness of virtual and mixed reality tools for training operators of car service maintenance

The debate on effectiveness of virtual and mixed reality (VR/MR) tools for training professionals and operators is long-running with prominent contributions arguing that there are several shortfalls of experimental approaches and assessment criteria reported within the literature. In the automotive context, although car-makers were pioneers in the use of VR/MR tools for supporting designers, researchers started only recently to explore the effectiveness of VR/MR systems as mean for driving external operators of service centres to acquire the procedural skills necessary for car maintenance processes. In fact, from 463 journal articles on VR/MR tools for training published in the last thirty years, we identified only eight articles in which researchers experimentally tested the effectiveness of VR/MR tools for training service operators’ skills. To survey the current findings and the deficiencies of these eight studies, we use two main drivers: (i) a well-known framework of organizational training programmes, and (ii) a list of eleven evaluation criteria widely applied by researchers of different fields for assessing the effectiveness of training carried out with VR/MR systems. The analysis that we present allows us to: (i) identify a trend among automotive researchers of focusing their analysis only on car service operators’ performance in terms of time and errors, by leaving unexplored important pre- and post-training aspects that could affect the effectiveness of VR/MR tools to deliver training contents – e.g., people skills, previous experience, cibersickness, presence and engagement, usability and satisfaction and (ii) outline the future challenges for designing and assessing VR/MR tools for training car service operators.     

Virtual reality application influences cognitive load-mediated creativity components and creative performance in engineering design

Virtual reality (VR) can promote design performance, and may generate a high cognitive load and affect creative design thinking as well. In order to examine the effect of VR application on cognitive load and engineering design creativity, this study recruited 81 eighth-grade students as participants and employed a non-equivalent-groups quasi-experimental design to examine how VR application influences creativity components and creative performance. A creativity component scale (creative motivation, creative thinking, and professional cognition), a measurement of creative performance, and a cognitive load scale were used to collect data. The study also investigated the ability of VR application to predict creativity components and creative performance, using cognitive load as the mediator. The conclusions are as follows: (1) VR application had marked effects on professional cognition and creative motivation but no effect on creative thinking skills. (2) VR application had marked effects on the novelty and usefulness aspects of creative performance, particularly with respect to usefulness. (3) VR application had marked effects on extraneous and germane cognitive load, but no influence on intrinsic cognitive load, and (4) Both VR application and cognitive load were able to predict creativity components and creative performance. Their abilities to predict creative performance approached an acceptable level. Nevertheless, VR application had no effect on general creative thinking skills; therefore, other teaching strategies are needed to strengthen creative thinking.     

Code vector density in topographic mappings: Scalar case

The author derives some new results that build on his earlier work (1989) of combining vector quantization (VQ) theory and topographic mapping (TM) theory. A VQ model (with a noisy transmission medium) is used to model the processes that occur in TMs, which leads to the standard TM training algorithm, albeit with a slight modification to the encoding process. To emphasize this difference, the model is called a topographic vector quantizer (TVQ). In the continuum limit of the one-dimensional (scalar) TVQ. It is found that the density of code vectors is proportional to P(x)(a) (alpha=1/3) assuming that the transmission medium introduces additive noise with a zero-mean, symmetric, monotically decreasing probability density. This result is dramatically different from the result that is predicted when the standard TM training algorithm is used with a uniform symmetric neighborhood [-n, +n], and it is noted that this difference arises entirely from using minimum distortion rather than nearest neighbor encoding.     

The efficacy of a medical virtual reality simulator for training phlebotomy

OBJECTIVE: The present study compared the effectiveness of a virtual reality (VR) simulator for training phlebotomy with that of a more traditional approach using simulated limbs. BACKGROUND: Phlebotomy, or drawing blood, is one of the most common medical procedures; yet, there are no universal standards for training and assessing performance. The absence of any standards can lead to injuries and inaccurate test results if the procedure is improperly performed. METHOD: Twenty 3rd-year medical students were trained under one of the two methods and had their performance assessed with a 28-item checklist. RESULTS: The results showed that performance improvements were limited to those who trained with the simulated limbs, and a detailed comparison of the two systems revealed several functional and physical differences that may explain these findings. CONCLUSION: Participants trained with simulated limbs performed better than those trained with a VR simulator; however, the metrics recorded by the VR system may address some aspects of performance that could eventually prove beneficial. APPLICATION: The present study highlights the potential for medical simulators to improve patient safety by enabling trainees to practice procedures on devices instead of patients. Applications of this research include training, performance assessment, and design of simulator systems.     

Motion retargeting and evaluation for VR-based training of free motions

Virtual reality (VR) has emerged as one of the important and effective tools for education and training. Most VR-based training systems are situation based, where the trainees are trained for discrete decision making in special situations presented by the VR environments. In contrast, this paper discusses the application of VR to a different class of training, for learning free motion, often required in sports and the arts. We propose a VR-based motion-training framework that contains an intuitive motion-guiding interface, posture-oriented motion retargeting, and an evaluation and advice scheme for corrective feedback. Applications of the proposed framework to simple fencing training and a dance imitation game are demonstrated.     

Comparison of NASA-TLX scale,modified Cooper–Harper scale and mean inter-beat interval as measures of pilot mental workload during simulated flight tasks

Abstract

The sensitivity of NASA-TLX scale, modified Cooper–Harper (MCH) scale and the mean inter-beat interval (IBI) of successive heart beats, as measures of pilot mental workload (MWL), were evaluated in a flight training device (FTD). Operational F/A-18C pilots flew instrument approaches with varying task loads. Pilots’ performance, subjective MWL ratings and IBI were measured. Based on the pilots’ performance, three performance categories were formed; high-, medium- and low-performance. Values of the subjective rating scales and IBI were compared between categories. It was found that all measures were able to differentiate most task conditions and there was a strong, positive correlation between NASA-TLX and MCH scale. An explicit link between IBI, NASA-TLX, MCH and performance was demonstrated. While NASA-TLX, MCH and IBI have all been previously used to measure MWL, this study is the first one to investigate their association in a modern FTD, using a realistic flying mission and operational pilots.

Practitioner summary: NASA-TLX scale, MCH scale and the IBI were evaluated in a flight training device. All measures were able to differentiate most task conditions and there was a positive correlation between NASA-TLX and MCH scale. An explicit link between IBI, NASA-TLX, MCH and performance was demonstrated.

Abbreviations: ANOVA: Analysis of Variance; ECG: Electrocardiograph; F/A: fighter/attack; ft: feet; FTD: flight training device; G: Gravity; km: kilometer; m: meter; m/s: meters per second; MWL: mental workload; MCH: modified cooper-harper; NASA-TLX: NASA Task Load Index; NM: Nautical Mile; NN: normal-to-normal; IBI: inter-beat interval; ILS: Instrument Landing System; RR: R-Wave to R-Wave; SD: standard deviation; TTP: tactics, techniques and procedures; WTSAT: Weapon Tactics and Situation Awareness Trainer     

Motivational and cognitive benefits of training in immersive virtual reality based on multiple assessments

The main objective of this study was to examine the effectiveness of immersive virtual reality (VR) as a medium for delivering laboratory safety training. We specifically compare an immersive VR simulation, a desktop VR simulation, and a conventional safety manual. The sample included 105 first year undergraduate engineering students (56 females).  We include five types of learning outcomes including post‐test enjoyment ratings; pre‐ to post‐test changes in intrinsic motivation and self‐efficacy; a post‐test multiple choice retention test; and two behavioral transfer tests. Results indicated that the groups did not differ on the immediate retention test, suggesting that all three media were equivalent in conveying the basic knowledge. However, significant differences were observed favoring the immersive VR group compared to the text group on the two transfer tests involving the solving problems in a physical lab setting (d = 0.54, d = 0.57), as well as enjoyment (d = 1.44) and increases in intrinsic motivation (d = 0.69) and self‐efficacy (d = 0.60). The desktop VR group scored significantly higher than the text group on one transfer test (d = 0.63) but not the other (d= 0.11), as well as enjoyment (d =1.11) and intrinsic motivation (d =0.83).     

Effectiveness of VR-based training on improving construction workers’ knowledge,skills, and safety behavior in robotic teleoperation

The emergence of construction robotics and automation has produced an urgent and vast need for construction workers to reskill and upskill for the future of work. Virtual Reality (VR)-based training has been considered and investigated as a safe and cost-effective training method that allows workers to be exposed to hazardous tasks with negligible actual safety risks in comparison to existing training methods (hands-on, lecture-based, apprenticeship training). This paper aims to investigate the impact of VR-based training on construction workers’ knowledge acquisition, operational skills, and safety behavior during robotic teleoperation compared to the traditional in-person training method. Fifty construction workers were randomly assigned to complete either VR-based or in-person training for operating a demolition robot. We used quantitative and qualitative data analyses to answer our research questions. Our results indicate that VR-based training was associated with a significant increase in knowledge, operational skills, and safety behavior compared to in-person training. Our findings suggest that VR-based training not only provides a viable and effective option for future training programs but a valuable option for construction robotics safety and skill training.     

A desktop VR prototype for industrial training applications

The recent advances in computer graphics has spurred interest from both academics and industries in virtual reality (VR) enabled training applications. This paper presents a desktop VR prototype for industrial training applications. It is designed and implemented as a general shell by providing the data interface to import both the virtual environment models and specific domain knowledge. The geometric models of the virtual environment are constructed using feature-based modelling and assembly function by external CAD tools, and then transferred into the prototype through a conversion module. A hierarchical structure is proposed to partition and organise these imported virtual environment models. Based on this structure, a visibility culling approach is developed for fast rendering and user interaction. The case study has demonstrated the functionality of the proposed prototype system by applying it to a maintenance training application for a refinery bump system, which, in general, has a large number of polygons and a certain depth complexity. Significant speedup in both context rendering and response to user manipulations has been achieved to provide the user with a fast system response within the desktop virtual environment. Compared with the immersive VR system, the proposed system has offered an affordable and portable training media for industrial applications.The work was done in Nanyang Technological University.     

Process and Outcome-based Evaluation between Virtual Reality-driven and Traditional Construction Safety Training

The emerging digital technologies such as virtual reality (VR) provide an alternative platform for construction safety training. In order to explore how digital-driven technologies affect the effectiveness of safety training, there is a need to empirically test the differences in performance between digital 3D/VR safety training and traditional 2D/paper approach. This research conducted a performance evaluation that emphasises both the training process and learning outcomes of trainees based on researchers’ self-developed immersive construction safety training platform. Data related to physiological indicators such as skin resistance were collected to measure safety performance before and after the training. The detailed measurement indicators included nine categories (e.g., immersion, inspiration) to form a holistic list of evaluation dimensions. The findings revealed that VR-driven immersive safety training outperformed the traditional way for trainees in terms of both process and outcome-based indicators. Results confirmed that safety training was no longer constrained by understanding or memorizing 2D information (texts and images). Instead, trainees experienced a stronger sense of embodied cognition through the immersive experience and multi-sensory engagement by interacting with the VR-driven system. By engaging the theory of embodied cognition, this research provides both the empirical evidence and in-depth analysis of how immersive virtual safety training outperforms traditional training in terms of both training process and outcomes.