Creation of a new set of dynamic virtual reality faces for the assessment and training of facial emotion recognition ability

The ability to recognize facial emotions is target behaviour when treating people with social impairment. When assessing this ability, the most widely used facial stimuli are photographs. Although their use has been shown to be valid, photographs are unable to capture the dynamic aspects of human expressions. This limitation can be overcome by creating virtual agents with feasible expressed emotions. The main objective of the present study was to create a new set of dynamic virtual faces with high realism that could be integrated into a virtual reality (VR) cyberintervention to train people with schizophrenia in the full repertoire of social skills. A set of highly realistic virtual faces was created based on the Facial Action Coding System. Facial movement animation was also included so as to mimic the dynamism of human facial expressions. Consecutive healthy participants (n = 98) completed a facial emotion recognition task using both natural faces (photographs) and virtual agents expressing five basic emotions plus a neutral one. Repeated-measures ANOVA revealed no significant difference in participants’ accuracy of recognition between the two presentation conditions. However, anger was better recognized in the VR images, and disgust was better recognized in photographs. Age, the participant’s gender and reaction times were also explored. Implications of the use of virtual agents with realistic human expressions in cyberinterventions are discussed.     

Using immersive game-based virtual reality to teach fire-safety skills to children

Virtual reality (VR) has been used both to simulate situations that are too dangerous to practice in real life and as a tool to help children learn. This study was conducted as part of a larger more comprehensive long-term research project which aims to combine the two techniques and demonstrate a novel application of the result, using immersive VR to help children learn about fire hazards and practice escape techniques. In the current study, a CAVE was used to immerse participants in a fire scene. To improve the children’s motivation for learning over prior VR fire-safety training methods, game-like interface interaction techniques were used and students were encouraged to explore the virtual world. Rather than being passive viewers, as in prior related studies, the children were given full control to navigate through the virtual environment and to interact with virtual objects using a game pad and a 6DOF wand. Students identified home fire hazards with a partner and then practiced escaping from a simulated fire in the virtual environment. To test for improved motivation, a user study was completed. Results indicate that students were more engaged by the new game-like learning environment and that they reported that they found the experience fun and intriguing. Their enhanced enthusiasm for what is relatively standard fire-safety information demonstrates the promise of using game-based virtual environments for vital but otherwise tedious fire-safety skills training for children.     

The impact of an innovative instructional intervention on the acquisition of oral presentation skills in higher education

The present study focuses on the design and development of an instructional approach to develop oral presentation skills. The theoretical base builds on the social cognitive perspective, and self-regulated learning. The aim of the study is to investigate whether the design of a multimedia-based instructional format - comprising of a standardised multimedia instruction, practical activities and feedback - will enhance oral presentation skills. In the study, the differential effect of three ‘modes of feedback’ on performance has been researched. The results reveal that oral presentation skills did improve significantly after the instruction. The multimedia nature of the design was favoured by all participants. In contrast, no significant impact of feedback was found. Nevertheless, feedback proved to be a useful process that was highly welcomed by participants.     

A comparative evaluation of viewing metaphors on psychophysical skills education in an interactive virtual environment

In an empirical evaluation, we examined the effect of viewing condition on psychophysical skills education in an interactive 3D simulation to train users in electrical circuitry. We compared an immersive head-mounted display (HMD)-based viewing metaphor versus a limited, desktop-based virtual reality (DVR) viewing metaphor with interaction using a spatial user interface. Psychophysical skills education involves the association of cognitive functions with motor functions to make the task autonomous with repeated practice. In electrical circuitry, this is demonstrated by the fine movements involved in handling and manipulating components on the electrical circuit, particularly while measuring electrical parameters. We created an interactive circuitry simulation (IBAS) where participants could learn about electrical measurement instruments such as the ammeter, voltmeter and multimeter, in a simulated breadboard VR system. Twenty-four participants utilized the simulation (12 in each condition), and the quantitative and qualitative aspects of psychophysical skills education with respect to the viewing metaphor were examined. Each viewing condition in IBAS was head-tracked and non-stereoscopic. Perspective correction was coupled with head-tracking in the DVR condition. The key quantitative measures were cognitive questionnaires addressing different levels of Bloom’s cognitive taxonomy and a real-world psychophysical task addressing various levels of Dave’s psychomotor taxonomy. The qualitative measures were the Witmer–Singer sense of presence questionnaire and self-report. Results suggest that there was a significant increase in cognition post-experiment in both DVR and HMD viewing conditions in levels of knowledge, application, analysis and evaluation. Results also revealed a significant learning benefit with respect to the higher level concepts pertaining to evaluation in the HMD condition as compared to DVR. Participants seem to have enjoyed a greater level of affordance in task performance and spent a larger amount of time to complete the simulated exercises as well as manually maneuvered to further distances in the HMD viewing condition as compared to DVR viewing.     

Reality-based interaction affecting mental workload in virtual reality mental arithmetic training

ABSTRACT

The concept of digital game-based learning (DGBL) evolves rapidly together with technological enhancements of virtual reality (VR) and smart phones. However, the mental workload (MWL) that VR-training applications demand and motivational qualities originating from user experience (UX) should be identified in order to create effective and enjoyable training/learning challenges that fit with individual users’ capabilities. This study examined the effects of reality-based interaction (RBI) and VR on measures of student motivation and MWL, in a mental arithmetic game for secondary school pupils. In a randomised controlled trial with sixty school children, a mental arithmetic game was tested with three different interaction and two different presentation methods – VR RBI, VR head-mounted-display tapping and tablet flick-gesture. Results found a significant effect of RBI on MWL but no differences in enjoyment of training were found between VR-experience and tablet training-experience. In fact, adding the gaming-context to the mental arithmetic task created an enjoyable, motivating experience regardless of presentation or interaction-style.     

Learning disabilities and visual-motor skills; comparing assessment from a hapto-virtual reality tool and Bender-Gestalt test

Previous investigations conducted on post-secondary adult students with learning disabilities (LD) suggest that deficits in visual-motor skills contribute to difficulties in written expression which impact academic achievement. Intervention strategies for individuals with LD include assistive computer-based technologies (ATs) to compensate for or maximize performance. However, research fails to assess the impact of ATs on performance, learning, and motivation of students with LD. Also, one of the limitations of ATs is that they cannot be used for assessment and training and there are very few methods to assess or train visual-motor skills in this population. The present study explores the usefulness of a hapto-visual virtual reality motor skills assessment (MSA) device for visual-motor functioning in adults with and without LD. This is a preliminary step of developing an intervention to improve impaired visual-motor skills in adults with LD. A sample of 22 male and female university students with and without LD had their visual-motor skills pretested using a standard paper-and-pencil Bender-Gestalt (BG) test and were compared according to their performance on the MSA tool. We hypothesized that our LD participants’ performance would be significantly lower than our control participants on the VR task in terms of number of errors and speed. Results showed that participants without LD performed better and more rapidly on the VR task than participants with LD. There were no correlations between the BG and MSA performance. We did not find significant differences between the groups on the Bender-Gestalt scores, previous experience with video game, arousal, and mood. Our results suggest that a novel 3D virtual reality tool such as the MSA can potentially discriminate motor function of people with and without LD; however, the difference between both may also be due to a lack of problem-solving ability in LD.     

Effect of virtual reality and whole-body heating on motion sickness severity: A combined and individual stressors approach

BackgroundVirtual reality (VR) use is limited by the potential side effects of prolonged exposure to vection, leading to motion sickness. Air temperature (T_a) may exacerbate the severity of such side effects through a synergistic interaction. This study assessed the individual and combined impact of a hot T_a and VR on motion sickness severity.MethodThirteen healthy volunteers were exposed to a 20 min visual stimulus, across four experimental conditions: N_CS: 22 °C T_a with computer screen; N_VR: 22 °C T_a with VR; H_CS: 35 °C T_a with computer screen; H_VR: 35 °C T_a with VR. Motion sickness was assessed via fast motion sickness scale (FMS) and simulator sickness questionnaire (SSQ). Physiological indices of motion sickness including, sweat rate, rectal temperature, cutaneous vascular conductance (CVC), skin temperature, blood pressure and heart rate were also examined.ResultsFMS and SSQ ratings indicate a significant main effect for VR, increasing sickness severity (p < 0.001). A significant main effect of T_a was observed for SSQ, but not FMS ratings (FMS, p = 0.07; SSQ, p < 0.04). Despite trends towards synergism, no interaction (T_a × VR) was observed for FMS (p = 0.2) or SSQ scores (p = 0.07), indicating an additive response. Synergistic trends were also observed for sweat rate and CVC.ConclusionSynergism between VR and heat on motion sickness remains unclear, possibly as a result of considerable inter-individual variation in the reported subjective responses. Understanding of the questions raised by this study inform safe working guidelines for the use of VR in commercial and occupational settings.     

Using auditory event-related EEG potentials to assess presence in virtual reality

The feeling of presence in a virtual reality (VR) is a concept without a standardized objective measurement. In the present study, we used event-related brain potentials (ERP) of the electroencephalogram (EEG) elicited by tones, which are not related to VR, as an objective indicator for the presence experience within a virtual environment. Forty participants navigated through a virtual city and rated their sensation of being in the VR (experience of presence), while hearing frequent standard tones and infrequent deviant tones, which were irrelevant for the VR task. Different ERP components elicited by the tones were compared between participants experiencing a high level of presence and participants with a low feeling of presence in the virtual city. Early ERP components, which are more linked to automatic stimulus processing, showed no correlation with presence experience. In contrast, an increased presence experience was associated with decreased late negative slow wave amplitudes, which are associated with central stimulus processing and allocation of attentional resources. This result supports the assumption that increased presence is associated with a strong allocation of attentional resources to the VR, which leads to a decrease of attentional resources available for processing VR-irrelevant stimuli. Hence, ERP components elicited by the tones are reduced. Particularly, frontal negative slow waves turned out to be accurate predictors for presence experience. Summarizing, late ERPs elicited by VR-irrelevant tones differ as a function of presence experience in VR and provide a valuable method for measuring presence in VR.     

Effect of VR technology matureness on VR sickness

In this paper relationship of perceived virtual reality (VR) sickness phenomenon with different generations of virtual reality head mounted displays (VR HMD) is presented. Action content type omnidirectional video clip was watched by means of four HMDs of different levels of technological matureness, with a 2D monitor used as a reference point. In addition to subjective estimation of VR sickness effects by means of the SSQ questionnaire, psychophysiology of the participants was monitored. Participant’s electrodermal activity, heart rate, skin temperature and respiration rate were measured. Results of the study indicate differences between HMDs in both SSQ score and changes of physiology. Skin conductance was found to be significantly correlated with VR sickness. Mobile HMD did not induce significantly higher levels of VR sickness. Disorientation SSQ was proven to be a useful tool for assessing the VR sickness effects.

     

Self-reported discomfort when using commercially targeted virtual reality equipment in discomfort distraction

Commercially targeted virtual reality (VR) equipment is gaining popularity and might be a viable tool for pain distraction. This experimental research aimed to discover whether active distraction techniques (such as commercially targeted VR and video games) result in reduced subjective discomfort relative to passive distraction techniques. The study examined a healthy adult population who experienced an experimentally induced discomfort task. Participants were 27 adults, 14 females and 13 males. Participants completed four tasks, a baseline measure of physical discomfort, video clip distraction (passive distraction), video game distraction (active distraction) and exploring a VR world using an Oculus Rift head-mounted display (active distraction). In all four test conditions, participants were asked to sit on a chair holding their non-dominant leg at a height of approximately 30 cm from the floor, up to a maximum of 5 min. Counterbalancing of task order was conducted to reduce effects of participant fatigue. The participants indicated significantly reduced self-reported discomfort in the active distraction tasks when compared to the passive distraction tasks. While the findings demonstrate the effectiveness of a commercially targeted VR technology in increasing pain tolerance, the relative benefits of this technology over non-immersive video games are not apparent.     

Perceiving interpersonally-mediated risk in virtual environments

Using virtual reality (VR) to examine risky behavior that is mediated by interpersonal contact, such as agreeing to have sex, drink, or smoke with someone, offers particular promise and challenges. Social contextual stimuli that might trigger impulsive responses can be carefully controlled in virtual environments (VE), and yet manipulations of risk might be implausible to participants if they do not feel sufficiently immersed in the environment. The current study examined whether individuals can display adequate evidence of presence in a VE that involved potential interpersonally-induced risk: meeting a potential dating partner. Results offered some evidence for the potential of VR for the study of such interpersonal risk situations. Participants’ reaction to the scenario and risk-associated responses to the situation suggested that the embodied nature of virtual reality override the reality of the risk’s impossibility, allowing participants to experience adequate situational embedding, or presence.     

Improvement of Japanese adults' English speaking skills via experiences speaking to a robot

This paper reports how robot‐assisted language learning (RALL) impacts Japanese adults' English speaking skills. With existing research on RALL focusing on children, there is little evidence indicating RALL's effects on adults. We developed a RALL system comprising a robot, a tablet, and designed learning materials for speaking practice. To explore the effects of this system on the speaking skills of adults, we conducted a pretest and posttest designed pilot experiment. The participants in this study were nine female Japanese university students. The participants used the system for speaking practice for a total of 30 min per day for 7 days. The results of comparing the pretests with the posttests demonstrated that the adults' English speaking skills were enhanced; in particular, their speaking accuracy, fluency, and pronunciation improved greatly. This study contributes to providing clear evidence of the effects of RALL on Japanese adults' English speaking skills.     

Platform is not destiny: Embodied learning effects comparing 2D desktop to 3D virtual reality STEM experiences

Experiences in immersive 3D virtual reality (VR) are more presence-inducing, and so it may be tempting to claim that content will be learned better in VR. This randomized control trial study on natural selection challenges that assumption. This study answers the question of whether learning STEM in an immersive 3D VR environment is always superior to learning via a 2D monitor (PC). This is a 2 × 2 × 3 design. The first factor is platform immersivity (low = 2D PC, or high = 3D VR), the second factor is level of embodiment (lower = watching playback video, or higher = using mouse/controller to agentically manipulate content), and the third factor is test time (pretest, posttest, and follow-up). There was a significant main effect for embodiment, the high embodied and agentic groups learned the most. There was not a main effect for platform, because the participants in the low embodied VR group performed significantly worse than the three other groups. Although, the one high embodied, VR group learned and retained the most knowledge. A path-analysis revealed that the effect of platform was significantly mediated by presence, agency, and engagement. The smaller learning gain in the low embodied VR condition suggests that participants come to the immersive VR experience with expectations about agency and control of the virtual content, and when those expectations were not met, the disconnect was deleterious for learning. More agentic and interactive control of manipulable virtual content is encouraged. Design is critical, and platform is not destiny.     

Effects of virtual presence and learning outcome using low-end virtual reality systems

Low-cost technology is essential to integrate Virtual Reality (VR) into educative institutions around the world. However, low-cost technology usually refers to low-end technology, which may compromise the level of immersion of the VR system. This study evaluates whether low-end and high-end VR systems achieve a comparable learning outcome regardless their immersion level. We also analyze the relationship between virtual presence and the learning outcome arising from a VR educational experience. An evaluation with 42 participants was conducted. We measured learning outcome and virtual presence under three different configurations, namely: a desktop computer, a low-end VR system, and a high-end VR system. The impact of simulator sickness was also analyzed. Results revealed a lower learning outcome in the less immersive configuration (i.e. desktop) and a similar learning outcome in both low-end and high-end VR systems. Even though low-end VR systems are less immersive and produce a lower level of virtual presence than high-end VR systems, the results support the use of low-end VR systems for educative applications.     

Effects of line length,number of lines,line spacing,and font size on reading performance of Chinese text in virtual reality environment

The increasing importance of virtual reality (VR) necessitates reading in VR environments. Nonetheless, there is a lack of research on the factors influencing reading performance (speed and accuracy) in VR. The purpose of this study is to investigate the influence of line length, number of lines, line spacing, and font size on the response time and accuracy of a Chinese comprehension task in VR. Thirty-two participants were required to read Chinese passages and answer 32 multiple-choice questions in VR. The results indicated that line spacing significantly affected reading speed. Meanwhile, the interaction between the number of lines and line spacing had a significant impact on reading accuracy. To improve reading speed and accuracy, it is advisable to use single-line spacing, 20 characters per line, and ensure that the vertical distance created by the number of lines and line spacing does not exceed the optimal field of view.     

Motivation,engagement, and performance across multiple virtual reality sessions and levels of immersion

This study investigated changes in learners' motivation, engagement, performance, and spatial reasoning over time and across different levels of virtual reality (VR) immersion. Undergraduate participants explored a virtual solar system via a moderately immersive or highly immersive VR platform over three sessions. In a third condition, participants initially learned with moderate immersion and transitioned to higher immersion after the second session. Following research on novelty effects, we explored whether subjective experiences and performance would decline over time (e.g., decreasing motivation or performance) as participants became familiar with the virtual environment and tools. However, we hypothesized that transitional immersion (i.e., switching from moderate to higher immersion) might lead to a renewed sense of novelty. Results suggested that both moderate and higher levels of immersion were motivating, engaging, and supportive of learning. In contrast to predictions based on novelty effects, these outcomes did not decline overall as learners gained familiarity with the systems. However, transitional immersion emerged as a promising and testable pedagogical approach for future VR education. All participants also showed gains in spatial reasoning.     

Effects of postural stability,active control,exposure duration and repeated exposures on HMD induced cybersickness

Cybersickness is common during head-mounted display (HMD) based virtual reality. This study examined whether it is possible to: (1) identify people who are more susceptible to this cybersickness; and (2) find general ways to reduce its occurrence and severity. Our participants were exposed to HMD-based virtual reality four times over two different days (using “Freedom Locomotion VR”). During these 10-min trials, participants were either free-standing or posturally restrained as they actively controlled or passively viewed their locomotion through the virtual environment. Cybersickness was found to increase steadily over time during each exposure. While this cybersickness was markedly reduced on day 2 (compared to day 1), it was not significantly altered by either the use of postural restraints or active locomotion control. However, the sick and well participants in our study were found to differ in terms of their spontaneous postural activity (before they entered virtual reality). We found that the participants who experienced stronger vection also tended to report more severe cybersickness in this study. These findings suggest that we should be able to identify people who are more susceptible to cybersickness and help them become more resistant to it (via repeated exposures to HMD-based virtual reality).     

Exploring the relationship between presence and enjoyment in a virtual museum

The Augmented Representation of Cultural Objects (ARCO) system, developed as a part of an EU ICT project, provides museum curators with software and interface tools to develop web-based virtual museum exhibitions by integrating augmented reality (AR) and 3D computer graphics. ARCO technologies could also be deployed in order to implement educational kiosks placed in real-world museums. The main purpose of the system is to offer an entertaining, informative and enjoyable experience to virtual museum visitors. This paper presents a formal usability study that has been undertaken in order to explore participants’ perceived ‘sense of being there’ and enjoyment while exposed to a virtual museum exhibition in relation to real-world visits. The virtual museum implemented was based on an existing gallery in the Victoria and Albert Museum, London, UK. It is of interest to determine whether a high level of presence results in enhanced enjoyment. After exposure to the system, participants completed standardized presence questionnaires related to the perceived realism of cultural artifacts referred to as AR objects’ presence, as well as to participants’ generic perceived presence in the virtual museum referred to as VR presence. The studies conducted indicate that previous experience with ICTs (Information and Communication Technologies) did not correlate with perceived AR objects’ presence or VR presence while exposed to a virtual heritage environment. Enjoyment and both AR objects’ presence and VR presence were found to be positively correlated. Therefore, a high level of perceived presence could be closely associated with satisfaction and gratification which contribute towards an appealing experience while interacting with a museum simulation system.     

The physiological mirror—a system for unconscious control of a virtual environment through physiological activity

This paper introduces a system for real-time physiological measurement, analysis, and metaphorical visualization within a virtual environment (VE). Our goal is to develop a method that allows humans to unconsciously relate to parts of an environment more strongly than to others, purely induced by their own physiological responses to the virtual reality (VR) displays. In particular, we exploit heart rate, respiration, and galvanic skin response in order to control the behavior of virtual characters in the VE. Such unconscious processes may become a useful tool for storytelling or assist guiding participants through a sequence of tasks in order to make the application more interesting, e.g., in rehabilitation. We claim that anchoring of subjective bodily states to a virtual reality (VR) can enhance a person’s sense of realism of the VR and ultimately create a stronger relationship between humans and the VR.     

Human–Computer Interface Issues in Controlling Virtual Reality With Brain–Computer Interface

We have integrated the Graz brain–computer interface (BCI) system with a highly immersive virtual reality (VR) Cave-like system. This setting allows for a new type of experience, whereby participants can control a virtual world using imagery of movement. However, current BCI systems still have many limitations. In this article we present two experiments exploring the different constraints posed by current BCI systems when used in VR. In the first experiment we let the participants make free choices during the experience and compare their BCI performance with participants using BCI without free choice; this is unlike most previous work in this area, in which participants are requested to obey cues. In the second experiment we allowed participants to control a virtual body with motor imagery. We provide both quantitative and subjective results, regarding both BCI accuracy and the nature of the subjective experience in this new type of setting.