Seamless interaction in virtual reality

Jot is a novel research interface for virtual reality modeling. This system seamlessly integrates and applies a variety of virtual and physical tools, each customized for specific tasks. The Jot interface not only moves smoothly from one tool to another but also physically and cognitively matches individual tools to the tasks they perform. In particular, we exploit the notion that gestural interaction is more direct, in many cases, than traditional widget based interaction. We also respect the time tested observation that some operations-even conceptually three dimensional ones-are better performed with 1D or 2D input devices, whereas other operations are more naturally performed using stereoscopic views, higher DOF input devices, or both. Ultimately we strive for a 3D modeling system with an interface as transparent as the interaction afforded by a pencil and a sheet of paper. For example, the system should facilitate the tasks of drawing and erasing and provide an easy transition between the two. Jot emerged from our previous work on a mouse based system, called Sketch, for gesturally creating imprecise 3D models. Jot extends Sketch's functionality to a wider spectrum of modeling, from concept design to detailed feature based parametric parts. Jot also extends the interaction in Sketch to better support individual modeling tasks. We extended Sketch's gestural framework to integrate interface components ranging from traditional desktop interface widgets to context sensitive gestures to direct manipulation techniques originally designed for immersive VR     

This is NOT a game [virtual reality]

Training exercises with a large degree of realism have always been the goal of the armed forces. To a large extent, the US military has succeeded in creating the training simulations needed to turn out the world's most successful fighting forces, but at a staggering cost. The paper discusses the use of simulation and virtual reality in military training.     

A PC-based distributed multiple display virtual reality system

In this paper, a PC-based distributed multiple-display virtual reality system is constructed. The system is a PC-based CAVE environment. Five PCs are used in this system; one is the master responsible for receiving user's input signals, the others are clients for images rendering. Devices are added to synchronize the RGB signals from client PCs.

A software library is developed to drive the system properly. The software is designed in object-oriented language and flexibility for different interface devices is considered.

With the proposed system, developers can easily elaborate PC-based virtual reality applications and build PC virtual worlds at much lower cost.     

A framework for fidelity evaluation of immersive virtual reality systems

Virtual Reality - Developments in visual and tracking systems have expanded virtual reality (VR) applications and led to VR becoming a powerful tool for decision making, planning, and conducting...     

An objective measure for the visual fidelity of virtual reality and the risks of falls in a virtual environment

Despite decades of development of virtual reality (VR) devices and VR’s recent renaissance, it has been difficult to measure these devices’ effectiveness in immersing the observer. Previously, VR devices have been evaluated using subjective measures of presence, but in this paper, we suggest that postural stability can be used to objectively assess visual fidelity of VR headsets. We validated this measure by testing known differences between the devices. This study also aimed to determine the stability of healthy participants, while in a stable virtual world, compared to eyes-open and eyes-closed conditions and therefore provide a standard of safety requirements for future experimentation. Participants’ ability to maintain a stable centre of pressure was measured using a Wii Balance Board, covered by a foam pad. Stability in eyes-open and eyes-closed conditions was compared with: (1) an iPod Touch in a simple Google cardboard style headset, (2) the Oculus Rift Development Kits (DK) DK1, DK2, with and without the tracking of linear head movements, and (3) the Samsung Gear VR. With a stable VR visual stimulus, the eyes-open condition allowed for significantly greater postural stability than the other conditions, which supports the validity of posturography as a measure of visual fidelity. Further, the iPod Touch, with its narrow field of view and rudimentary software, was significantly less effective at destabilising participants with visual perturbations than the other headsets, with their wider field of view and time warping. Unexpected results are discussed with respect to the possible limitations of the experimental design.     

Evaluating the usability of virtual reality user interfaces

A walkthrough method for evaluating virtual reality (VR) user interfaces is described and illustrated with a usability assessment of a virtual business park application. The method is based on a theory of interaction that extends Norman's model of action. A walkthrough analysis method uses three models derived from the theory. The first model describes goal-oriented task action, the second exploration and navigation in virtual worlds, while the third covers interaction in response to system initiative. Each stage of the model is associated with generic design properties that specify the necessary support from the system for successful interaction. The evaluation method consists of a checklist of questions using the properties and following the model cycle. Use of the method uncovered several usability problems. Approaches to evaluation of VR applications and future work are discussed.     

Evaluating the usability of virtual reality user interfaces

A walkthrough method for evaluating virtual reality (VR) user interfaces is described and illustrated with a usability assessment of a virtual business park application. The method is based on a theory of interaction that extends Norman's model of action. A walkthrough analysis method uses three models derived from the theory. The first model describes goal-oriented task action, the second exploration and navigation in virtual worlds, while the third covers interaction in response to system initiative. Each stage of the model is associated with generic design properties that specify the necessary support from the system for successful interaction. The evaluation method consists of a checklist of questions using the properties and following the model cycle. Use of the method uncovered several usability problems. Approaches to evaluation of VR applications and future work are discussed.     

Personalised e-learning through an educational virtual reality game using Web services

This paper describes how a personalised educational game architecture has been used in conjunction with Web services to provide remote access to the system. The educational game is a virtual reality adventure game that performs affective user modelling by measuring emotional characteristics of users. Virtual reality (VR) games are so popular among children and adolescents that can be used for the purposes of educational software to render it more attractive and motivating. The benefits of such an application can be maximised if it is available over the Web. Software applications that operate over the Web are targeted to a wide range of users. Hence they need a high degree of adaptivity and dynamic individualisation to each user that interacts with the application. This should include the students’ emotional state that affects their learning. However, the environment of a Web-based VR-game that performs user modelling is so demanding that the technology of Web services is necessary for its effective operation and interoperability. Moreover, reusability may be achieved for the user modelling component.

A comparative study using an autostereoscopic display with augmented and virtual reality

Advances in display devices are facilitating the integration of stereoscopic visualisation in our daily lives. However, autostereoscopic visualisation has not been extensively exploited. In this paper, we present a system that combines augmented reality (AR) and autostereoscopic visualisation. We also present the first study that compares different aspects using an autostereoscopic display with AR and virtual reality (VR), in which 39 children from 8 to 10 years old participated. In our study, no statistically significant differences were found between AR and VR. However, the scores were very high in nearly all of the questions, and the children also scored the AR version higher in all cases. Moreover, the children explicitly preferred the AR version (81%). For the AR version, a strong and significant correlation was found between the use of the autostereoscopic screen in games and seeing the virtual object on the marker. For the VR version, two strong and significant correlations were found. The first correlation was between the ease of play and the use of the rotatory controller. The second correlation was between depth perception and the game global score. Therefore, the combinations of AR and VR with autostereoscopic visualisation are possibilities for developing edutainment systems for children.     

Head-mounted display versus desktop for 3D navigation in virtual reality: a user study

Virtual Reality (VR) has been constantly evolving since its early days, and is now a fundamental technology in different application areas. User evaluation is a crucial step in the design and development of VR systems that do respond to users’ needs, as well as for identifying applications that indeed gain from the use of such technology. Yet, there is not much work reported concerning usability evaluation and validation of VR systems, when compared with the traditional desktop setup. The paper presents a user study performed, as a first step, for the evaluation of a low-cost VR system using a Head-Mounted Display (HMD). That system was compared to a traditional desktop setup through an experiment that assessed user performance, when carrying out navigation tasks in a game scenario for a short period. The results show that, although users were generally satisfied with the VR system, and found the HMD interaction intuitive and natural, most performed better with the desktop setup.

Personal Varrier: Autostereoscopic virtual reality display for distributed scientific visualization

As scientific data sets increase in size, dimensionality, and complexity, new high resolution, interactive, collaborative networked display systems are required to view them in real-time. Increasingly, the principles of virtual reality (VR) are being applied to modern scientific visualization. One of the tenets of VR is stereoscopic (stereo or 3d) display; however the need to wear stereo glasses or other gear to experience the virtual world is encumbering and hinders other positive aspects of VR such as collaboration. Autostereoscopic (autostereo) displays presented imagery in 3d without the need to wear glasses or other gear, but few qualify as VR displays. The Electronic Visualization Laboratory (EVL) at the University of Illinois at Chicago (UIC) has designed and built a single-screen version of its 35-panel tiled Varrier display, called Personal Varrier. Based on a static parallax barrier and the Varrier computational method, Personal Varrier provides a quality 3d autostereo experience in an economical, compact form factor. The system debuted at iGrid 2005 in San Diego, CA, accompanied by a suite of distributed and local scientific visualization and 3d teleconferencing applications. The CAVEwave National LambdaRail (NLR) network was vital to the success of the stereo teleconferencing.     

Hand-adaptive user interface: improved gestural interaction in virtual reality

Most interactive user interfaces (UIs) for virtual reality (VR) applications are based on the traditional eye-centred UI design principle, which primarily considers the user’s visual searching efficiency and comfort, but the hand operation performance and ergonomics are relatively less considered. As a result, the hand interaction in VR is often criticized as being less efficient and precise. In this paper, the user’s arm movement features, such as the choice of the hand being used and hand interaction position, are hypothesized to influence the interaction results derived from a VR study. To verify this, we conducted a free hand target selection experiment with 24 participants. The results showed that (a) the hand choice had a significant effect on the target selection results: for a left hand interaction, the targets located in spaces to the left were selected more efficiently and accurately than those in spaces to the right; however, in a right hand interaction, the result was reversed, and (b) the free hand interactions at lower positions were more efficient and accurate than those at higher positions. Based on the above findings, this paper proposes a hand-adaptive UI technique to improve free hand interaction performance in VR. A comprehensive comparison between the hand-adaptive UI and traditional eye-centred UI was also conducted. It was shown that the hand-adaptive UI resulted in a higher interaction efficiency and a lower physical exertion and perceived task difficulty than the traditional UI.

     

Developing speech input for virtual reality applications: A reality based interaction approach

An input device should be natural and convenient for a user to transmit information to a computer, and should be designed from an understanding of the task to be performed and the interrelationship between the task and the device from the perspective of the user. In order to investigate the potential of speech input as a reality based interaction device, this paper presents the findings of a study that investigated speech input in a VR application. Two independent user trials were combined within the same experimental design to evaluate the commands that users employed when they used free speech in which they were not restricted to a specific vocabulary. The study also investigated when participants were told they were either talking to a machine (e.g. a speech recognition system) or instructing another person to complete a VR based task. Previous research has illustrated that when users are limited to a specific vocabulary, this can alter the interaction style employed. The findings from this research illustrate that the interaction style users employ are very different when they are told they are talking to a machine or another person. Using this knowledge, recommendations can be drawn for the development of speech input vocabularies for future VR applications.     

The effect of visual and interaction fidelity on spatial cognition in immersive virtual environments

Accuracy of memory performance per se is an imperfect reflection of the cognitive activity (awareness states) that underlies performance in memory tasks. The aim of this research is to investigate the effect of varied visual and interaction fidelity of immersive virtual environments on memory awareness states. A between groups experiment was carried out to explore the effect of rendering quality on location-based recognition memory for objects and associated states of awareness. The experimental space, consisting of two interconnected rooms, was rendered either flat-shaded or using radiosity rendering. The computer graphics simulations were displayed on a stereo head-tracked head mounted display. Participants completed a recognition memory task after exposure to the experimental space and reported one of four states of awareness following object recognition. These reflected the level of visual mental imagery involved during retrieval, the familiarity of the recollection, and also included guesses. Experimental results revealed variations in the distribution of participants' awareness states across conditions while memory performance failed to reveal any. Interestingly, results revealed a higher proportion of recollections associated with mental imagery in the flat-shaded condition. These findings comply with similar effects revealed in two earlier studies summarized here, which demonstrated that the less "naturalistic" interaction interface or interface of low interaction fidelity provoked a higher proportion of recognitions based on visual mental images.     

Comparison of visual fatigue caused by head-mounted display for virtual reality and two-dimensional display using objective and subjective evaluation

This study aimed to evaluate objective and subjective visual fatigue experienced before and after performing a visual task while using a head-mounted display for virtual reality (VR-HMD) and two-dimensional (2D) display. Binocular fusion maintenance (BFM) was measured using a binocular open-view Shack–Hartmann wavefront aberrometer equipped with liquid crystal shutters. Twelve healthy subjects performed the BFM test and completed a questionnaire regarding subjective symptoms before and after performing a visual task that induces low visually induced motion sickness (VIMS). BFM (p = .87) and total subjective eye symptom scores (p = .38) were not significantly different between both groups, although these values were significantly lower after the visual task than before the task within both groups (p < .05). These findings suggest that visual fatigue after using a VR-HMD is not significantly different from that after using a 2D display in the presence of low-VIMS VR content.

Practitioner summary: Objective and subjective evaluation of visual fatigue were not significantly different with the use of a head-mounted display for virtual reality (VR-HMD) and two-dimensional display. These results should be valuable not only to engineers developing VR content but also to researchers involved in the evaluation of visual fatigue using VR-HMD.

Abbreviations: VR: virtual reality; VR-HMD: head-mounted display for virtual reality; BFM: binocular fusion maintenance; BWFA: binocular open-view Shack–Hartmann wavefront aberrometer     

Investigation of spatial ability test completion times in virtual reality using a desktop display and the Gear VR

Virtual Reality - The interaction time of students who did spatial ability tests in a virtual reality environment is analyzed. The spatial ability test completion times of 240 and 61 students were...     

Solid modelling in a virtual reality environment

This paper presents a constraint-based methodology for intuitive and precise solid modelling in a virtual reality (VR) environment. A hierarchically structured and constraint-based data model is developed to support solid modelling in the VR environment. A constraint reasoning engine is also developed to automatically deduce allowable motions for precise constraint-based 3D manipulations. A prototype system of product modelling has been successfully developed, and experimental results demonstrate the advantage of precise solid modelling through constraint-based manipulation in virtual environments.     

Update rates and fidelity in virtual environments

Interaction is the primary characteristic of a Virtual Environment and update rate is normally taken as an index or measure of the interactivity of the system. The speed of many systems is dictated by the slowest component which is often the Computer Image Generator (CIG). It is common for the workload of the CIG to vary and hence the performance of the system. This paper shows how a variable update rate can produce undesirable results. Two solutions to this problem are presented: service degradation and worst-case. In the case of the CIG, service degradation would require the quality of the image to be reduced such that the time taken never exceeds a given deadline. The worst-case technique works by finding the longest time taken to render any view and then uses that as the deadline for completion. The support of predictive methods is one of several benefits of this approach. An implementation of the worst-case technique is described which takes finer control over the CIG than usual and may be applied to many existing systems with little modification.     

虚拟培训中的虚实联动技术

针对传统培训方式的不足,提出了虚实联动的培训方式.通过构建虚拟设备,硬件可编程逻辑控制器(PLC)、PC、虚拟设备三者之间的通信,实现虚拟设备和实体设备之间的联系,并且通过现场控制系统采集实体设备运行数据,将实体设备运行和故障等信息在虚拟设备中实时表现出来,拓展了虚拟培训的新思路.以喷印机虚拟联动系统为例论述了虚拟联动技术的必要性和可行性.     

A multimodal virtual reality interface for 3D interaction with VTK

The object-oriented visualization Toolkit (VTK) is widely used for scientific visualization. VTK is a visualization library that provides a large number of functions for presenting three-dimensional data. Interaction with the visualized data is controlled with two-dimensional input devices, such as mouse and keyboard. Support for real three-dimensional and multimodal input is non-existent. This paper describes VR-VTK: a multimodal interface to VTK on a virtual environment. Six degree of freedom input devices are used for spatial 3D interaction. They control the 3D widgets that are used to interact with the visualized data. Head tracking is used for camera control. Pedals are used for clutching. Speech input is used for application commands and system control. To address several problems specific for spatial 3D interaction, a number of additional features, such as more complex interaction methods and enhanced depth perception, are discussed. Furthermore, the need for multimodal input to support interaction with the visualization is shown. Two existing VTK applications are ported using VR-VTK to run in a desktop virtual reality system. Informal user experiences are presented. Arjan J. F. Kok is an assistant professor at the Department of Computer Science at the Open University of the Netherlands. He studied Computer Science at the Delft University of Technology, The Netherlands. He received his Ph.D. from the same university. He worked as a Scientist for TNO (Netherlands Organization for Applied Scientific Research) and as assistant professor at the Eindhoven University of Technology before he joined the Open University. His research interests are visualization, virtual reality, and computer graphics. Robert van Liere studied Computer Science at the Delft University of Technology, the Netherlands. He received his Ph.D. with the thesis “Studies in Interactive Scientific Visualization” at the University of Amsterdam. Since 1985, he has worked at CWI, the Center for Mathematics and Computer Science in Amsterdam in which he is the head of CWI’s visualization research group. Since 2004, he holds a part-time position as full professor at the Eindhoven University of Technology. His research interests are in interactive data visualization and virtual reality. He is a member of IEEE.