A geoscience perspective on immersive 3D gridded data visualization

We describe visualization software, Visualizer, that was developed specifically for interactive, visual exploration in immersive virtual reality (VR) environments. Visualizer uses carefully optimized algorithms and data structures to support the high frame rates required for immersion and the real-time feedback required for interactivity. As an application developed for VR from the ground up, Visualizer realizes benefits that usually cannot be achieved by software initially developed for the desktop and later ported to VR. However, Visualizer can also be used on desktop systems (unix/linux-based operating systems including Mac OS X) with a similar level of real-time interactivity, bridging the “software gap” between desktop and VR that has been an obstacle for the adoption of VR methods in the Geosciences. While many of the capabilities of Visualizer are already available in other software packages used in a desktop environment, the features that distinguish Visualizer are: (1) Visualizer can be used in any VR environment including the desktop, GeoWall, or CAVE, (2) in non-desktop environments the user interacts with the data set directly using a wand or other input devices instead of working indirectly via dialog boxes or text input, (3) on the desktop, Visualizer provides real-time interaction with very large data sets that cannot easily be viewed or manipulated in other software packages. Three case studies are presented that illustrate the direct scientific benefits realized by analyzing data or simulation results with Visualizer in a VR environment. We also address some of the main obstacles to widespread use of VR environments in scientific research with a user study that shows Visualizer is easy to learn and to use in a VR environment and can be as effective on desktop systems as native desktop applications.     

Virtual orthopedic surgery training

Medicine is one of the most promising areas for emerging computer graphics and virtual reality techniques. VR training simulators let surgeons practice on virtual body tissue and get the same feedback they would experience in performing a real operation. Hybrid VR systems permit medical practitioners to view the patient overlaid with 3D data sets derived from 3D scanners, thus providing doctors and surgeons with pseudo X-ray vision. While currently available immersive VR surgery systems usually require expensive hardware and software, we developed a desktop VR orthopedic surgery training system that can run on commonly available personal computers     

Effects of virtual presence and learning outcome using low-end virtual reality systems

Low-cost technology is essential to integrate Virtual Reality (VR) into educative institutions around the world. However, low-cost technology usually refers to low-end technology, which may compromise the level of immersion of the VR system. This study evaluates whether low-end and high-end VR systems achieve a comparable learning outcome regardless their immersion level. We also analyze the relationship between virtual presence and the learning outcome arising from a VR educational experience. An evaluation with 42 participants was conducted. We measured learning outcome and virtual presence under three different configurations, namely: a desktop computer, a low-end VR system, and a high-end VR system. The impact of simulator sickness was also analyzed. Results revealed a lower learning outcome in the less immersive configuration (i.e. desktop) and a similar learning outcome in both low-end and high-end VR systems. Even though low-end VR systems are less immersive and produce a lower level of virtual presence than high-end VR systems, the results support the use of low-end VR systems for educative applications.     

Frontiers in User-Computer Interaction

In this age of (near-)adequate computing power, the power and usability of the user interface is as key to an application's success as its functionality. Most of the code in modern desktop productivity applications resides in the user interface. But despite its centrality, the user interface field is currently in a rut: the WIMP (Windows, Icons, Menus, Point-and-Click GUI based on keyboard and mouse) has evolved little since it was pioneered by Xerox PARC in the early '70s. Computer and display form factors will change dramatically in the near future and new kinds of interaction devices will soon become available. Desktop environments will be enriched not only with PDAs such as the Newton and Palm Pilot, but also with wearable computers and large-screen displays produced by new projection technology, including office-based immersive virtual reality environments. On the input side, we will finally have speech-recognition and force-feedback devices. Thus we can look forward to user interfaces that are dramatically more powerful and better matched to human sensory capabilities than those dependent solely on keyboard and mouse. 3D interaction widgets controlled by mice or other interaction devices with three or more degrees of freedom are a natural evolution from their two-dimensional WIMP counterparts and can decrease the cognitive distance between widget and task for many tasks that are intrinsically 3D, such as scientific visualization and MCAD. More radical post-WIMP UIs are needed for immersive virtual reality where keyboard and mouse are absent. Immersive VR provides good driving applications for developing post-WIMP UIs based on multimodal interaction that involve more of our senses by combining the use of gesture, speech, and haptics.     

Motivational and cognitive benefits of training in immersive virtual reality based on multiple assessments

The main objective of this study was to examine the effectiveness of immersive virtual reality (VR) as a medium for delivering laboratory safety training. We specifically compare an immersive VR simulation, a desktop VR simulation, and a conventional safety manual. The sample included 105 first year undergraduate engineering students (56 females).  We include five types of learning outcomes including post‐test enjoyment ratings; pre‐ to post‐test changes in intrinsic motivation and self‐efficacy; a post‐test multiple choice retention test; and two behavioral transfer tests. Results indicated that the groups did not differ on the immediate retention test, suggesting that all three media were equivalent in conveying the basic knowledge. However, significant differences were observed favoring the immersive VR group compared to the text group on the two transfer tests involving the solving problems in a physical lab setting (d = 0.54, d = 0.57), as well as enjoyment (d = 1.44) and increases in intrinsic motivation (d = 0.69) and self‐efficacy (d = 0.60). The desktop VR group scored significantly higher than the text group on one transfer test (d = 0.63) but not the other (d= 0.11), as well as enjoyment (d =1.11) and intrinsic motivation (d =0.83).     

Extending the desktop workplace by a portable virtual reality system

Users are increasingly recognizing the potential of virtual reality (VR) technology for applications such as data analysis, design review, product development, production planning, marketing, training, etc. The currently established workflow is to design and construct at a desktop system with CAD or modeling software, and visualize and evaluate the results at one or more VR centers equipped with CAVEs or Powerwalls.Discussions with users of VR installations have shown that there is a demand for smaller and more cost efficient VR installations. We have proposed the concept of a small VR system, PI-casso, based on user requirements, guidelines for office workplaces and some end-user tests which showed important limitations and the ergonomics problems of current VR installations. PI-casso is a compact, fully immersive VR system which complements the classic desktop workplace.In this paper we describe a set of user requirements and the results of the design in forming end-user tests, in addition to the concept and the technical specifications of the newly developed system. The first prototype of PI-casso was demonstrated at HCII 2003, where specialists from the human factors/ergonomics and the VR communities used our system and provided suggestions for improvement. This expert feedback was used to develop the improved versions described in this paper.     

PC上虚拟现实仿真的实现

使用目前先进的ＶＲＭＬ语言进行三维构型,并分别利用ＶｒｍｌＳｃｒｉｐｔ,ＪａｖａＳｃｒｉｐｔ或Ｊａｖａ语言设计仿真器,然后在三维环境下实时驱动对象运动,实现了在普通ＰＣ上的虚拟现实仿真。在不对计算机硬件增加额外要求的情况下,达到了虚拟现实仿真的基本要求,对上述难题给出了一种解决框架,用一个Ｍ／Ｍ／１／Ｋ排队模型进行了实例分析,并对今后的研究进行了探讨。     

A hardware-independent virtual reality development system

Simulating virtual reality (VR) hardware allows programs to be written in a desktop environment without constant use of limited VR resources. Rather than shifting constantly between VR and workstation environments, developers at the Electronic Visualization Laboratory (EVL) wanted to be able to test VR applications on the normal workstation console. We therefore created a software simulator for VR development. It simulates various VR system features with an interface that runs on an ordinary workstation. The simulator is implemented as part of the CAVE library, the programming library originally written to support the CAVE hardware. It can, however, be used to develop applications for several VR systems, including ImmersaDesks and head-coupled displays. The library itself has been designed so that use of the simulator or any supported hardware is entirely transparent to application code     

Alice: rapid prototyping for virtual reality

Virtual reality has sparked many people's imaginations, but writing VR programs remains difficult. Besides the obvious problems of managing arcane I/O devices (trackers, gloves, and so on), the programs must allow the participant to operate effectively in the immersive environment. Virtual environments present a new medium for both the participant and the programmer/author. In the University of Virginia's User Interface Group, we believe the best way to accelerate development in a new medium such as VR is to provide tools that allow people without highly technical backgrounds to create programs for it. These novice authors must be able to quickly try different nuances of an idea. They must be able to easily ask “what if” questions. To support this goal, we are developing Alice, a rapid prototyping environment that can generate VR environments. The name “Alice” honors Lewis Carroll's heroine, who explored a rapidly changing, dynamic environment     

Three views of virtual reality: nonimmersive virtual reality

Nonimmersive virtual reality (VR), which places the user in a 3D environment that can be directly manipulated with a conventional graphics workstation using a monitor, a keyboard; and a mouse, is discussed. The scene is displayed with the same 3D depth cues used in immersive VR: perspective view, hidden-surface elimination, color, texture, lighting, shading and shadows. As in immersive VR, animation and simulation are interactively controlled in response to the user's direct manipulation. Much of the technology used to support immersive and nonimmersive VR is the same. They use the same 3D modeling and rendering and many of the same interaction techniques. The advantages and applications of nonimmersive VR systems are discussed. Immersive and nonimmersive VR systems are compared and hybrid possibilities are reviewed     

A virtual reality keyboard with realistic haptic feedback in a fully immersive virtual environment

This study presents a 3D virtual reality (VR) keyboard system with realistic haptic feedback. The system uses two five-fingered data gloves to track finger positions and postures, uses micro-speakers to create simulated vibrations, and uses a head-mounted display (HMD) for 3D display. When users press a virtual key in the VR environment, the system can provide realistic simulated key click haptic feedback to users. The results of this study show that the advantages of the haptic VR keyboard are that users can use it when wearing HMDs (users do not need to remove HMDs to use the VR keyboard), the haptic VR keyboard can pop-up display at any location in the VR environments (users do not need to go to a specific location to use an actual physical keyboard), and the haptic VR keyboard can be used to provide realistic key click haptic feedback (which other studies have shown enhances user performance). The results also show that the haptic VR keyboard system can be used to create complex vibrations that simulate measured vibrations from a real keyboard and enhance keyboard interaction in a fully immersive VR environment.     

Geometric calibration of head-mounted displays and its effects on distance estimation

Head-mounted displays (HMDs) allow users to observe virtual environments (VEs) from an egocentric perspective. However, several experiments have provided evidence that egocentric distances are perceived as compressed in VEs relative to the real world. Recent experiments suggest that the virtual view frustum set for rendering the VE has an essential impact on the user's estimation of distances. In this article we analyze if distance estimation can be improved by calibrating the view frustum for a given HMD and user. Unfortunately, in an immersive virtual reality (VR) environment, a full per user calibration is not trivial and manual per user adjustment often leads to mini- or magnification of the scene. Therefore, we propose a novel per user calibration approach with optical see-through displays commonly used in augmented reality (AR). This calibration takes advantage of a geometric scheme based on 2D point - 3D line correspondences, which can be used intuitively by inexperienced users and requires less than a minute to complete. The required user interaction is based on taking aim at a distant target marker with a close marker, which ensures non-planar measurements covering a large area of the interaction space while also reducing the number of required measurements to five. We found the tendency that a calibrated view frustum reduced the average distance underestimation of users in an immersive VR environment, but even the correctly calibrated view frustum could not entirely compensate for the distance underestimation effects.     

M2S maps: supporting real-world navigation with mobile VR

Mobile devices are becoming increasingly integrated into our society. In addition to entertaining people with applications like pervasive games, mobile devices can also help to address cognitive challenges people face in the real world. This paper, by drawing on research findings from cognitive psychology and geography, explores a design to use mobile VR to help people overcome one cognitive barrier in navigation, which is to establish the correspondence between 2D spatial information found in maps and 3D entities they perceive from the real world. The design offers users multi-format, multi-scale, and semantic (M²S) maps, ranging from 2D maps to 3D immersive environments, and helps users to connect 2D maps to the real world through 3D environments which are equipped with semantic representation and animation techniques. Consequently, users can apply various kinds of spatial knowledge, 2D or 3D, in understanding the real world as well as assisting in navigation. This research enhances the design repertoire of mobile VR, and suggests a way to integrate virtual environments into people’s real-world life by examining the cognitive implications of 3D models on users’ activities.     

桌面式虚拟现实维修训练系统的研究与应用

计算机虚拟现实技术提供一个具有直观感和沉浸感的人机界面,可作为一种用于维修训练的有效工具,但目前的相关研究主要针对沉浸式和增强型虚拟现实,开发成本高且应用性不强。该文提出一种面向对象的桌面式虚拟现实维修训练系统原型,利用面向对象的方法,将基于最优拆卸序列的维修模式运用到系统中。通过某型机械装置的虚拟拆卸,验证了该系统的有效性。     

State of the Art of Molecular Visualization in Immersive Virtual Environments

Visualization plays a crucial role in molecular and structural biology. It has been successfully applied to a variety of tasks, including structural analysis and interactive drug design. While some of the challenges in this area can be overcome with more advanced visualization and interaction techniques, others are challenging primarily due to the limitations of the hardware devices used to interact with the visualized content. Consequently, visualization researchers are increasingly trying to take advantage of new technologies to facilitate the work of domain scientists. Some typical problems associated with classic 2D interfaces, such as regular desktop computers, are a lack of natural spatial understanding and interaction, and a limited field of view. These problems could be solved by immersive virtual environments and corresponding hardware, such as virtual reality head-mounted displays. Thus, researchers are investigating the potential of immersive virtual environments in the field of molecular visualization. There is already a body of work ranging from educational approaches to protein visualization to applications for collaborative drug design. This review focuses on molecular visualization in immersive virtual environments as a whole, aiming to cover this area comprehensively. We divide the existing papers into different groups based on their application areas, and types of tasks performed. Furthermore, we also include a list of available software tools. We conclude the report with a discussion of potential future research on molecular visualization in immersive environments.     

Immersive Multiplayer Games With Tangible and Physical Interaction

In this paper, we present a new immersive multiplayer game system developed for two different environments, namely, virtual reality (VR) and augmented reality (AR). To evaluate our system, we developed three game applications-a first-person-shooter game (for VR and AR environments, respectively) and a sword game (for the AR environment). Our immersive system provides an intuitive way for users to interact with the VR or AR world by physically moving around the real world and aiming freely with tangible objects. This encourages physical interaction between players as they compete or collaborate with other players. Evaluation of our system consists of users' subjective opinions and their objective performances. Our design principles and evaluation results can be applied to similar immersive game applications based on AR/VR.     

Multi-modal virtual environments for education with haptic and olfactory feedback

It has been suggested that immersive virtual reality (VR) technology allows knowledge-building experiences and in this way provides an alternative educational process. Important key features of constructivist educational computer-based environments for science teaching and learning, include interaction, size, transduction and reification. Indeed, multi-sensory VR technology suits very well the needs of sciences that require a higher level of visualization and interaction. Haptics that refers to physical interactions with virtual environments (VEs) may be coupled with other sensory modalities such as vision and audition but are hardly ever associated with other feedback channels, such as olfactory feedback. A survey of theory and existing VEs including haptic or olfactory feedback, especially in the field of education is provided. Our multi-modal human-scale VE VIREPSE (virtual reality platform for simulation and experimentation) that provides haptic interaction using a string-based interface called SPIDAR (space interface device for artificial reality), olfactory and auditory feedbacks is described. An application that allows students experiencing the abstract concept of the Bohr atomic model and the quantization of the energy levels has been developed. Different configurations that support interaction, size and reification through the use of immersive and multi-modal (visual, haptic, auditory and olfactory) feedback are proposed for further evaluation. Haptic interaction is achieved using different techniques ranging from desktop pseudo-haptic feedback to human-scale haptic interaction. Olfactory information is provided using different fan-based olfactory displays (ODs). Significance of developing such multi-modal VEs for education is discussed.     

V-Pong: an immersive table tennis simulation

Most applications for immersive virtual environments (VEs) allow slow-or medium-speed user interaction. Examples of this kind of interaction include changing an object's position, triggering an action, or setting a control parameter. To broaden the application range for VR systems, we need to integrate technologies that allow for faster VE-user movements. This raises the question, What response times can we achieve with VR systems built from standard recent hardware components? To answer this question, we created a table tennis simulation as this game involves fast user movements and has moderate space requirements. In this article we report on the realization of our immersive table tennis simulation, V-Pong.     

The Role of 3-D Sound in Human Reaction and Performance in Augmented Reality Environments

Three-dimensional sound's effectiveness in virtual reality (VR) environments has been widely studied. However, due to the big differences between VR and augmented reality (AR) systems in registration, calibration, perceptual difference of immersiveness, navigation, and localization, it is important to develop new approaches to seamlessly register virtual 3-D sound in AR environments and conduct studies on 3-D sound's effectiveness in AR context. In this paper, we design two experimental AR environments to study the effectiveness of 3-D sound both quantitatively and qualitatively. Two different tracking methods are applied to retrieve the 3-D position of virtual sound sources in each experiment. We examine the impacts of 3-D sound on improving depth perception and shortening task completion time. We also investigate its impacts on immersive and realistic perception, different spatial objects identification, and subjective feeling of "human presence and collaboration". Our studies show that applying 3-D sound is an effective way to complement visual AR environments. It helps depth perception and task performance, and facilitates collaborations between users. Moreover, it enables a more realistic environment and more immersive feeling of being inside the AR environment by both visual and auditory means. In order to make full use of the intensity cues provided by 3-D sound, a process to scale the intensity difference of 3-D sound at different depths is designed to cater small AR environments. The user study results show that the scaled 3-D sound significantly increases the accuracy of depth judgments and shortens the searching task completion time. This method provides a necessary foundation for implementing 3-D sound in small AR environments. Our user study results also show that this process does not degrade the intuitiveness and realism of an augmented audio reality environment     

运用Java3D体系结构实现虚拟对象控制

讨论了使用Java3D控制体系结构实现虚拟对象控制,以及使用虚拟现实开发环境构建3D平台所遇到的某些问题,通过虚拟对象控制机制成功地实现一系列虚拟空间行为动作。