State of the Art of Molecular Visualization in Immersive Virtual Environments

Visualization plays a crucial role in molecular and structural biology. It has been successfully applied to a variety of tasks, including structural analysis and interactive drug design. While some of the challenges in this area can be overcome with more advanced visualization and interaction techniques, others are challenging primarily due to the limitations of the hardware devices used to interact with the visualized content. Consequently, visualization researchers are increasingly trying to take advantage of new technologies to facilitate the work of domain scientists. Some typical problems associated with classic 2D interfaces, such as regular desktop computers, are a lack of natural spatial understanding and interaction, and a limited field of view. These problems could be solved by immersive virtual environments and corresponding hardware, such as virtual reality head-mounted displays. Thus, researchers are investigating the potential of immersive virtual environments in the field of molecular visualization. There is already a body of work ranging from educational approaches to protein visualization to applications for collaborative drug design. This review focuses on molecular visualization in immersive virtual environments as a whole, aiming to cover this area comprehensively. We divide the existing papers into different groups based on their application areas, and types of tasks performed. Furthermore, we also include a list of available software tools. We conclude the report with a discussion of potential future research on molecular visualization in immersive environments.     

Navigation help in 3D worlds: some empirical evidences on use of sound

The concept of Interaction Locus (IL) has been introduced to help the users to orient, navigate, and identify relevant interaction areas in 3D Virtual Environments (VEs). The IL is a multimodal concept: it adds to the 3D visual scene parallel information channels that are perceived by other senses. In particular, the IL emphasizes the role of music as a navigation aid in a VE. This paper reports three user-evaluations of different IL enriched virtual worlds, and in particular of the role of the IL auditory component. Results suggest that audio in 3D plays not only an aesthetic role, which the users greatly appreciate, but also a functional role simplifying navigation and helping the users to recognise scenes in the environment. Such a functional role however is subordinated to a proper understanding of the link between music and virtual space. While these experiments refer to desktop virtual reality environments, their findings are general enough to inform the design of navigational tools for other segments of the mixed reality domain.

Interaction with a desktop virtual environment: a 2D view into a 3D world

With the development of computer software and hardware in the past few years, it has been possible to produce effective training virtual environments on everyday personal computers with little expert training required for users or designers. However, the development of the equipment that enables this has brought little coinciding research on what features to include when designing these environments. Despite these increased advances in PC capabilities for desktop virtual environments (VEs), there are still limitations on the number of objects that can be programmed to be interactive, usually due to restrictions on programming time and cost. As a result, it is often left to the programmer to decide which of the objects included to increase the realism of the environment will be interactive and which aesthetic. The work presented in this paper is an experiment that aims to establish a guide for environment designers to aid effective environment interaction development by identifying key elements in a VE design.

Integration of manufacturing services into virtual environments over the internet

Virtual reality (VR) is a rapidly developing technology that has a wide spectrum of industrial and commercial applications. Networked (distributed or shared) virtual environments (VE) are of growing interest to modern manufacturing industry; a dominating use of networked virtual manufacturing environments (VMEs) is on-line visualisation and collaborative control of 3D information. This has to be supported by real-time data transfer. To meet a broad range of common requirements for Internet-based VE communications, particularly for virtual manufacturing and collaborative design and control, this paper presents a networked virtual environment system that is designed to support networked virtual design and manufacturing. The system is implemented with manufacturing message specification (MMS) standards so as to integrate a range of manufacturing services into networked VEs over the Internet.     

Effects of virtual environment platforms on emotional responses

The goal of the current study was to investigate the effects of different virtual environment (VE) technologies (i.e., desktop, head mounted display, or fully immersive platforms) on emotional arousal and task performance. Fifty-three participants were recruited from a college population. Reactivity to stressful VEs was examined in three VE systems from desktop to high-end fully immersive systems. The experiment was a 3 (desktop system, head mounted display, and six wall system) × 2 (high- and low-stressful VE) within subject design, with self-reported emotional arousal and valence, skin conductance, task performance, presence, and simulator sickness examined as dependent variables. Replicating previous studies, the fully immersive system induced the highest sense of presence and the head mounted display system elicited the highest amount of simulator sickness. Extending previous studies, the results demonstrated that VE platforms were associated with different patterns in emotional responses and task performance. Our findings suggest that different VE systems may be appropriate for different scientific purposes when studying stress reactivity using emotionally evocative tasks.     

Spatial representations of virtual mazes: the role of visual fidelity and individual differences

Twenty-four people learned three versions of a room-sized maze: a wire-frame desktop virtual environment (VE), a normal surface-rendered desktop VE, and a real-world maze. Differences among the mental representations formed from each environment were measured with pointing and distance estimation tasks in a real-world version of each maze. People were more accurate at pointing after having learned the real and wire-frame VE maze than the surface-rendered VE maze; however, this effect was small compared with the effect of individual differences. Differences in gender, spatial ability, and prior computer experience were significantly related to the ability to acquire spatial information from the desktop VE. There was a high correlation between spatial knowledge when it was measured in the VE and spatial knowledge measured in the real world. Actual or potential applications include the design of effective VE training systems.     

A VE framework to study visual perception and action

In the real world, vision operates in harmony with self-motion yielding the observer to unambiguous perception of the three-dimensional (3D) space. In laboratory conditions, because of technical difficulties, researchers studying 3D perception have often preferred to use the substitute of a stationary observer, somehow neglecting aspects of the action-perception cycle. Recent results in visual psychophysics have proved that self-motion and visual processes interact, leading the moving observer to interpret a 3D virtual scene differently from a stationary observer. In this paper we describe a virtual environment (VE) framework which presents very interesting characteristics for designing experiments in visual perception during action. These characteristics arise in a number of ways from the design of a unique motion capture device. First, its accuracy and the minimal latency in position measurement; second, its ease of use and the adaptability to different display interfaces. Such a VE framework enables the experimenter to recreate stimulation conditions characterised by a degree of sensory coherence typical of the real world. Moreover, because of its accuracy and flexibility, the same device can be used as a measurement tool to perform elementary but essential calibration procedures. The VE framework has been used to conduct two studies which compare the perception of 3D variables of the environment in moving and in stationary observers under monocular vision. The first study concerns the perception of absolute distance, i.e. the distance separating an object and the observer. The second study refers to the perception of the orientation of a surface both in the absence and presence of conflicts between static and dynamic visual cues. In the two cases, the VE framework has enabled the design of optimal experimental conditions, permitting light to be shed on the role of action in 3D visual perception.     

Measurement of presence and its consequences in virtual environments

A sense of presence is one of the critical components required by any effective virtual environment (VE). In contrast, side effects such as sickness may be produced in some virtual environments, detracting from the enjoyment or usefulness of the VE and from subsequent performance of the participant. Both presence and sickness in virtual environments are multifactorial phenomena not easily amenable to understanding or measurement. The first experiment reported here compares use of direct performance measures and rating scales to assess presence, whilst varying the VE display medium (head mounted and desktop displays) and whether or not sound was used in the VE. The second experiment addresses associations between presence, sickness and enjoyment of virtual environment participation. There was enough comparability between a reflex response within the VE and the rating scales to justify future exploration of the former measure of presence. A number of explanations are given for the partial association found between presence and sickness.     

Accessing information through multimodal 3D environments: towards universal access

3D environments represent a great opportunity for universal access to information, as they offer an intuitive interaction paradigm, similar to what is experienced by humans in their everyday lives. In spite of that, several 3D interfaces are characterized by poor structures and are hard to navigate. This paper presents the multimodal concept of the Interaction Locus (IL) as a means to give structure to 3D scenes, helping the user to interact with and access information inside them. The concept was initially developed with particular reference to desktop virtual reality (2.5 D virtual reality), but it is general enough to be extended to other contexts, such as real 3D scenes. The final part of this work shows how the IL concept addresses the need for a unified authoring methodology, capable of allowing access to different target user groups from a variety of different devices.     

Virtual Sample Processing: Extending the Reach of Multimedia

Traditional multimedia systems deal with only a few basic media: text, graphics, audio and video. However, many other types of media, such as ultrasound, infrared and RF signals, can be represented by streams of data samples and processed within multimedia applications. In this paper, we introduce some of these new media domains and identify interesting opportunities enabled by their software-based processing. We also describe our SpectrumWare testbed for experimenting with these new media types and report on our experience to date.We believe that the time has come to broaden the scope of multimedia to include any form of sampled information. Advances in processor and analog-to-digital conversion technology have brought raw sample streams within the grasp of desktop computers and media processing systems. Coupling these new media types with software-based processing allows the construction of virtual devices that can handle different sample sources, modify their behavior based on information extracted from the media, and transform information between media domains.     

Driving simulation: challenges for VR technology

Virtual driving environments represent a challenging test for virtual reality technology. We present an overview of our work on the problems of scenario and scene modeling for virtual environments (VEs) in the context of the Iowa Driving Simulator (IDS). The requirements of driving simulation-a deterministic real-time software system that integrates components for user interaction, simulation, and scenario and scene modeling-make it a valuable proving ground for VE technologies. The goal of our research is not simply to improve driving simulation, but to develop technology that benefits a wide variety of VE applications. For example, our work on authoring high-fidelity VE databases and on directable scenarios populated with believable agents also targets applications involving interaction with simulated, walking humans and training in the operation of complex machinery. This work has benefited greatly from the experience of developing components for a full-scale operational VE system like IDS, and we believe that many other proposed VE technologies would similarly benefit from such real-world testing     

一个基于Web的网络虚拟现实系统的设计与实现

虚拟现实技术发展很快,有广泛的工业和商业应用。网络虚拟现实技术已越来越为工业制造业所关注,虚拟制造环境(VME)的一个主要用途是在线可视化和对3D信息的协作控制,这需要实时数据传输的支持。为了满足基于Internet的VE通信需求,特别是为满足虚拟制造和协作设计和控制,该文设计并实现了一个支持网络虚拟设计和制造的网络虚拟现实系统,此系统用MMS协议实现VE通信,并将工业控制服务集成于基于Internet的虚拟环境中。     

Educational virtual environments: design rationale and architecture

The use of collaborative virtual environments in e-learning is one of the most promising uses of virtual reality technology. While much research has been done in the area of networked virtual environments corresponding to the sharing of events, very little research has been done on specific services and functionality. However both the requirements and the kind of the offered services affect significantly the design of a system. In this paper we present the design and implementation of a platform suitable for educational virtual environments, which are collaborative virtual environments aiming at offering collaborative e-learning services to the users. Apart from the platform itself, we present the technological choices, and a new method for sharing virtual environments. Furthermore, we present an e-learning environment to support e-learning services using collaborative virtual environments from both the technical and functional point of view, along with the end-user evaluation results.

The Hybrid World of Virtual Environments

Much of the work concerned with virtual environments has addressed the development of new rendering technologies or interaction techniques. As the technology matures and becomes adopted in a wider range of applications, there is, however, a need to better understand how this technology can be accommodated in software engineering practice. A particular challenge presented by virtual environments is the complexity of the interaction that is supported, and sometimes necessary, for a particular task. Methods such as finite-state automata which are used to represent and design dialogue components for more conventional interfaces, e.g. using direct manipulation within a desktop model, do not seem to capture adequately the style of interaction that is afforded by richer input devices and graphical models. In this paper, we suggest that virtual environments are, fundamentally, what are known as hybrid systems. Building on this insight, we demonstrate how techniques developed for modelling hybrid systems can be used to represent and understand virtual interaction in a way that can be used in the specification and design phases of software development, and which have the potential to support prototyping and analysis of virtual interfaces.     

Realization of reconfigurable virtual environments for virtual testing

1 Introduction The scalability of virtual environments (VEs) that allow users to change and modify their contents and context depending on application requirements has been a de facto feature of virtual reality technology. Engineering uses such as virtual testing and exper- imenting applications demand VEs to be highly or fully reconfigurable. In reality, however, satisfactory reconfiguration of a virtual environment is yet to be achieved, due to the high accuracy of spatial data in- put and…     

Navigational tools for desktop virtual environment interfaces

Desktop systems typically rely on a two-dimensional (2D) software interface and general purpose hardware (mouse, keyboard) for interaction with a three-dimensional (3D) virtual environment. These interfaces must provide all the functionality required to navigate through and interact with the virtual environment, yet research into the usability aspects of the tools presented on these software interfaces indicates that the majority of users experience some degree of frustration when using them to perform even relatively simple tasks. This paper begins with a study of usability issues for interfaces to virtual environments on desktop systems, and details a series of experiments performed to evaluate the usability of a number of navigational tools. Participants were tested on the time taken to complete a number of navigational tasks with a series of interfaces presenting different navigational tools. The tools presented were a speed control function, a you-are-here (YAH) map, a function enabling the user to mark and teleport to any location within the presented environment, and an undo function. Results indicate that the visual presentation of navigational aids improves navigation performance, in terms of the time taken to complete tasks, and also improves user satisfaction with the desktop system.     

几何建模技术在虚拟校园漫游系统开发中的应用

虚拟漫游系统的开发是虚拟现实技术的一个重要应用.以华北电力大学保定二校区为虚拟仿真环境,使用3DSMAX建模工具对校园中的各种模型进行建模,使用多种基于几何建模的场景优化措施,并基于Google Earth卫星截图进行场景的精确布局设计.结合Virtools虚拟现实开发平台,设计了虚拟校园漫游系统的开发流程,并在微机平台上实现了一个场景复杂的校园实时漫游系统.该校园漫游系统不但为用户提供了友好的Web界面,而且设计并实现了基于导航图控制的人机交互机制,便于综合使用多种漫游方式快速漫游校园的场景.     

Towards Environment-to-Environment (E2E) multimedia communication systems

We present an approach to connect multiple remote environments over web for natural interaction among people and objects. Focus of current communication and telepresence systems severely restrict user affordances in terms of movement, interaction, peripheral vision, spatio-semantic integrity and even information flow. These systems allow information transfer rather than experiential interaction. We propose Environment-to-Environment (E2E) as a new paradigm for communication which allows users to interact in natural manner using text, audio, and video by connecting environments. Each Environment is instrumented using as many different types of sensors as may be required to detect presence and activity of objects. This object position and activity information is used by a scalable event-based multimodal information system called EventServer to share the appropriate experiential information with other environments as well as to present incoming multimedia information on right displays and speakers. This paper describes the design principles for E2E communication, discusses system architecture, and gives our experience in implementing prototypes of such systems in telemedicine and office collaboration applications. We also discuss the research challenges and a road-map for creating more sophisticated E2E applications in near future.