虚拟场景中环境声源仿真技术综述

环境声音作为日常生活中分布最为广泛的一类声音,是人们获取外部信息的重要来源.近十几年来,随着用户对虚拟场景真实度要求不断提升,为虚拟场景打造同步、真实的环境音效已成为构建高度沉浸式虚拟环境不可或缺的一部分.其中环境声源仿真作为打造真实感虚拟环境音效的基石,得到了研究人员的广泛关注与探索.与传统的人工声源仿真相比,通过算...     

Virtual Reality: How Much Immersion Is Enough?

Solid evidence of virtual reality's benefits has graduated from impressive visual demonstrations to producing results in practical applications. Further, a realistic experience is no longer immersion's sole asset. Empirical studies show that various components of immersion provide other benefits - full immersion is not always necessary. The goal of immersive virtual environments (VEs) was to let the user experience a computer-generated world as if it were real - producing a sense of presence, or "being there," in the user's mind.     

The Role of 3-D Sound in Human Reaction and Performance in Augmented Reality Environments

Three-dimensional sound's effectiveness in virtual reality (VR) environments has been widely studied. However, due to the big differences between VR and augmented reality (AR) systems in registration, calibration, perceptual difference of immersiveness, navigation, and localization, it is important to develop new approaches to seamlessly register virtual 3-D sound in AR environments and conduct studies on 3-D sound's effectiveness in AR context. In this paper, we design two experimental AR environments to study the effectiveness of 3-D sound both quantitatively and qualitatively. Two different tracking methods are applied to retrieve the 3-D position of virtual sound sources in each experiment. We examine the impacts of 3-D sound on improving depth perception and shortening task completion time. We also investigate its impacts on immersive and realistic perception, different spatial objects identification, and subjective feeling of "human presence and collaboration". Our studies show that applying 3-D sound is an effective way to complement visual AR environments. It helps depth perception and task performance, and facilitates collaborations between users. Moreover, it enables a more realistic environment and more immersive feeling of being inside the AR environment by both visual and auditory means. In order to make full use of the intensity cues provided by 3-D sound, a process to scale the intensity difference of 3-D sound at different depths is designed to cater small AR environments. The user study results show that the scaled 3-D sound significantly increases the accuracy of depth judgments and shortens the searching task completion time. This method provides a necessary foundation for implementing 3-D sound in small AR environments. Our user study results also show that this process does not degrade the intuitiveness and realism of an augmented audio reality environment     

Path planning directed motion control of virtual humans in complex environments

Natural motion synthesis of virtual humans have been studied extensively, however, motion control of virtual characters actively responding to complex dynamic environments is still a challenging task in computer animation. It is a labor and cost intensive animator-driven work to create realistic human motions of character animations in a dynamically varying environment in movies, television and video games. To solve this problem, in this paper we propose a novel approach of motion synthesis that applies the optimal path planning to direct motion synthesis for generating realistic character motions in response to complex dynamic environment. In our framework, SIPP (Safe Interval Path Planning) search is implemented to plan a globally optimal path in complex dynamic environments. Three types of control anchors to motion synthesis are for the first time defined and extracted on the obtained planning path, including turning anchors, height anchors and time anchors. Directed by these control anchors, highly interactive motions of virtual character are synthesized by motion field which produces a wide variety of natural motions and has high control agility to handle complex dynamic environments. Experimental results have proven that our framework is capable of synthesizing motions of virtual humans naturally adapted to the complex dynamic environments which guarantee both the optimal path and the realistic motion simultaneously.     

Hybrid inverse motion control for virtual characters interacting with sound synthesis Application to percussion motion

The ever growing use of virtual environments requires more and more engaging elements for enhancing user experiences. Specifically regarding sounding virtual environments, one promising option to achieve such realism and interactivity requirements is the use of virtual characters interacting with sounding objects. In this paper, we focus as a case study on virtual characters playing virtual music instruments. We address more specially the real-time motion control and interaction of virtual characters with their sounding environment for proposing engaging and compelling virtual music performances. Combining physics-based simulation with motion data is a recent approach to finely represent and modulate this motion-sound interaction, while keeping the realism and expressivity of the original captured motion. We propose a physically-enabled environment in which a virtual percussionist interacts with a physics-based sound synthesis algorithm. We introduce and extensively evaluate the Hybrid Inverse Motion Control (HIMC), a motion-driven hybrid control scheme dedicated to the synthesis of upper-body percussion movements. We also propose a physics-based sound synthesis model with which the virtual character can interact. Finally, we present an architecture offering an effective way to manage heterogenous data (motion and sound parameters) and feedback (visual and sound) that influence the resulting virtual percussion performances.     

Occlusion in collaborative augmented environments

Augmented environments superimpose computer enhancements on the real world. Such augmented environments are well suited for collaboration of multiple users. To improve the quality and consistency of the augmentation the occlusion of real objects by computer-generated objects and vice versa has to be implemented. We present methods how this can be done for a tracked user's body and other real objects and how irritating artifacts due to misalignments can be reduced. Our method is based on simulating the occlusion of virtual objects by a representation of the user modeled as kinematic chains of articulated solids. Smoothing the border between virtual world and occluding real reduces registration and modeling errors of this model. Finally, an implementation in our augmented environment and the resulting improvements are presented.     

Localization performance of binaurally recorded sounds with and without training

One of the problems associated with listening to binaurally recorded sound events is localization confusions. The main objective of this investigation was to find out whether a short training session prior to listening to binaural recordings through headphones would facilitate correct spatial perception of the sound field. Focus was on the localization of the sound stimuli in median plane. Sound signals were recorded with an artificial head in three different conditions namely, anechoic, highly reverberant and moderately reverberant. Fourteen subjects participated in the listening tests. All subjects were required to localize all virtual sound stimuli under two different conditions. The first condition had a short training session binaurally recorded in the same environments as preceeding sound stimuli, and only sound stimuli recorded in the same environment were presented. The second condition did not have a training session, and sound stimuli recorded in different environments were presented. Results showed that a short training session prior to listening to binaurally recorded sounds through headphones was useful as it facilitated localization performance. The biggest effect was in reduced amount of sounds perceived inside the head. It was most pronounced for sound stimuli recorded in anechoic environment.     

A virtual reality keyboard with realistic haptic feedback in a fully immersive virtual environment

This study presents a 3D virtual reality (VR) keyboard system with realistic haptic feedback. The system uses two five-fingered data gloves to track finger positions and postures, uses micro-speakers to create simulated vibrations, and uses a head-mounted display (HMD) for 3D display. When users press a virtual key in the VR environment, the system can provide realistic simulated key click haptic feedback to users. The results of this study show that the advantages of the haptic VR keyboard are that users can use it when wearing HMDs (users do not need to remove HMDs to use the VR keyboard), the haptic VR keyboard can pop-up display at any location in the VR environments (users do not need to go to a specific location to use an actual physical keyboard), and the haptic VR keyboard can be used to provide realistic key click haptic feedback (which other studies have shown enhances user performance). The results also show that the haptic VR keyboard system can be used to create complex vibrations that simulate measured vibrations from a real keyboard and enhance keyboard interaction in a fully immersive VR environment.     

An investigation into the modelling of virtual objects with sound vibration properties

In order to increase the power of virtual environments, several different attempts have been made to incorporate sound interactivity in some form. For example, several implementations of virtual environments permit the playing of a previously recorded soundfile upon the triggering of an associated event. The user may then, for instance, perceive the sound of a creaky door when one is opened. However, a relatively more effective system for entertaining joint audio and visual response may be derived by using physical modelling techniques. We have undertaken a pilot investigation in which virtual objects are implemented in a manner such that they implicitly possess vibration properties analogous to that of the real world. Consequently these objects are able to vibrate in response to stimulus. The vibrations may be visually perceived as, for example, wave patterns on the surface of an object, and acoustically perceived by mapping values representative of surface displacement to a loudspeaker. This paper discusses the current state of the project.     

Natural perspective projections for head-mounted displays

The display units integrated in today's head-mounted displays (HMDs) provide only a limited field of view (FOV) to the virtual world. In order to present an undistorted view to the virtual environment (VE), the perspective projection used to render the VE has to be adjusted to the limitations caused by the HMD characteristics. In particular, the geometric field of view (GFOV), which defines the virtual aperture angle used for rendering of the 3D scene, is set up according to the display field of view (DFOV). A discrepancy between these two fields of view distorts the geometry of the VE in a way that either minifies or magnifies the imagery displayed to the user. It has been shown that this distortion has the potential to affect a user's perception of the virtual space, sense of presence, and performance on visual search tasks. In this paper, we analyze the user's perception of a VE displayed in a HMD, which is rendered with different GFOVs. We introduce a psychophysical calibration method to determine the HMD's actual field of view, which may vary from the nominal values specified by the manufacturer. Furthermore, we conducted two experiments to identify perspective projections for HMDs, which are identified as natural by subjects--even if these perspectives deviate from the perspectives that are inherently defined by the DFOV. In the first experiment, subjects had to adjust the GFOV for a rendered virtual laboratory such that their perception of the virtual replica matched the perception of the real laboratory, which they saw before the virtual one. In the second experiment, we displayed the same virtual laboratory, but restricted the viewing condition in the real world to simulate the limited viewing condition in a HMD environment. We found that subjects evaluate a GFOV as natural when it is larger than the actual DFOV of the HMD--in some cases up to 50 percent--even when subjects viewed the real space with a limited field of view.     

Augmented reality using personal projection and retroreflection

The support of realistic and flexible training simulations for military, law enforcement, emergency response, and other domains has been an important motivator for the development of augmented reality technology. An important vision for achieving this goal has been the creation of a versatile “stage” for physical, emotional, and cognitive training that combines virtual characters and environments with real world elements, such as furniture and props. This paper presents REFLCT, a mixed reality projection framework that couples a near-axis personal projector design with tracking and novel retroreflective props and surfaces. REFLCT provides multiple users with personalized, perspective-correct imagery that is uniquely composited for each user directly into and onto a surrounding environment, without any optics positioned in front of the user’s eyes or face. These characteristics facilitate team training experiences which allow users to easily interact with their teammates while wearing their standard issue gear. REFLCT can present virtual humans who can make deictic gestures and establish eye contact without the geometric ambiguity of a typical projection display. It can also display perspective-correct scenes that require a realistic approach for detecting and communicating potential threats between multiple users in disparate locations. In addition to training applications, this display system appears to be well matched with other user interface and application domains, such as asymmetric collaborative workspaces and personal information guides.     

Height effects in real and virtual environments

The study compared human perceptions of height, danger, and anxiety, as well as skin conductance and heart rate responses and postural instability effects, in real and virtual height environments. The 24 participants (12 men, 12 women), whose average age was 23.6 years, performed "lean-over-the-railing" and standing tasks on real and comparable virtual balconies, using a surround-screen virtual reality (SSVR) system. The results indicate that the virtual display of elevation provided realistic perceptual experience and induced some physiological responses and postural instability effects comparable to those found in a real environment. It appears that a simulation of elevated work environment in a SSVR system, although with reduced visual fidelity, is a valid tool for safety research. Potential applications of this study include the design of virtual environments that will help in safe evaluation of human performance at elevation, identification of risk factors leading to fall incidents, and assessment of new fall prevention strategies.     

基于X3D实现虚拟镜面成像效果的研究

真实感是虚拟现实的重要特性之一,而镜面成像现象使现实世界变得更加多姿多彩。这篇文章主要介绍在基于X3D的虚拟世界中实现镜面成像效果以增强虚拟现实环境真实感的机理和方法,将在构建虚拟现实环境的真实感方面具有广阔的应用前景。     

虚拟环境中的低成本实时立体声显示

三维的听觉信息在多模态交互与集成的虚拟环境巾得到了广泛的应用.以基于HLA平台的装甲兵对抗演示系统为背景,探讨了三维的听觉信息生成和显示问题,根据系统对声音显示节点的要求,研究了扬声器阵列的立体声显示原理、影响人对声音感知的因素、虚拟环境中声音的定位以及声音的距离仿真.基于矢量基幅值相移(VBAP)的方法,提出了在长方体的演示厅空间中建立扬声器阵列的方法,并在系统仿真过程中将虚拟环境中声源分为循环声、非循环声和环境背景声,并对它们的实现和处理过程以及实体可听距离等问题进行了研究.     

An overview of physically-based modelling techniques for virtual environments

A class of virtual environments is concerned with the representation of behaviour that is apparent in the real world. In order to model this behaviour, sophisticated physical models are required. The development of these models, classed asphysically-based modelling, is based upon the fundamental concepts of Newtonian dynamics. Considerable research into physically-based modelling has already been conducted by the computer graphics community, permitting realistic animation of object motion. The application of physical models to virtual environments poses further problems, not least that of real-time execution in a fully interactive environment. This paper gives an overview of the existing computer graphics research concerned with physically-based modelling, discussing the merits and problems of various techniques in terms of the requirements of virtual environment.     

Demand characteristics in assessing motion sickness in a virtual environment: or does taking a motion sickness questionnaire make you sick?

The experience of motion sickness in a virtual environment may be measured through pre and postexperiment self-reported questionnaires such as the Simulator Sickness Questionnaire (SSQ). Although research provides converging evidence that users of virtual environments can experience motion sickness, there have been no controlled studies to determine to what extent the user's subjective response is a demand characteristic resulting from pre and posttest measures. In this study, subjects were given either SSQ's both pre and postvirtual environment immersion, or only postimmersion. This technique tested for contrast effects due to demand characteristics in which administration of the questionnaire itself suggested to the participant that the virtual environment may produce motion sickness. Results indicate that reports of motion sickness after immersion in a virtual environment are much greater when both pre and postquestionnaires are given than when only a posttest questionnaire is used. The implications for assessments of motion sickness in virtual environments are discussed.     

Locales: supporting large multiuser virtual environments

Creators of multiuser virtual environments naturally desire to make them large in spatial extent, large in numbers of objects, and large in numbers of users interacting with the environment. However, doing so creates several problems: efficiently managing the flow of large amounts of data between large numbers of users, representing precise position and velocity information about objects arrayed across a large volume of space, and allowing designers to create parts of a virtual environment separately and combine them together later. The concept of locales is based on the idea that even in a very large virtual world, most of what a single user can observe at a given moment is nevertheless local in nature. You would expect a large virtual world to be large primarily because, like a city, it combines a large number of relatively small, localized activities not because it contains very large individual activities. Locales divide a virtual world into chunks that can be processed separately. This division is purely an implementation issue-it is not apparent to the user. A user sees several locales at once-generally the locale containing the user's point of view and those neighboring it. The user does not see any seams between the locales nor any abrupt changes as the point of view moves from one locale to another, thereby changing the neighborhood set     

Measurement of presence and its consequences in virtual environments

A sense of presence is one of the critical components required by any effective virtual environment (VE). In contrast, side effects such as sickness may be produced in some virtual environments, detracting from the enjoyment or usefulness of the VE and from subsequent performance of the participant. Both presence and sickness in virtual environments are multifactorial phenomena not easily amenable to understanding or measurement. The first experiment reported here compares use of direct performance measures and rating scales to assess presence, whilst varying the VE display medium (head mounted and desktop displays) and whether or not sound was used in the VE. The second experiment addresses associations between presence, sickness and enjoyment of virtual environment participation. There was enough comparability between a reflex response within the VE and the rating scales to justify future exploration of the former measure of presence. A number of explanations are given for the partial association found between presence and sickness.     

Intelligent Cruise-Control Navigation: A new navigation/travel method for use in virtual environments

An important feature of virtual reality is the facility for the user to move around a virtual environment in a natural and easily controlled manner. Navigation, also called locomotion, travel or motion, involves changing the perspective of the user in the virtual environment (VE). It allows the user to move in the VE as well as reorient themselves to look at the world differently. Natural locomotion methods are able to contribute to a sense of presence and reality. The illusion of presence can be lost through unnatural experiences during travel in the VE. This can be caused by poor interactive metaphors or by experiences which do not agree with the user's everyday understanding of the real world. This paper focuses on the navigation method in the VE, one of the major interfaces for the interactivity between human and VE in virtual reality circumstances and worlds. It proposes a new navigation method. Intelligent Cruise-Control Navigation (ICCN), which provides a natural and user-centred approach to navigation in the VE and can improve the user's sense of reality and presence. ICCN is composed of three major phases: Constant Velocity Navigation, Collision Detection and Avoidance, and Path Adjustment. The ICCN can reduce the user's fatigue and improve the user's presence in the VE. The small experimental study reported in this paper suggests that the ICCN will be a natural, straightforward, and useful navigation interface in VE.