Using 3D sound as a navigational aid in virtual environments

As current virtual environments are less visually rich than real-world environments, careful consideration must be given to their design to ameliorate the lack of visual cues. One important design criterion in this respect is to make certain that adequate navigational cues are incorporated into complex virtual worlds. In this paper we show that adding 3D spatialized sound to a virtual environment can help people navigate through it. We conducted an experiment to determine if the incorporation of 3D sound (a) helps people find specific locations in the environment, and (b) influences the extent to which people acquire spatial knowledge about their environment. Our results show that the addition of 3D sound did reduce time taken to locate objects in a complex environment. However, the addition of sound did not increase the amount of spatial knowledge users were able to acquire. In fact, the addition of 3D auditory sound cues appears to suppress the development of overall spatial knowledge of the virtual environment.     

Using 3D sound as a navigational aid in virtual environments

As current virtual environments are less visually rich than real-world environments, careful consideration must be given to their design to ameliorate the lack of visual cues. One important design criterion in this respect is to make certain that adequate navigational cues are incorporated into complex virtual worlds. In this paper we show that adding 3D spatialized sound to a virtual environment can help people navigate through it. We conducted an experiment to determine if the incorporation of 3D sound (a) helps people find specific locations in the environment, and (b) influences the extent to which people acquire spatial knowledge about their environment. Our results show that the addition of 3D sound did reduce time taken to locate objects in a complex environment. However, the addition of sound did not increase the amount of spatial knowledge users were able to acquire. In fact, the addition of 3D auditory sound cues appears to suppress the development of overall spatial knowledge of the virtual environment.     

A domain-oriented end-user design environment for generating interactive 3D virtual chemistry experiments

Three-dimensional virtual worlds are potentially feasible for building virtual educational environments. However, educators face technical challenges to apply these technologies because creating virtual educational environments based on virtual worlds demands 3D modeling and programming skills. This paper proposes a method to lower the technical barriers through domain-oriented interfaces with languages and environments that are familiar to educators. A domain-oriented end-user design environment, iVirtualWorld, is designed and developed to implement the proposed method in a specific educational domain, namely introductory chemistry experiments. This web-based environment provides end-users with domain-oriented building blocks, which can be assembled to create 3D virtual chemistry experiments. A usability evaluation and a comparative case study are designed to evaluate the system among chemistry educators, who are the target audience. The usability evaluation contains a task requiring participants to create a 3D virtual chemistry experiment and a voluntary semi-structured interview. The case study compares a virtual experiment generated using iVirtualWorld with an experiment in a commercial virtual chemistry laboratory system. The results show that 1) the domain-oriented end-user design environment enables participants to generate the 3D virtual chemistry experiment within 30 min; 2) participants gain confidence on creating 3D virtual experiments by themselves using iVirtualWorld; 3) participants confirm the usefulness of applying the system in introductory chemistry education; and 4) iVirtualWorld is considered more intuitive and straightforward for students to focus on finishing the experiment without being distracted than the commercial virtual chemistry laboratory system. Areas that can benefit from the system most and areas where the system is less effective are identified by participants. The responses also reveal the limitations of the current system and suggest directions for future improvement.     

Evaluation of a low-cost 3D sound system for immersive virtual reality training systems

Since head mounted displays (HMD), datagloves, tracking systems, and powerful computer graphics resources are nowadays in an affordable price range, the usage of PC-based "virtual training systems" becomes very attractive. However, due to the limited field of view of HMD devices, additional modalities have to be provided to benefit from 3D environments. A 3D sound simulation can improve the capabilities of VR systems dramatically. Unfortunately, realistic 3D sound simulations are expensive and demand a tremendous amount of computational power to calculate reverberation, occlusion, and obstruction effects. To use 3D sound in a PC-based training system as a way to direct and guide trainees to observe specific events in 3D space, a cheaper alternative has to be provided, so that a broader range of applications can take advantage of this modality. To address this issue, we focus in this paper on the evaluation of a low-cost 3D sound simulation that is capable of providing traceable 3D sound events. We describe our experimental system setup using conventional stereo headsets in combination with a tracked HMD device and present our results with regard to precision, speed, and used signal types for localizing simulated sound events in a virtual training environment     

A Virtual Environment for Teaching Social Skills: AViSSS

For typical adolescents, developing appropriate social skills can be difficult. For those with Asperger's syndrome (AS), this task is much more difficult and can be a determining factor in school success. Recent studies indicate that students with AS, who are often visual learners, can learn social skills through computer-based tasks. To help individuals with AS deal with social challenges, researchers have developed the Animated Visual Supports for Social Skills (AViSSS) 3D virtual environment. In AViSSS, a rendering engine aids the rendering of animations, characters, objects, and environments. As students interact in the virtual environment, AViSSS presents them with various situations and possible responses. Each response delivers a different simulated result, along with an explanation of the choice.     

The learner characteristics,features of desktop 3D virtual reality environments,and college chemistry instruction: A structural equation modeling analysis

We examined a model of the impact of a 3D desktop virtual reality environment on the learner characteristics (i.e. perceptual and psychological variables) that can enhance chemistry-related learning achievements in an introductory college chemistry class. The relationships between the 3D virtual reality features and the chemistry learning test as it relates to the selected perceptual (spatial orientation and usability) and psychological (self-efficacy and presence) variables were analyzed using the structural equation modeling approach. The results supported all the hypothesized relationships except one. Usability strongly mediated the relationship between 3D virtual reality features, spatial orientation, self-efficacy, and presence. Spatial orientation and self-efficacy had statistically significant, positive impact on the chemistry learning test. The results indicate that 3D virtual reality-based instruction is effective for enhancing students’ chemistry achievement. Overall, this study contributed a research model that can help increase the effectiveness of desktop virtual reality environments for enhancing spatial ability and science achievement. Moreover, this study provides insight to science educators, instructional designers, and multimedia developers who are interested in designing science-based instruction using instructional design principles.     

The wise cursor: assisted selection in 3D serious games

In recent years, the evolution of 3D graphics hardware and software has lead to a growing interest for serious games in three-dimensional virtual environments for learning, training, and rehabilitation. Many of these games are based on a first-person-shooter paradigm in which users navigate through the environment, select, and manipulate virtual objects. The target users of these applications are not necessarily usual gamers, and they often have difficulties in navigating and interacting in the 3D environment. This paper proposes the wise cursor, a new method for selection that improves the usability and accessibility of mouse-driven serious games in 3D environments. At each user click, the proposed method computes a list of objects candidates to be selected and their probability of being the desired one. Depending on the uncertainty of the probability distribution, either one object is selected or a mechanism to clarify the selection is proposed. In the former case, if the selected object is within the user avatar’s scope, the action associated to it is realized, otherwise the application automatically navigates toward it. In this way, selection and navigation are eased. The empirical results of the usability tests show that this technique is fast, practical, and that it requires little user’s skills. Thus, it can make serious games usable for a wider range of users who can concentrate on the training objectives without technological barriers.     

Real‐Time Analytic Antialiased Text for 3‐D Environments

Text is a crucial component of 3‐D environments and virtual worlds for user interfaces and wayfinding. Implementing text using standard antialiased texture mapping leads to blurry and illegible writing which hinders usability and navigation. While super‐sampling removes some of these artifacts, distracting artifacts can still impede legibility, especially for recent high‐resolution head‐mounted displays. We propose an analytic antialiasing technique that efficiently computes the coverage of text glyphs, over pixel footprints, designed to run at real‐time rates. It decomposes glyphs into piecewise‐biquadratics and trapezoids that can be quickly area‐integrated over a pixel footprint to provide crisp legible antialiased text, even when mapped onto an arbitrary surface in a 3‐D virtual environment.     

Using virtual environments to train firefighters

Using virtual environments for training and mission rehearsal gives US Navy firefighters an edge in fighting real fires. A test run on the ex-USS Shadwell measured the improvement. The results suggest that virtual environments serve effectively for training and mission rehearsal for shipboard firefighting. VE training provides a flexible environment where a firefighter can not only learn an unfamiliar part of the ship, but also practice tactics and procedures for fighting a fire by interacting with simulated smoke and fire without risking lives or property. These tests proved a successful first step in developing a new training technology for shipboard firefighting based on immersive virtual environments. The tests also indicated potential areas for improvement, requiring additional research. User interaction techniques for manipulating objects in VEs need further study, along with usability studies to determine their effectiveness or utility. Other areas that could enhance VE training systems include more natural and intuitive I/O devices such as 3D sound, speech and natural language input, integrated multimedia and hypermedia instruction, and multi user interaction     

An experimental study on the role of 3D sound in augmented reality environment

Investigation of augmented reality (AR) environments has become a popular research topic for engineers, computer and cognitive scientists. Although application oriented studies focused on audio AR environments have been published, little work has been done to vigorously study and evaluate the important research questions of the effectiveness of three-dimensional (3D) sound in the AR context, and to what extent the addition of 3D sound would contribute to the AR experience.

Thus, we have developed two AR environments and performed vigorous experiments with human subjects to study the effects of 3D sound in the AR context. The study concerns two scenarios. In the first scenario, one participant must use vision only and vision with 3D sound to judge the relative depth of augmented virtual objects. In the second scenario, two participants must cooperate to perform a joint task in a game-based AR environment.

Hence, the goals of this study are (1) to access the impact of 3D sound on depth perception in a single-camera AR environment, (2) to study the impact of 3D sound on task performance and the feeling of ‘human presence and collaboration’, (3) to better understand the role of 3D sound in human–computer and human–human interactions, (4) to investigate if gender can affect the impact of 3D sound in AR environments. The outcomes of this research can have a useful impact on the development of audio AR systems, which provide more immersive, realistic and entertaining experiences by introducing 3D sound. Our results suggest that 3D sound in AR environment significantly improves the accuracy of depth judgment and improves task performance. Our results also suggest that 3D sound contributes significantly to the feeling of human presence and collaboration and helps the subjects to ‘identify spatial objects’.     

Using immersive game-based virtual reality to teach fire-safety skills to children

Virtual reality (VR) has been used both to simulate situations that are too dangerous to practice in real life and as a tool to help children learn. This study was conducted as part of a larger more comprehensive long-term research project which aims to combine the two techniques and demonstrate a novel application of the result, using immersive VR to help children learn about fire hazards and practice escape techniques. In the current study, a CAVE was used to immerse participants in a fire scene. To improve the children’s motivation for learning over prior VR fire-safety training methods, game-like interface interaction techniques were used and students were encouraged to explore the virtual world. Rather than being passive viewers, as in prior related studies, the children were given full control to navigate through the virtual environment and to interact with virtual objects using a game pad and a 6DOF wand. Students identified home fire hazards with a partner and then practiced escaping from a simulated fire in the virtual environment. To test for improved motivation, a user study was completed. Results indicate that students were more engaged by the new game-like learning environment and that they reported that they found the experience fun and intriguing. Their enhanced enthusiasm for what is relatively standard fire-safety information demonstrates the promise of using game-based virtual environments for vital but otherwise tedious fire-safety skills training for children.     

Applied virtual environments to support learning of social interaction skills in users with Asperger's Syndrome

Asperger's Syndrome (AS) is an autistic spectrum disorder characterised by normal to high IQ but with marked impairment in social skills. Successful social skills training appears to be best achieved either in situ or in role-play situations where users can explore different outcomes resulting from their social behaviour. Single user virtual environments (SVEs) provide an opportunity for users with AS to learn social interaction skills in a safe environment which they can visit as many times as they like.The use of game-like tasks can provide an incentive and can also be used to guide the user through progressive learning stages. Collaborative virtual environments (CVEs) allow several users to interact simultaneously within the virtual environment, each taking different perspectives or role-play characters. Within the AS interactive project a series of SVEs and CVEs have been developed in collaboration with users and professional groups with an overall aim of supporting social skills learning. Initial evaluation studies have been carried out which have been used to both inform and refine the design of these virtual environments (VEs) as well as giving an insight into the understanding and interpretation of these technologies by users with AS.     

Process of design and usability evaluation of a telepsychology web and virtual reality system for the elderly: Butler

Butler is a multi-application system based on Internet technology and virtual reality, designed to meet the needs of elderly users. Its objectives are to assist elderly populations in accessing technology and new forms of communication, replicating, as far as possible the way of lineal interaction that this population group had traditionally to interact with information in order to facilitate the first contacts with technology. Butler is equipped with e-mail, videoconferencing, blogging, access to the Internet through a well-known web search engine, image and sound gallery, new contacts searcher, and two virtual reality environments. In this paper we describe Butler's most relevant features taking into account the usability design process and the usability evaluation of the system, which, after pilot testing, have proven to be highly acceptable and satisfactory to users. We also present the results of evaluations pertaining to the iconography used in the program, the navigation and users' help design process, and the first functional prototype, all of which were designed and redesigned with the aid of web heuristics and guidelines for the web design for elderly users. Results suggest important features to be taken into account when designing and developing this sort of application for the elderly.     

The influence of different sensory cues as selection feedback and co-location in presence and task performance

For some applications based on virtual reality technology, presence and task performance are important factors to validate the experience. Different approaches have been adopted to analyse the extent to which certain aspects of a computer-generated environment may enhance these factors, but mainly in 2D graphical user interfaces. This study explores the influence of different sensory modalities on performance and the sense of presence experienced within a 3D environment. In particular, we have evaluated visual, auditory and active haptic feedback for indicating selection of virtual objects. The effect of spatial alignment between proprioceptive and visual workspaces (co-location) has also been analysed. An experiment has been made to evaluate the influence of these factors in a controlled 3D environment based on a virtual version of the Simon game. The main conclusions obtained indicate that co-location must be considered in order to determine the sensory needs during interaction within a virtual environment. This study also provides further evidence that the haptic sensory modality influences presence to a higher extent, and that auditory cues can reduce selection times. Conclusions obtained provide initial guidelines that will help designers to set out better selection techniques for more complex environments, such as training simulators based on VR technology, by highlighting different optimal configurations of sensory feedback.     

A transatlantic research and development environment [3D virtualgraphics]

Distributed 3D virtual environments can help researchers conduct experiments globally with remotely located participants. We discuss challenges and opportunities for the shared work environment. Our world is entering an age where our current understanding of telecommunications and graphics computing is constantly challenged. With the availability of global information highways, 3D graphical intercontinental collaboration will become a part of our daily work routine. Already, world class auto makers are attempting to reduce car development time to two-year cycles, enlisting global engineering teams. However, this process requires new tools such as shared 3D CAD and distributed product data management systems. The Fraunhofer Center for Research in Computer Graphics (CRCG) in the United States and the Fraunhofer Institute for Computer Graphics (IGD) in Germany, are looking ahead into this new age by establishing a transcontinental computer graphics research effort and a proposed G-7 testbed. We are studying how 3D computer graphics and virtual environments can aid global collaborative work. We have focused our research efforts on determining how computer networks can transform the distributed workplace into a shared environment, allowing real time interaction among people and processes without regard to their location     

Creation and evaluation of a multi-sensory virtual assembly environment

The multi-modal information presentation, integrated into the virtual environment （VE）, has potential for stimulating different senses, improving the user＇s impression of immersion, and increasing the amount of information that is accepted and processed by the user＇s perception system. The increase of the useful feedback information may reduce the user＇s cognitive load, thus enhancing the user＇s efficiency and performance while interacting with VEs. This paper presents our creation of a multi-sensory virtual assembly environment （VAE） and the evaluation of the effects of multi-sensory feedback on the usability. The VAE brings together complex technologies such as constraint-based assembly simulation, optical motion tracking technology, and real-time 3D sound generation technology around a virtual reality workbench and a common software platform. The usability evaluation is in terms of its three attributes： efficiency of use, user satisfaction, and reliability. These are addressed by using task completion times （TCTs）, questionnaires, and human performance error rates （HPERs）, respectively. Two assembly tasks have been used to perform the experiments, using sixteen participants. The outcomes showed that the multi-sensory feedback could improve the usability. They also indicated that the integrated feedback offered better usability than either feedback used in isolation. Most participants preferred the integrated feedback to either feedback （visual or auditory） or no feedback. The participants＇ comments demonstrated that nonrealistic or inappropriate feedback had negative effects on the usability, and easily made them feel frustrated. The possible reasons behind the outcomes are also analysed by using a unifying human computer interaction framework. The implications, concluded from the outcomes of this work, can serve as useful guidelines for improving VE system design and implementation.     

Navigation in hypertext through virtual environments

Users of hypertext systems require some form of navigation aid or 'browser'. Typically these have been produced as schematic representations of the hypertext network. The advent of 'virtual reality' technology, and particularly less costly forms of this, offers new possibilities for hypertext navigation whereby information can be presented in the form of a virtual environment. Users can then move around the virtual world in much the same conceptual fashion as they move around the real world. Based upon an extensive review of hypertext navigation problems and the aiding systems proposed to deal with these, four different types of navigation aid have been built; these are schematic and spatial representations in two and three dimensions. Of particular interest is the 3D spatial representation, built as a virtual environment in Virtus Walkthrough; the proposition is that such a virtual world navigation aid will offer substantial usability advantages for hypertext databases.     

基于多视频的虚实融合系统

提出了一种基于多视频的虚实融合可视化系统的构建方法,旨在将真实世界中的图像和视频融合到虚拟场景中,用视频图像中的纹理和动态信息去丰富虚拟场景,提高虚拟环境的真实性,得到一种增强的虚拟环境.利用无人机采集图像来重建虚拟场景,并借助图像特征点的匹配来实现视频图像的注册.然后利用投影纹理映射技术,将图像投影到虚拟场景中.视频中的动态物体由于在虚拟环境中缺失对应的三维模型,直接投影,当视点发生变化时会产生畸变.首先检测和追踪这些物体,然后尝试使用多种显示方式来解决畸变问题.此外,系统还考虑有重叠区域的多视频之间的融合.实验结果表明,所构造的虚实融合环境是十分有益的.     

基于虚拟原型的概念设计描述模型V-desModel

传统的概念设计方法由于缺乏真实感的交互手段,难以直观表达设计者的意图.基于虚拟原型的概念设计不仅能为设计者提供逼真的虚拟设计环境,而且充分体现了现代设计的成本低、周期短以及灵活性强等特点.针对概念设计的特点以及虚拟原型的特征分类,提出了基于虚拟原型的概念设计模型V-desModel,其核心是利用产品视图模型描述设计对象,将虚拟特征概念融入视图模型中,并采用可扩展"三维实体-约束图"来描述设计对象之间的约束关系.V-desModel模型能有效地支持基于虚拟原型的概念设计过程,较好地解决了概念设计中产品虚拟原