A comparison of guidance cues in desktop virtual environments

Designers of educational and entertainment desktop virtual environments (VEs) have employed a variety of cues for motivating users to perform actions or adopt particular viewpoints. However, there has been little formal study comparing user responses to such cues. This paper reports the results of a preliminary study of five cues (agents, signs, man-made landmarks, environmental landmarks, and trails) for motivating actions in virtual environments. Given a sample task of navigating to a target destination, no significant differences between the cues were observed in terms of overall success or speed. However, significant differences between the cues were found on other measures, including minimization of detours (trails) and awareness of guidance (agents, signs, trails). Frequency of desktop VE usage was also found to influence performance.     

Improving virtual reality navigation tasks using a haptic vest and upper body tracking

Haptic feedback is an important component of immersive virtual reality (VR) applications that is often suggested to complement visual information through the sense of touch. This paper investigates the use of a haptic vest in navigation tasks. The haptic vest produces a repulsive vibrotactile feedback from nearby static virtual obstacles that augments the user spatial awareness. The tasks require the user to perform complex movements in a 3D cluttered virtual environment, like avoiding obstacles while walking backwards and pulling a virtual object. The experimental setup consists of a room-scale environment. Our approach is the first study where a haptic vest is tracked in real time using a motion capture device so that proximity-based haptic feedback can be conveyed according to the actual movement of the upper body of the user.User study experiments have been conducted with and without haptic feedback in virtual environments involving both normal and limited visibility conditions. A quantitative evaluation was carried out by measuring task completion time and error (collision) rate. Multiple haptic rendering techniques have also been tested. Results show that under limited visibility conditions proximity-based haptic feedback generated by a wearable haptic vest can significantly reduce the number of collisions with obstacles in the virtual environment.     

Gravity-Referenced Attitude Display for Mobile Robots: Making Sense of What We See

Attitude control refers to controlling the pitch and roll of a mobile robot. As environments become more complex and cues to a robot's pose become sparser, it becomes easy for a teleoperator using an egocentric (camera) display to lose situational awareness. Reported difficulties with teleoperated robots frequently involve rollovers and sometimes even failure to realize that a robot has rolled over. Attitude information has conventionally been displayed separately from the camera view using an artificial horizon graphic or individual indicators for pitch and roll. Information from separate attitude displays may be difficult to integrate with an ongoing navigation task and may lead to errors. In this paper, we report an experiment in which a simulated robot is maneuvered over rough exterior and interior terrains to compare a gravity-referenced view with a separated attitude indication. Results show shorter task times and better path choices for users of the gravity-referenced view but no reduction in rollovers     

Handling of Virtual Contact in Immersive Virtual Environments: Beyond Visuals

This paper addresses the issue of improving the perception of contact that users make with purely virtual objects in virtual environments. Because these objects have no physical component, the user's perceptual understanding of the material properties of the object, and of the nature of the contact, is hindered, often limited solely to visual feedback. Many techniques for providing haptic feedback to compensate for the lack of touch in virtual environments have been proposed. These systems have increased our understanding of the nature of how humans perceive contact. However, providing effective, general-purpose haptic feedback solutions has proven elusive. We propose a more-holistic approach, incorporating feedback to several modalities in concert. This paper describes a prototype system we have developed for delivering vibrotactile feedback to the user. The system provides a low-cost, distributed, portable solution for incorporating vibrotactile feedback into various types of systems. We discuss different parameters that can be manipulated to provide different sensations, propose ways in which this feedback can be combined with feedback of other modalities to create a better understanding of virtual contact, and describe possible applications.     

Spatialized Vibrotactile Feedback Improves Goal-Directed Movements in Cluttered Virtual Environments

Spatial awareness in virtual reality (VR) is a dominant research topic. It plays an essential role in the assessment of human operators’ behavior in simulated tasks, notably for the evaluation of the feasibility of manual maintenance tasks in cluttered industrial settings. In such contexts, it is decisive to evaluate the spatial and temporal correspondence between the operator’s movement kinematics and that of his/her virtual avatar in the virtual environment (VE). Often, in a cluttered VE, direct kinesthetic (force) feedback is limited or absent. We tested whether vibrotactile (cutaneous) feedback would increase visuo-proprioceptive consistency, spatial awareness, and thus the validity of VR studies, by augmenting the perception of the operator’s contact(s) with virtual objects. We present experimental results obtained using a head-mounted display (HMD) during a goal-directed task in a cluttered VE. Results suggest the contribution of spatialized vibrotactile feedback to visuo-proprioceptive consistency.     

Applications of the Responsive Workbench

Many interface devices for virtual reality provide full immersion inside the virtual environment. This is appropriate for numerous applications that emphasize navigating through a virtual space. However, a large class of problems exists for which navigation is not the critical issue. Rather, these applications demand a fine-granularity visualization and interaction with virtual objects and scenes. This applies to a host of other applications typically performed on a desktop, table or workbench. Responsive Workbench technology offers a new way to develop virtual environments for this rather sizable class of applications. The Responsive Workbench operates by projecting a computer-generated, stereoscopic image off a mirror and through a table surface. Using stereoscopic shuttered glasses, users observe a 3D image displayed above the tabletop. By tracking the group leader's head and hand movements, the Responsive Workbench permits changing the view angle and interacting with the 3D scene. Other group members observe the scene as the group leader manipulates it, facilitating communication among observers. Typical methods for interacting with virtual objects on the workbench include speech recognition, gesture recognition and a simulated laser pointer (stylus). This article features Responsive Workbench applications from four institutions that have pioneered this technology. The four applications are: visualization, situational awareness, collaborative production modeling and planning, and a virtual windtunnel     

Interactive middleware architecture for lifelog based context awareness

Due to the development of IT convergence, a wide variety of information is being produced and distributed rapidly in digital form. Lifelog based context awareness is a technology that provides a service automatically based on perceived situational information in ubiquitous environments. To offer customized services to users, the technology of acquiring lifelog based context information in real time is the most important consideration. We propose the interactive middleware architecture for lifelog based context awareness in distributed and ubiquitous environments. Conventional middleware to support ubiquitous environments stores and manages the situational information and service content acquired by centralized storage or a DBMS. Centralized situational information and service content management may impede the autonomy of mobile nodes and the interoperation between different middle software. The proposed method designs a system that can distribute and manage situational information in mobile nodes using mobile devices in distributed and ubiquitous environments and share the service content between interactive middleware through publication. The application system designed in this study was used in a scenario providing situational perception based mobile service and proved to be useful.     

Enhancing virtual environment spatial awareness training and transfer through tactile and vestibular cues

Haptic interaction has been successfully incorporated into a variety of virtual environment (VE) systems, yet designing multimodal VE training systems remains challenging as each cue incorporated during training should maximise learning and training transfer. This study examined the impact of incorporating two independent, spatialised tactile cues and vestibular cues into a military VE training environment with the goal of empirically examining whether such cues could enhance performance within the training environment and also that knowledge and skills gained during training could transfer to another environment. The results showed that tactile cues enhanced spatial awareness and performance during both repeated training and within a transfer environment, yet there were costs associated when two independent tactile cues were presented during training. In addition, results suggest that spatial awareness benefits from a tactile point indicator may be impacted by vestibular cues, as performance benefits were seen when tactile cues were paired with head tracking. To fully realise training potential, it is essential to determine how best to leverage multimodal capacity of VE training systems by identifying how multimodal training cues may advance knowledge, skills and attitudes of trainees. Results from this study provide design guidelines for incorporating tactile cues in VE training environments to enhance spatial awareness.     

Piles across space: Breaking the real-estate barrier on small-display devices

We describe an implementation that has users ‘flick’ notes, images, audio, and video files onto virtual, imaginary piles beyond the display of small-screen devices. Multiple sets of piles can be maintained in persistent workspaces. Two user studies yielded the following: Participants developed mental schemes to remember virtual pile locations, and they successfully reinstated pile locations after several days, while situated in varying environments. Alignment of visual cues on screen with surrounding physical cues in situ accelerated a sorting task when compared to other, non-aligned visual cues. The latter, however, yielded better long-term retention.     

Design and evaluation of virtual environments mechanisms to support remote collaboration on complex process diagrams

Many organizational analysis tasks are solved by collaborating teams. In technology-mediated collaborations, enabling relevant visual cues is a core issue with existing technology. We explore whether avatars can provide relevant cues in collaborative virtual environments. To do so, we develop a research prototype for a collaborative virtual environment that utilises avatars to improve workspace awareness in collaborative tasks. We test this solution through two studies, qualitative and quantitative, in which participants have to collaborate to jointly validate and correct a diagrammatic model of operational procedures.Our evaluations provide both positive and negative results about the proposed prototype. Collaboration patterns changed and became significantly easier, but, task performance was only slightly improved. Together, these findings inform both the ongoing development of collaborative 3D virtual environments and the role of technology-mediated collaboration for validating and fixing models of processes.     

Influence of contextual objects on spatial interactions and viewpoints sharing in virtual environments

Collaborative virtual environments (CVEs) are 3D spaces in which users share virtual objects, communicate, and work together. To collaborate efficiently, users must develop a common representation of their shared virtual space. In this work, we investigated spatial communication in virtual environments. In order to perform an object co-manipulation task, the users must be able to communicate and exchange spatial information, such as object position, in a virtual environment. We conducted an experiment in which we manipulated the contents of the shared virtual space to understand how users verbally construct a common spatial representation of their environment. Forty-four students participated in the experiment to assess the influence of contextual objects on spatial communication and sharing of viewpoints. The participants were asked to perform in dyads an object co-manipulation task. The results show that the presence of a contextual object such as fixed and lateralized visual landmarks in the virtual environment positively influences the way male operators collaborate to perform this task. These results allow us to provide some design recommendations for CVEs for object manipulation tasks.     

Spatial conferencing using a wearable computer

Wearable computers provide constant access to computing and communications resources; however, there are many unanswered questions as to how this computing power can be used to enhance communication. We describe a wearable augmented reality communication space that uses spatialised 3D graphics and audio cues to aid communication. The user is surrounded by virtual avatars of the remote collaborators that they can interact with using natural head and body motions. The use of spatial cues means that the conferencing space can potentially support dozens of simultaneous users. We report on two experiments that show users can understand speakers better with spatial rather than non-spatial audio, and that minimal visual cues may be sufficient to distinguish between speakers. Additional informal user studies with real conference participants suggest that wearable communication spaces may offer significant advantages over traditional communication devices.     

The Role of 3-D Sound in Human Reaction and Performance in Augmented Reality Environments

Three-dimensional sound's effectiveness in virtual reality (VR) environments has been widely studied. However, due to the big differences between VR and augmented reality (AR) systems in registration, calibration, perceptual difference of immersiveness, navigation, and localization, it is important to develop new approaches to seamlessly register virtual 3-D sound in AR environments and conduct studies on 3-D sound's effectiveness in AR context. In this paper, we design two experimental AR environments to study the effectiveness of 3-D sound both quantitatively and qualitatively. Two different tracking methods are applied to retrieve the 3-D position of virtual sound sources in each experiment. We examine the impacts of 3-D sound on improving depth perception and shortening task completion time. We also investigate its impacts on immersive and realistic perception, different spatial objects identification, and subjective feeling of "human presence and collaboration". Our studies show that applying 3-D sound is an effective way to complement visual AR environments. It helps depth perception and task performance, and facilitates collaborations between users. Moreover, it enables a more realistic environment and more immersive feeling of being inside the AR environment by both visual and auditory means. In order to make full use of the intensity cues provided by 3-D sound, a process to scale the intensity difference of 3-D sound at different depths is designed to cater small AR environments. The user study results show that the scaled 3-D sound significantly increases the accuracy of depth judgments and shortens the searching task completion time. This method provides a necessary foundation for implementing 3-D sound in small AR environments. Our user study results also show that this process does not degrade the intuitiveness and realism of an augmented audio reality environment     

基于可穿戴计算的车间作业辅助应用研究

将可穿戴计算技术引入制造业设备维护应用领域,设计可穿戴作业辅助应用系统。首先结合无线传感、穿戴传感等构建车间模拟实验环境,设计可穿戴计算机系统原型;进而针对环境信息实体的上下文特征,结合领域知识本体、模糊决策树等方法,建立了一种新的上下文驱动的可穿戴作业辅助应用构建模式;最后给出了应用实例及分析。     

The design-by-adaptation approach to universal access: learning from videogame technology

This paper proposes an alternative approach to the design of universally accessible interfaces to that provided by formal design frameworks applied ab initio to the development of new software. This approach, design-by-adaptation, involves the transfer of interface technology and/or design principles from one application domain to another, in situations where the recipient domain is similar to the host domain in terms of modelled systems, tasks and users. Using the example of interaction in 3D virtual environments, the paper explores how principles underlying the design of videogame interfaces may be applied to a broad family of visualization and analysis software which handles geographical data (virtual geographic environments, or VGEs). One of the motivations behind the current study is that VGE technology lags some way behind videogame technology in the modelling of 3D environments, and has a less-developed track record in providing the variety of interaction methods needed to undertake varied tasks in 3D virtual worlds by users with varied levels of experience. The current analysis extracted a set of interaction principles from videogames which were used to devise a set of 3D task interfaces that have been implemented in a prototype VGE for formal evaluation.     

Spectrum-based synthesis of vibrotactile stimuli: active footstep display for crinkle of fragile structures

When a human crinkles or scrunches a fragile object, for which the yield force is very small that it is hardly perceived, they identify the material of the object based on tactile stimuli delivered to the skin. In addition, humans are able to recognize materials even when they are crinkled at different speeds. In order to realize these human recognition features of the crinkle of a fragile object, we develop a vibrotactile synthesis method. This method synthesizes the vibrotactile acceleration stimuli in response to a crinkle speed based on the preliminarily measured acceleration spectra. Using this method, we develop an active footstep display that presents a virtual crinkle of fragile structures made of different materials to its users. Experimental participants could identify three of the four types of virtual structure materials at rates significantly higher than the chance level. The four materials were copy and typing paper, aluminum foil, and polypropylene film. Furthermore, the trends of answer ratios exhibit good correspondence with those for the real cylindrical fragile objects. We conclude that the developed method is valid for the virtual crinkle of fragile structures and will enhance the validity of virtual reality systems, such as a virtual walkthrough system.     

Evaluation of virtual fixtures for a robot programming by demonstration interface

We investigate the effectiveness of several types of virtual fixtures in a robot programming by demonstration interface. We show that while all types of virtual fixtures examined yield a significant reduction in the number of errors in tight tolerance peg-in-hole tasks, color and sound fixtures generally outperform a tactile fixture in terms of both execution time of successful trials and error rate. We have found also that when users perceive that the task is very difficult but the system is providing some help by means of a virtual fixture, they tend to spend more time trying to achieve a successful task execution. Thus, for difficult tasks the benefits of virtual fixturing are better reflected in a reduction of the error rate than in a decreased execution time. We conjecture that these trends are related to the limitations of currently available interfaces for human-robot interaction through virtual environments and to the different strategies adopted by the users to cope with such limitations in high-accuracy tasks.     

动态网络环境下的透明服务组合

动态网络环境(如网格、Ad Hoc Wireless Network等)使面向服务的应用面临许多新问题.如何使用户能够透明、动态地按需使用各种服务是其中的研究热点之一.文中提出了一种虚拟服务模型,在此基础上定义了虚拟服务的组合运算,给出了服务动态查找的算法.此外,文中还介绍了支持虚拟服务透明组合的P2P服务组合原型系统,并在实验基础上对虚拟服务及其支撑引擎进行了客观评价.实验表明,该文所述虚拟服务及支撑引擎能使面向服务应用较好地适应动态网络环境下的服务变化,使用户能够透明地按需使用各种资源.     

Project Aura: toward distraction-free pervasive computing

The most precious resource in a computer system is no longer its processor, memory, disk, or network, but rather human attention. Aura aims to minimize distractions on a user's attention, creating an environment that adapts to the user's context and needs. Aura is specifically intended for pervasive computing environments involving wireless communication, wearable or handheld computers, and smart spaces. Human attention is an especially scarce resource in such environments, because the user is often preoccupied with walking, driving, or other real-world interactions. In addition, mobile computing poses difficult challenges such as intermittent and variable-bandwidth connectivity, concern for battery life, and the client resource constraints that weight and size considerations impose. To accomplish its ambitious goals, research in Aura spans every system level: from the hardware, through the operating system, to applications and end users. Underlying this diversity of concerns, Aura applies two broad concepts. First, it uses proactivity, which is a system layer's ability to anticipate requests from a higher layer. In today's systems, each layer merely reacts to the layer above it. Second, Aura is self-tuning: layers adapt by observing the demands made on them and adjusting their performance and resource usage characteristics accordingly. Currently, system-layer behavior is relatively static. Both of these techniques will help lower demand for human attention.     

Dedicated workspaces: Faster resumption times and reduced cognitive load in sequential multitasking

Studies show that virtual desktops have become a widespread approach to window management within desktop environments. However, despite their success, there is no experimental evidence of their effect on multitasking. In this paper, we present an experimental study incorporating 16 participants in which a traditional Windows 7 environment is compared to one augmented by virtual desktops. Within the experimental condition, each virtual desktop acts as a dedicated workspace devoted to an independent goal-oriented task, as opposed to the control condition where only one single workspace is available to perform the same tasks. Results show that adopting virtual desktops as dedicated workspaces allows for faster task resumption (10 s faster on average) and reduced cognitive load during sequential multitasking. Within our experiment the majority of users already benefited from using dedicated workspaces after three switches to a previously suspended task, as the time lost on setting up workspaces was compensated for by faster subsequent task resumption. These results provide a strong argument for supporting goal-oriented dedicated workspaces within desktop environments.