A virtual reality keyboard with realistic haptic feedback in a fully immersive virtual environment

This study presents a 3D virtual reality (VR) keyboard system with realistic haptic feedback. The system uses two five-fingered data gloves to track finger positions and postures, uses micro-speakers to create simulated vibrations, and uses a head-mounted display (HMD) for 3D display. When users press a virtual key in the VR environment, the system can provide realistic simulated key click haptic feedback to users. The results of this study show that the advantages of the haptic VR keyboard are that users can use it when wearing HMDs (users do not need to remove HMDs to use the VR keyboard), the haptic VR keyboard can pop-up display at any location in the VR environments (users do not need to go to a specific location to use an actual physical keyboard), and the haptic VR keyboard can be used to provide realistic key click haptic feedback (which other studies have shown enhances user performance). The results also show that the haptic VR keyboard system can be used to create complex vibrations that simulate measured vibrations from a real keyboard and enhance keyboard interaction in a fully immersive VR environment.     

Walk-up VR: virtual reality beyond projection screens

Today, VR research and development efforts often focus on the continual innovation of interaction styles and metaphors for virtual environments (VEs). New tools and interaction devices aim to increase the immersive experience rather than support seamless integration with real work scenarios. Even though users may soon perceive odours within VEs, real, task-oriented interaction within these environments will continue to lag behind. Combining these efforts can result in new user interfaces that reduce the cumbersome barriers prevalent in VEs today, finally unleashing the latent impact of this technology in everyday life. Implementing this vision requires an interdisciplinary and applied approach to integrate VR into the workplace. Mixed-reality display capabilities, useful multimodal interaction and perceptual, intuitive interfaces are major components of such an application-oriented and human-centered approach, for which we coined the term walk-up VR. The paper discusses augmented and virtual reality as contributing technologies     

An experimental investigation of menu selection for immersive virtual environments: fixed versus handheld menus

Virtual Reality - With the development of consumer-grade virtual reality (VR) systems, the interface and interaction design for immersive virtual environments have become a critical issue for VR...     

Virtual hell: a trip through the flames

In immersive virtual environments, our eyes and ears are usually the most important channels of perception. However, you might expect that their creators would include other senses in VEs so as to represent the largest range of stimuli. After all, humans are creatures of many senses and rely upon all of them in day-to-day living. In crafting virtual worlds for exploration, VE developers usually aim for as much realism as possible given the constraints of equipment and cost. Unfortunately, force and tactile feedback still have a long way to go, although tactile feedback models can simulate local geometry, texture reproduction, and heat flux and temperature for replicating collision detection. To investigate the latter options, I developed a complete thermal feedback system, the VR Thermal Kit, which includes different components to model, control, and physically generate stimuli on the operator's skin (usually the hand and exposed parts of the body). The results of my study concentrate on interaction paradigms and haptic rendering. They also enhance the level of immersion     

Improving virtual reality navigation tasks using a haptic vest and upper body tracking

Haptic feedback is an important component of immersive virtual reality (VR) applications that is often suggested to complement visual information through the sense of touch. This paper investigates the use of a haptic vest in navigation tasks. The haptic vest produces a repulsive vibrotactile feedback from nearby static virtual obstacles that augments the user spatial awareness. The tasks require the user to perform complex movements in a 3D cluttered virtual environment, like avoiding obstacles while walking backwards and pulling a virtual object. The experimental setup consists of a room-scale environment. Our approach is the first study where a haptic vest is tracked in real time using a motion capture device so that proximity-based haptic feedback can be conveyed according to the actual movement of the upper body of the user.User study experiments have been conducted with and without haptic feedback in virtual environments involving both normal and limited visibility conditions. A quantitative evaluation was carried out by measuring task completion time and error (collision) rate. Multiple haptic rendering techniques have also been tested. Results show that under limited visibility conditions proximity-based haptic feedback generated by a wearable haptic vest can significantly reduce the number of collisions with obstacles in the virtual environment.     

Geometric calibration of head-mounted displays and its effects on distance estimation

Head-mounted displays (HMDs) allow users to observe virtual environments (VEs) from an egocentric perspective. However, several experiments have provided evidence that egocentric distances are perceived as compressed in VEs relative to the real world. Recent experiments suggest that the virtual view frustum set for rendering the VE has an essential impact on the user's estimation of distances. In this article we analyze if distance estimation can be improved by calibrating the view frustum for a given HMD and user. Unfortunately, in an immersive virtual reality (VR) environment, a full per user calibration is not trivial and manual per user adjustment often leads to mini- or magnification of the scene. Therefore, we propose a novel per user calibration approach with optical see-through displays commonly used in augmented reality (AR). This calibration takes advantage of a geometric scheme based on 2D point - 3D line correspondences, which can be used intuitively by inexperienced users and requires less than a minute to complete. The required user interaction is based on taking aim at a distant target marker with a close marker, which ensures non-planar measurements covering a large area of the interaction space while also reducing the number of required measurements to five. We found the tendency that a calibrated view frustum reduced the average distance underestimation of users in an immersive VR environment, but even the correctly calibrated view frustum could not entirely compensate for the distance underestimation effects.     

The virtual playground: an educational virtual reality environment for evaluating interactivity and conceptual learning

The research presented in this paper aims at investigating user interaction in immersive virtual learning environments, focusing on the role and the effect of interactivity on conceptual learning. The goal has been to examine if the learning of young users improves through interacting in (i.e. exploring, reacting to, and acting upon) an immersive virtual environment (VE) compared to non-interactive or non-immersive environments. Empirical work was carried out with more than 55 primary school students between the ages of 8 and 12, in different between-group experiments: an exploratory study, a pilot study, and a large-scale experiment. The latter was conducted in a virtual environment designed to simulate a playground. In this “Virtual Playground,” each participant was asked to complete a set of tasks designed to address arithmetical “fractions” problems. Three different conditions, two experimental virtual reality (VR) conditions and a non-VR condition, that varied the levels of activity and interactivity, were designed to evaluate how children accomplish the various tasks. Pre-tests, post-tests, interviews, video, audio, and log files were collected for each participant, and analysed both quantitatively and qualitatively. This paper presents a selection of case studies extracted from the qualitative analysis, which illustrate the variety of approaches taken by children in the VEs in response to visual cues and system feedback. Results suggest that the fully interactive VE aided children in problem solving but did not provide a strong evidence of conceptual change as expected; rather, it was the passive VR environment, where activity was guided by a virtual robot, that seemed to support student reflection and recall, leading to indications of conceptual change.     

V-Pong: an immersive table tennis simulation

Most applications for immersive virtual environments (VEs) allow slow-or medium-speed user interaction. Examples of this kind of interaction include changing an object's position, triggering an action, or setting a control parameter. To broaden the application range for VR systems, we need to integrate technologies that allow for faster VE-user movements. This raises the question, What response times can we achieve with VR systems built from standard recent hardware components? To answer this question, we created a table tennis simulation as this game involves fast user movements and has moderate space requirements. In this article we report on the realization of our immersive table tennis simulation, V-Pong.     

支持力反馈的沉浸式物理学习环境的构建

随着虚拟现实(VR)技术的发展,沉浸式学习环境在教育教学领域应用前景日趋广阔,如物理实验仿真教学.然而,现有虚拟学习环境大多只能提供视觉与听觉的交互,不支持力触觉交互,存在弊端.项目组将力反馈技术应用于虚拟学习环境,描述一个支持力反馈的沉浸式物理学习环境的总体框架和开发流程.使用Touch力反馈设备,借助Unity3D...     

The influence of different sensory cues as selection feedback and co-location in presence and task performance

For some applications based on virtual reality technology, presence and task performance are important factors to validate the experience. Different approaches have been adopted to analyse the extent to which certain aspects of a computer-generated environment may enhance these factors, but mainly in 2D graphical user interfaces. This study explores the influence of different sensory modalities on performance and the sense of presence experienced within a 3D environment. In particular, we have evaluated visual, auditory and active haptic feedback for indicating selection of virtual objects. The effect of spatial alignment between proprioceptive and visual workspaces (co-location) has also been analysed. An experiment has been made to evaluate the influence of these factors in a controlled 3D environment based on a virtual version of the Simon game. The main conclusions obtained indicate that co-location must be considered in order to determine the sensory needs during interaction within a virtual environment. This study also provides further evidence that the haptic sensory modality influences presence to a higher extent, and that auditory cues can reduce selection times. Conclusions obtained provide initial guidelines that will help designers to set out better selection techniques for more complex environments, such as training simulators based on VR technology, by highlighting different optimal configurations of sensory feedback.     

虚拟现实及其应用进展

虚拟现实是一种多通道的新型人机交互接口,它提供了具有沉浸感的智能虚拟环境并允许用户和该环境进行交互.计算机技术、交互技术和人工智能等相关技术的快速发展促进了虚拟现实技术的巨大进步,以此为基础的实际应用也得到了很快的发展和提高.描述了虚拟现实的涵义及其本质特征,归纳了适合使用虚拟现实技术的场合,并介绍了虚拟现实技术在教育、医疗、科学可视化和工程制造等方面的最新应用,最后对虚拟现实未来的应用做出了展望.     

Immersive Multiplayer Games With Tangible and Physical Interaction

In this paper, we present a new immersive multiplayer game system developed for two different environments, namely, virtual reality (VR) and augmented reality (AR). To evaluate our system, we developed three game applications-a first-person-shooter game (for VR and AR environments, respectively) and a sword game (for the AR environment). Our immersive system provides an intuitive way for users to interact with the VR or AR world by physically moving around the real world and aiming freely with tangible objects. This encourages physical interaction between players as they compete or collaborate with other players. Evaluation of our system consists of users' subjective opinions and their objective performances. Our design principles and evaluation results can be applied to similar immersive game applications based on AR/VR.     

Haptic Direct-Drive Robot Control Scheme in Virtual Reality

This paper explores the use of a 2-D (Direct-Drive Arm) manipulator for mechanism design applications based on virtual reality (VR). This article reviews the system include a user interface, a simulator, and a robot control scheme. The user interface is a combination of a virtual clay environment and human arm dynamics via robot effector handler. The model of the VR system is built based on a haptic interface device behavior that enables the operator to feel the actual force feedback from the virtual environment just as s/he would from the real environment. A primary stabilizing controller is used to develop a haptic interface device where realistic simulations of the dynamic interaction forces between a human operator and the simulated virtual object/mechanism are required. The stability and performance of the system are studied and analyzed based on the Nyquist stability criterion. Experiments on cutting virtual clay are used to validate the theoretical developments. It was shown that the experimental and theoretical results are in good agreement and that the designed controller is robust to constrained/unconstrained environment.     

VRMosaic: Web access from within a virtual environment

For the foreseeable future, users of virtual reality systems will likely spend more time in the “real” environment than in a virtual environment. In the “real” environment, users access much of their data using flat screen applications, which include tools such as authoring and analysis programs and even hyperlinked browsers. We believe that users will find VEs more appealing if they can import their flat screen applications. We also believe that the World Wide Web infrastructure, and supporting tools such as NCSA Mosaic, Netscape's Navigator, and Sun's HotJava, have become a de facto standard both for making data available and allowing limited interaction with that data. These beliefs motivated us to develop a technology for porting flat screen applications based on the 3.1 version of the InterViews toolkit into VR. NCSA Mosaic provided us with a real application that would: (1) test our infrastructure effectively; and (2) provide a compelling application example. We used our “2D interface in VR” infrastructure to port Mosaic into RealEyes, our VR system. Dubbed “VRMosaic”, this application lets users familiar with Mosaic access the Web from within an immersive VE. But VRMosaic is not just an embedded version of NCSA Mosaic-it also allows for VR specific features such as navigation within the VE     

Cooking up an interactive olfactory game display.

It's long been possible to give users outside an actual environment that environment's visual and auditory information and thus contribute to establishing presence. However, we've yet to establish much presence when users require olfactory information - such as in environments focused on foods, flowers, perfumes, or, in some cases, more offensive smells. Recently, several VR researchers have become interested in olfaction and olfactory displays that present smells in virtual environments (VEs). In this article, we describe our interactive olfactory display. One of our development goals is to confirm the assumption that users' interactions with the system increases presence. Thus, we used our interactive olfactory display to develop a cooking game in collaboration with electronic engineers and artists.     

A large haptic device for aircraft engine maintainability

The virtual reality for maintainability (Revima) VR system supports maintainability simulation in aeronautics. Within this project we have developed and integrated a haptic device, the large haptic interface for aeronautic maintainability (LHIfAM). We use this device to track hand movements and provide force feedback within the large geometric models that describe aircraft engines. The user movements are the same as those that occur when testing physical mock-ups. An integrated haptic device and VR system for testing aircraft engines reduces development costs and avoids the necessity of physical mock-ups formaintainability.     

Simultaneous remote haptic collaboration for assembling tasks

Stand-alone virtual environments (VEs) using haptic devices have proved useful for assembly/disassembly simulation of mechanical components. Nowadays, collaborative haptic virtual environments (CHVEs) are also emerging. A new peer-to-peer collaborative haptic assembly simulator (CHAS) has been developed whereby two users can simultaneously carry out assembly tasks using haptic devices. Two major challenges have been addressed: virtual scene synchronization (consistency) and the provision of a reliable and effective haptic feedback. A consistency-maintenance scheme has been designed to solve the challenge of achieving consistency. Results show that consistency is guaranteed. Furthermore, a force-smoothing algorithm has been developed which is shown to improve the quality of force feedback under adverse network conditions. A range of laboratory experiments and several real trials between Labein (Spain) and Queen’s University Belfast (Northern Ireland) have verified that CHAS can provide an adequate haptic interaction when both users perform remote assemblies (assembly of one user’s object with an object grasped by the other user). Moreover, when collisions between grasped objects occur (dependent collisions), the haptic feedback usually provides satisfactory haptic perception. Based on a qualitative study, it is shown that the haptic feedback obtained during remote assemblies with dependent collisions can continue to improve the sense of co-presence between users with regard to only visual feedback.     

Navigation in desktop virtual environments: an evaluation and recommendations for supporting usability

Virtual reality (VR) can provide useful tools for a variety of applications. However, for these tools to be effective, they must be easy to use. In virtual environments (VEs), usability is impaired by poorly designed navigation systems. Insufficient realism and missing physiological orientation and motion cues impair spatial learning in desktop VEs. Capabilities for navigation in a VE are far more varied than in reality; so much greater flexibility can be offered, but designing VEs with too many options can overwhelm users. To assist designers in building effective, usable navigation systems for VEs, navigation techniques must be evaluated to identify which features actually support users in accomplishing their tasks and which features create unnecessary problems.This study evaluates navigation in two different VEs to develop recommendations for the design of navigation systems in desktop VEs. The study consists of an objective assessment of navigation control dynamics, a guideline-based evaluation and a review of data collected during two experimental studies. The findings indicate that real-world constraints, specialised navigation techniques and feedback regarding location and direction of travel are needed to support navigation in desktop VEs.     

多模态人机交互综述

多模态人机交互旨在利用语音、图像、文本、眼动和触觉等多模态信息进行人与计算机之间的信息交换。在生理心理评估、办公教育、军事仿真和医疗康复等领域具有十分广阔的应用前景。本文系统地综述了多模态人机交互的发展现状和新兴方向,深入梳理了大数据可视化交互、基于声场感知的交互、混合现实实物交互、可穿戴交互和人机对话交互的研究进展以及国内外研究进展比较。本文认为拓展新的交互方式、设计高效的各模态交互组合、构建小型化交互设备、跨设备分布式交互、提升开放环境下交互算法的鲁棒性等是多模态人机交互的未来研究趋势。