Exploiting perceptual limitations and illusions to support walking through virtual environments in confined physical spaces

Head-mounted displays (HMDs) allow users to immerse in a virtual environment (VE) in which the user’s viewpoint can be changed according to the tracked movements in real space. Because the size of the virtual world often differs from the size of the tracked lab space, a straightforward implementation of omni-directional and unlimited walking is not generally possible. In this article we review and discuss a set of techniques that use known perceptual limitations and illusions to support seemingly natural walking through a large virtual environment in a confined lab space. The concept behind these techniques is called redirected walking. With redirected walking, users are guided unnoticeably on a physical path that differs from the path the user perceives in the virtual world by manipulating the transformations from real to virtual movements. For example, virtually rotating the view in the HMD to one side with every step causes the user to unknowingly compensate by walking a circular arc in the opposite direction, while having the illusion of walking on a straight trajectory. We describe a number of perceptual illusions that exploit perceptual limitations of motion detectors to manipulate the user’s perception of the speed and direction of his motion. We describe how gains of locomotor speed, rotation, and curvature can gradually alter the physical trajectory without the users observing any discrepancy, and discuss studies that investigated perceptual thresholds for these manipulations. We discuss the potential of self-motion illusions to shift or widen the applicable ranges for gain manipulations and to compensate for over- or underestimations of speed or travel distance in VEs. Finally, we identify a number of key issues for future research on this topic.     

A Comprehensive Review of Redirected Walking Techniques: Taxonomy,Methods, and Future Directions

Virtual reality (VR) allows users to explore and experience a computer-simulated virtual environment so that VR users can be immersed in a totally artificial virtual world and interact with arbitrary virtual objects. However, the limited physical tracking space usually restricts the exploration of large virtual spaces, and VR users have to use special locomotion techniques to move from one location to another. Among these techniques, redirected walking (RDW) is one of the most natural locomotion techniques to solve the problem based on near-natural walking experiences. The core idea of the RDW technique is to imperceptibly guide users on virtual paths, which might vary from the paths they physically walk in the real world. In a similar way, some RDW algorithms imperceptibly change the structure and layout of the virtual environment such that the virtual environment fits into the tracking space. In this survey, we first present a taxonomy of existing RDW work. Based on this taxonomy, we compare and analyze both contributions and shortcomings of the existing methods in detail, and find view manipulation methods offer satisfactory visual effect but the experience can be interrupted when users reach the physical boundaries, while virtual environment manipulation methods can provide users with consistent movement but have limited application scenarios. Finally, we discuss possible future research directions, indicating combining artificial intelligence with this area will be effective and intriguing.     

Velocity-dependent dynamic curvature gain for redirected walking

Redirected walking techniques allow people to walk in a larger virtual space than the physical extents of the laboratory. We describe two experiments conducted to investigate human sensitivity to walking on a curved path and to validate a new redirected walking technique. In a psychophysical experiment, we found that sensitivity to walking on a curved path was significantly lower for slower walking speeds (radius of 10 m versus 22 m). In an applied study, we investigated the influence of a velocity-dependent dynamic gain controller and an avatar controller on the average distance that participants were able to freely walk before needing to be reoriented. The mean walked distance was significantly greater in the dynamic gain controller condition, as compared to the static controller (22 m versus 15 m). Our results demonstrate that perceptually motivated dynamic redirected walking techniques, in combination with reorientation techniques, allow for unaided exploration of a large virtual city model.     

Redirecting walking and driving for natural navigation in immersive virtual environments

Walking is the most natural form of locomotion for humans, and real walking interfaces have demonstrated their benefits for several navigation tasks. With recently proposed redirection techniques it becomes possible to overcome space limitations as imposed by tracking sensors or laboratory setups, and, theoretically, it is now possible to walk through arbitrarily large virtual environments. However, walking as sole locomotion technique has drawbacks, in particular, for long distances, such that even in the real world we tend to support walking with passive or active transportation for longer-distance travel. In this article we show that concepts from the field of redirected walking can be applied to movements with transportation devices. We conducted psychophysical experiments to determine perceptual detection thresholds for redirected driving, and set these in relation to results from redirected walking. We show that redirected walking-and-driving approaches can easily be realized in immersive virtual reality laboratories, e. g., with electric wheelchairs, and show that such systems can combine advantages of real walking in confined spaces with benefits of using vehicle-based self-motion for longer-distance travel.     

虚拟现实大空间下多虚拟目标被动触觉交互方法

针对虚拟现实（VR）大空间下为重定向行走的用户提供被动触觉时存在的虚实交互目标无法一一对应的问题,提出了一种用两个物理代理作为触觉代理为多个虚拟目标提供触觉反馈的方法,以在基于人工势场（APF）的重定向行走过程中,交替地满足用户被动触觉的需求。针对重定向行走算法本身以及标定不精确等原因造成的虚实不对齐的问题,对虚拟目标的位置及朝向进行设计并且在交互阶段引入触觉重定向。仿真实验表明对虚拟目标位置和朝向的设计可以大幅降低对齐误差;而用户实验结果证明触觉重定向的引入进一步提升了交互准确性,且能为用户带来更丰富、更具沉浸感的体验。     

Where are you going? Using human locomotion models for target estimation

To explore virtual environments that are larger than the available physical tracking space by real walking, it is necessary to use so-called redirected walking. Redirection techniques allow the user to explore an unlimited virtual environment in a limited tracking space by introducing a small mismatch between a user’s real and virtual movement, thus preventing the user from colliding with the physical walls of the tracking space. Steering algorithms are used to select the most suitable redirection technique at any given time, depending on the geometry of the real and virtual environment. Together with prediction of a user’s future walking path, these algorithms select the best redirection strategy by an optimal control scheme. In this paper, a new approach for the prediction of a person’s locomotion target is presented. We use various models of human locomotion together with a set of possible targets to create a set of expected paths. These paths are then compared to the real path the user already traveled to calculate the probability of a certain target being the one the user is heading for. A new approach for comparing paths with each other is introduced and is compared to three others. For describing the human’s path to a given target, four different models are used and compared. To gather data for the comparison of the models against the real path, a user study was conducted. Based on the results of the user study, the paper concludes with a discussion on the prediction performance of the different approaches.     

Afforded actions as a behavioral assessment of physical presence in virtual environments

A particular affordance was used as a potential candidate for behavioral assessment of physical presence in virtual environments. The subjects’ task was to walk through a virtual aperture of variable widths. In the case of presence, the subjects’ body orientation, while walking, was hypothesized to be adapted to the width of the aperture and to their own shoulder width. Results show that most subjects adapted their behavior to both their body architecture and the virtual width constraints. These subjects exhibited a behavioral transition from frontal walking to body rotation while walking through broad to narrow apertures. The same behavioral transition has already been documented in real environments (Warren and Whang in J Exp Psychol Human Percept Perform 13(3):371–383, 1987). This behavioral adjustment is thus assumed to be an objective indication of presence. Beyond these results, the present study suggests that every afforded action could be a potential tool for sensorimotor assessment of physical presence. Parts of this research have been first presented at the 11th Annual International Workshop of Presence, Padova, Italy, 16-18 October 2008.     

The Torus Treadmill: realizing locomotion in VEs

Locomotion in virtual environments (VEs) remains one of the major problems in current virtual reality research. The most intuitive way to move about the real world is to travel on foot. People often feel a better sense of distance or direction while walking than while riding in a vehicle. This article discusses the development of a locomotion device that provides a sense of walking. In terms of natural interaction, the physical exertion of walking proves essential to locomotion. The research of my colleagues and I aims to give users a sense of walking while their position remains localized in the physical world. We've developed several prototypes of interface devices for walking. From the results of our research, we concluded that an infinite surface would offer an ideal means for giving people a sense of walking. Our device, called the Torus Treadmill, uses a torus-shaped surface to realize the locomotion interface. The surface employs 12 sets of treadmills connected side-by-side and driven in a perpendicular direction. These treadmills generate an infinite surface. We measured the motion of the users' feet with magnetic sensors. The floor moves in the opposite direction of the walker, canceling the motion of each step. The walker's position remains localized in the real world by this computer-controlled motion of the floor. The walker can freely change direction. An image of the virtual space appears in a head-mounted display corresponding to the walker's virtual position     

User-centered redirected walking and resetting with virtual feelers

Virtual Reality - With redirected walking (RDW), the exploration of an infinite virtual world with a small physical space has been enabled. This paper proposes a user-centered RDW (UC-RDW) and...     

Experiencing Nature: Embodying Animals in Immersive Virtual Environments Increases Inclusion of Nature in Self and Involvement With Nature

Immersive virtual environments (IVEs) produce simulations that mimic unmediated sensory experiences. 3 experiments (N = 228) tested how different modalities increase environmental involvement by allowing users to inhabit the body of animals in IVEs or watch the experience on video. Embodying sensory‐rich experiences of animals in IVEs led to greater feeling of embodiment, perception of being present in the virtual world, and interconnection between the self and nature compared to video. Heightened interconnection with nature elicited greater perceptions of imminence of the environmental risk and involvement with nature, which persisted for 1 week. Although the effect sizes were small to moderate, findings suggest that embodied experiences in IVEs may be an effective tool to promote involvement with environmental issues.     

Cognitive and synthetic behavior of avatars in intelligent virtual environments

In intelligent virtual environments (IVEs), it is a challenging research issue to provide the intelligent virtual actors (or avatars) with the ability of visual perception and rapid response to virtual world events. Modeling an avatar’s cognitive and synthetic behavior appropriately is of paramount important in IVEs. We propose a new cognitive and behavior modeling methodology that integrates two previously developed complementary approaches. We present expression cloning, walking synthetic behavior modeling, and an autonomous agent cognitive model for driving an avatar’s behavior. Facial expressions are generated using our own-developed rule-based state transition system. Facial expressions are further personalized for individuals by expression cloning. An avatar’s walking behavior is modeled using a skeleton model that is implemented by seven-motion sequences and finite state machines (FSMs). We discuss experimental results demonstrating the benefits of our approach.     

Exploring legibility of augmented reality X-ray

Virtual objects can be visualized inside real objects using augmented reality (AR). This visualization is called AR X-ray because it gives the impression of seeing through the real object. In standard AR, virtual information is overlaid on top of the real world. To position a virtual object inside an object, AR X-ray requires partially occluding the virtual object with visually important regions of the real object. In effect, the virtual object becomes less legible compared to when it is completely unoccluded. Legibility is an important consideration for various applications of AR X-ray. In this research, we explored legibility in two implementations of AR X-ray, namely, edge-based and saliency-based. In our first experiment, we explored on the tolerable amounts of occlusion to comfortably distinguish small virtual objects. In our second experiment, we compared edge-based and saliency-based AR X-ray methods when visualizing virtual objects inside various real objects. Moreover, we benchmarked the legibility of these two methods against alpha blending. From our experiments, we observed that users have varied preferences for proper amounts of occlusion cues for both methods. The partial occlusions generated by the edge-based and saliency-based methods need to be adjusted depending on the lighting condition and the texture complexity of the occluding object. In most cases, users identify objects faster with saliency-based AR X-ray than with edge-based AR X-ray. Insights from this research can be directly applied to the development of AR X-ray applications.     

Towards perceptual fidelity: Slant perception in real and interactive virtual environments

An innovative motoric measure of slant based on gait is proposed as the angle between the foot and the walking surface during walking. This work investigates whether the proposed action-based measure is affected by factors such as material and inclination of the walking surface. Experimental studies were conducted in a real environment set-up and in its virtual simulation counterpart evaluating behavioural fidelity and user performance in ecologically-valid simulations. In the real environment, the measure slightly overestimated the inclined path whereas in the virtual environment it slightly underestimated the inclined path. The results imply that the proposed slant measure is modulated by motoric caution. Since the “reality” of the synthetic environment was relatively high, performance results should have revealed the same degree of caution as in the real world, however, that was not the case. People become more cautious when the ground plane was steep, slippery, or virtual.     

Playing in the mud: Virtual worlds are real places

This paper is about how developers will know whether intelligent virtual environments (IVEs) are appropriate for the tasks set to them. There are several important research questions that need to be answered before they can even begin to build IVEs for some of the more promising applications, such as entertainment, education, collaboration on research and development, military, and other training. The main technical point is that every aspect of any IVE involves model, and the modeling needs to be addressed very directly and explicitly. This paper is more for the developers of such systems and the potential customers, than for the users. More is written about what is important to consider as one contemplates, for example, a difficult collaborative task, than about what it will be like to live in these IVEs. The authors believe that IVEs must be expanded to become virtual worlds (VWs), and that the modeling sophistication required for any of the serious applications described above is much more than is currently available in the VWs seen, even the ones with good graphics. Finally, the connection between Virtual Reality and CyberSpace is made, and the authors explain their expectations will be explained for the future of CyberSpace.     

Evaluation of direct manipulation using finger tracking for complex tasks in an immersive cube

A solution for interaction using finger tracking in a cubic immersive virtual reality system (or immersive cube) is presented. Rather than using a traditional wand device, users can manipulate objects with fingers of both hands in a close-to-natural manner for moderately complex, general purpose tasks. Our solution couples finger tracking with a real-time physics engine, combined with a heuristic approach for hand manipulation, which is robust to tracker noise and simulation instabilities. A first study has been performed to evaluate our interface, with tasks involving complex manipulations, such as balancing objects while walking in the cube. The user’s finger-tracked manipulation was compared to manipulation with a 6 degree-of-freedom wand (or flystick), as well as with carrying out the same task in the real world. Users were also asked to perform a free task, allowing us to observe their perceived level of presence in the scene. Our results show that our approach provides a feasible interface for immersive cube environments and is perceived by users as being closer to the real experience compared to the wand. However, the wand outperforms direct manipulation in terms of speed and precision. We conclude with a discussion of the results and implications for further research.     

Interactive 3D content insertion in images for multimedia applications

This article addresses the problem of creating interactive mixed reality applications where virtual objects interact in images of real world scenarios. This is relevant to create games and architectural or space planning applications that interact with visual elements in the images such as walls, floors and empty spaces. These scenarios are intended to be captured by the users with regular cameras or using previously taken photographs. Introducing virtual objects in photographs presents several challenges, such as pose estimation and the creation of a visually correct interaction between virtual objects and the boundaries of the scene. The two main research questions addressed in this article include, the study of the feasibility of creating interactive augmented reality (AR) applications where virtual objects interact in a real world scenario using the image detected high-level features and, also, verifying if untrained users are capable and motivated enough to perform AR initialization steps. The proposed system detects the scene automatically from an image with additional features obtained using basic annotations from the user. This operation is significantly simple to accommodate the needs of non-expert users. The system analyzes one or more photos captured by the user and detects high-level features such as vanishing points, floor and scene orientation. Using these features it will be possible to create mixed and augmented reality applications where the user interactively introduces virtual objects that blend with the picture in real time and respond to the physical environment. To validate the solution several system tests are described and compared using available external image datasets.     

Walking strategies of visually impaired people on trapezoidal- and sinusoidal-section tactile groundsurface indicators

The visual system in walking serves to perceive feedback or feed-forward signals. Therefore, visually impaired persons (VIP) have biased motor control mechanisms. The use of leading indicators (LIs) and long canes helps to improve their walking efficiency. The aims of this study were to compare the walking efficiency of VIP on trapezoidal- and sinusoidal-section LIs using an optoelectronic motion analysis system. VIP displayed a significantly longer stance phase, a shorter swing phase and shorter step and stride lengths when they walked on the sinusoidal LI than when they walked on the trapezoidal LI. Compared with the trapezoidal LI, VIP walking on the sinusoidal LI displayed significantly lower joint ranges of motion. The centre of mass lateral displacement was wider for VIP walking on the sinusoidal LI than on the trapezoidal LI. Some significant differences were also found in sighted persons walking on both LIs. In conclusion, the trapezoidal shape enabled visually impaired subjects to walk more efficiently, whereas the sinusoidal shape caused dynamic balance problems. STATEMENT OF RELEVANCE: These findings suggest that VIP can walk more efficiently, with a lower risk of falls, on trapezoidal-section than on sinusoidal-section LIs. These results should be considered when choosing the most appropriate ground tactile surface indicators for widespread use.     

An insect robot controlled by the emergence of gait patterns

A major problem with walking robots is how to control their walking under unpredictably changing environments. Most walking robots proposed to date can walk in limited environments in which gait patterns are kinematically but not dynamically determined in advance. This means that such robots cannot walk and adapt to changes in the world, while animals can walk flexibly and efficiently in the real world. It has been considered that flexibility and efficiency in animals originate in the pattern of emergence of control information. We have already clarified the mechanism of flexible and efficient generation of gait patterns in animals, so we have tried to make an insect robot based on these mechanisms which can walk and adapt to unpredictable changes in the environment. Since these mechanisms are quite new and are also applicable to other artificial systems, we discuss the emergence system as the control mechanism attaining the target state under the constraints of the real world. This work was presented, in part, at the Third International Symposium on Artificial Life and Robotics, Oita, Japan, January 19–21, 1998     

Capture the flag: mixed-reality social gaming with smart phones

The author developed a mobile, mixed-reality version of capture the flag, a popular genre of computer game. Our CTF game is novel in three ways. First, the smart phone is the main interface. Using the smart phone, players physically role-play virtual characters who try to capture enemy flags by traversing different landscapes. This approach creates a direct, real-time linkage between the real and virtual worlds. Second, players can move freely in the real world over a wide area while maintaining seamless real-time networked contact with other players in both the real and virtual worlds. Third, CTF explores novel tangible aspects of human physical movement and perception, both in the real-world playing environment and in interaction with the virtual world.