A framework to design 3D interaction assistance in constraints-based virtual environments

The equilibrium of complex systems often depends on a set of constraints. Thus, credible virtual reality modeling of these systems must respect these constraints, in particular for 3D interactions. In this paper, we propose a generic framework for designing assistance to 3D user interaction in constraints-based virtual environment that associates constraints, interaction tasks and assistance tools, such as virtual fixtures (VFs). This framework is applied to design assistance tools for molecular biology analysis. Evaluation shows that VF designed using our framework improve effectiveness of the manipulation task.     

Human knowledge acquisition from 3D interaction in virtual environments

Recently,virtual environment (VE) based design and planning,which is a kind of interaction intensive computing,has shown great potential.Most users of such systems are specialists or technicians and their 3D interactions with the system embed a great deal of knowledge and skills.Thus,how to integrate human knowledge and skills into VE is a great challenge to researchers in the human computer interaction field.This paper proposes a method for acquiring user knowledge from VE.The main ideas and work include,abstracting the interactive process and formalizing the interactive semantics,as well as providing a semantic model of an interactive information repository,from which user interactive processes can be retrieved and all kinds of application logics can be established.A virtual assembly planning system is introduced as an example of a practical application of this method.Experiments show that the related models can well capture user knowledge and retrieve the interaction process.     

Towards user centred design (UCD) in architecture based on immersive virtual environments

This paper describes a generic concept of how to combine the experience of user centred design (UCD) in the field of Human Computer Interaction (HCI) with the traditional approach of participatory design (PD) in an architectural design process. Even if some basic requirements of this generic method are not available yet, this paper will also describe an approach, which enables planners even now to involve end users by using virtual environments (VE) as immersive and spatial prototype. It will be described and illustrated by the way of example using the building project Centre of Virtual Engineering of the Fraunhofer Institute for Industrial Engineering (IAO) in Stuttgart. It demonstrates that the transfer of the UCD approach to architectural planning combined with the provision of an adequate prototype can make a significant contribution towards an increase in quality and performance in building and construction projects.     

Micro devices assembly using virtual environments

In this paper, the design and development of a physical and virtual environment to support assembly of micron sized devices is discussed. Micro assembly involves handling and assembling of micron sized parts (10⁻⁶ m). As manual assembly of micron-sized parts is difficult and tedious, the design of computer assisted micro-assembly techniques becomes crucial. An Integrated Physical and Virtual Reality (VR) environment has been created to aid in the assembly of such micro devices, which supports the establishment of an interface between a virtual and physical micro assembly work cell. The overall architecture and the various modules implemented in this environment are outlined in this paper.     

Multiple decoupled interaction: An interaction design approach for groupware interaction in co-located virtual environments

Interactive visualizations such as virtual environments and their associated input and interface techniques have traditionally focused on localized single-user interactions and have lacked co-present active collaboration mechanisms where two or more co-located users can share and actively cooperate and interact with the visual simulation. VR facilities such as CAVEs or PowerWalls, among many others, seem to promise such collaboration but due to the special requirements in terms of 3D input and output devices and the physical configuration and layout, they are generally designed to support an active controlling participant—the immersed user—and a passive viewing only audience. In this paper we explore the integration of different technologies, such as small handheld devices and wireless networks with VR/VEs in order to develop a technical and conceptual interaction approach that allows creation of a more ad hoc, interaction rich, multimodal and multi-device environment, where multiple users can access certain interactive capabilities of VE and support co-located collaboration.     

Cubic-Mouse-based interaction in virtual environments

Recently prop-based interfaces in combination with two-handed interaction techniques have become increasingly popular. Passive real-world props augment interaction through tactile feedback and often lead to more intuitive interaction. The Cubic Mouse is a new, 3D input device based on the prop idea. It consists of a cube-shaped box with three perpendicular rods passing through its center. We use a six-degrees-of-freedom (6-DOF) tracker embedded in the Cubic Mouse to track the device's position and orientation. The rods can be pushed and pulled, allowing the constrained input of three degrees of freedom. The 6-DOF Cubic Mouse, also allows rotation of the rods, which adds another three DOF. Altogether, we have a total of 12 analog degrees of freedom available with the Cubic Mouse, six DOF through the tracking and another six through the rods. In addition, the Cubic Mouse has application-programmable control buttons. We present a variety of interaction techniques developed around the Cubic Mouse, which we have implemented in various application prototypes. Our geo-scientific visualization system allows the exploration of data from the oil and gas industry in local and distributed virtual environments. With our automotive partners, we're developing a system for steering and visualizing crash simulations. Other application areas include medical visualization and terrain visualization     

Advanced interaction techniques in virtual environments

Fundamental to much of the Virtual Reality work is, in addition to high-level 3D graphical and multimedia scenes, research on advanced methods of interaction. The “visitor” of such virtual worlds must be able to act and behave intuitively, as he would in everyday situations, as well as receive expectable natural behaviour presented as feedback from the objects in the environment, in a way that he/she has the feeling of direct interaction with his/her application. In this paper we present several techniques to enrich the naturalness and enhance the user involvement in the virtual environment. We present how the user is enabled to grab objects without using any specific and elaborate hand gesture, which is more intuitive and close to the way humans are used to do. We also introduce a technique that makes it possible for the user to surround objects without any force-feedback interaction device. This technique allows the user to surround or “walk” with the virtual hand on the object's surface and look for the best position to grab it.     

Context-driven interaction in immersive virtual environments

There are many interaction tasks a user may wish to accomplish in an immersive virtual environment. A careful examination of these tasks reveals that they are often performed under different contexts. For each task and context, specialized interaction techniques can be developed. We present the context-driven interaction model: a design pattern that represents contextual information as a first-class, quantifiable component within a user interface and supports the development of context-sensitive applications by decoupling context recognition, context representation, and interaction technique development. As a primary contribution, this model provides an enumeration of important representations of contextual information gathered from across the literature and describes how these representations can effect the selection of an appropriate interaction technique. We also identify how several popular 3D interaction techniques adhere to this design pattern and describe how the pattern itself can lead to a more focused development of effective interfaces. We have constructed a formalized programming toolkit and runtime system that serves as a reference implementation of the context-driven model and a discussion is provided explaining how the toolkit can be used to implement a collection of representative 3D interaction interfaces.     

User-centered design and evaluation of virtual environments

We present a structured, iterative methodology for user-centered design and evaluation of VE user interaction. We recommend performing (1) user task analysis followed by (2) expert guidelines-based evaluation, (3) formative user-centered evaluation, and finally (4) comparative evaluation. In this article we first give the motivation and background for our methodology, then we describe each technique in some detail. We applied these techniques to a real-world battlefield visualization VE. Finally, we evaluate why this approach provides a cost-effective strategy for assessing and iteratively improving user interaction in VEs     

Adaptive interactions in shared virtual environments for heterogeneous devices

In this research, we propose a method to adapt the interfaces and interaction processes of heterogeneous devices. The proposed method models the interaction capability of the devices. The interaction is modeled by categorizing elementary actions and measuring its effect in semantic behaviors. With this model, an interaction process can be modified to a given device by changing sequence of elementary actions for each behavior. In the pilot test, we showed the possibility of interaction adaptation for different situations. With the proposed adaptation mechanism, interface and interaction can be modeled for the device independently and can be transferred over different interaction environments. Copyright © 2010 John Wiley & Sons, Ltd.     

Multimodal interaction for 2D and 3D environments [virtual reality]

The allure of immersive technologies is undeniable. Unfortunately, the user's ability to interact with these environments lags behind the impressive visuals. In particular, it's difficult to navigate in unknown visual landscapes, find entities, access information and select entities using six-degrees-of-freedom (6-DOF) devices. We believe multimodal interaction-specifically speech and gesture-will make a major difference in the usability of such environments     

The design space of dynamic interactive virtual environments

Virtual environments have become a key component of many fields and the critical component of virtual reality applications. Due to their virtual nature, they can accommodate an infinite number of possibilities. A theoretical work is presented, which decomposes those innumerous possibilities into concepts to help clarify the vast design space and provide insights into future applied research. We propose that what makes environments interesting and engaging is having worlds that are both active and reactive. This article explores the manifestations of those actions and reactions in what we term: dynamic components and interactions. We term worlds containing these dynamic interactive virtual environments (DIVE). An analysis of each component time was performed, with the purpose of providing a theoretical understanding of the respective design spaces. Initially, we collected the myriad possibilities of each component, e.g., the possible kinds of interactions. We point to examples throughout the field to ground and explain concepts presented. We then categorized of each area into taxonomies. The result of the analyses provides insights into the design space of virtual environments, exposes several avenues of research that are yet underexplored, and provides better understandings of ways in which DIVE creation can be supported.     

Human interaction with IoT-based smart environments

This paper describes concepts, design, implementation, and performance evaluation of a 3D-based user interface for accessing IoT-based Smart Environments (IoT-SE). The generic interaction model of the described work addresses some major challenges of Human-IoT-SE-Interaction such as cognitive overload associated with manual device selection in complex IoT-SE, loss of user control, missing system image or over-automation. To address these challenges we propose a 3D-based mobile interface for mixed-initiative interaction in IoT-SE. The 3D visualization and 3D UI, acting as the central feature of the system, create a logical link between physical devices and their virtual representation on the end user’s mobile devices. By so doing, the user can easily identify a device within the environment based on its position, orientation, and form, and access the identified devices through the 3D interface for direct manipulation within the scene. This overcomes the problem of manual device selection. In addition, the 3D visualization provides a system image for the IoT-SE, which supports users in understanding the ambience and things going on in it. Furthermore, the mobile interface allows users to control the amount and the way the IoT-SE automates the environment. For example, users can stop or postpone system triggered automatic actions, if they don’t like or want them. Users also can remove a rule forever. By so doing, users can delete smart behaviors of their IoT-SE. This helps to overcome the automation challenges. In this paper, we present the design, implementation and evaluation of the proposed interaction system. We chose smart meeting rooms as the context for prototyping and evaluating our interaction concepts. However, the presented concepts and methods are generic and could be adapted to similar environments such as smart homes. We conducted a subjective usability evaluation (ISO-Norm 9241/110) with 16 users. All in one the study results indicate that the proposed 3D-User Interface achieved a good high score according to the ISO-Norm scores.     

An experimental multimedia system allowing 3-D visualization andeye-controlled interaction without user-worn devices

In this paper, a new kind of human-computer interface allowing three-dimensional (3-D) visualization of multimedia objects and eye controlled interaction is proposed. In order to explore the advantages and limitations of the concept, a prototype system has been set up. The testbed includes a visual operating system for integrating novel forms of interaction with a 3-D graphic user interface, autostereoscopic (free-viewing) 3-D displays with close adaptation to the mechanisms of binocular vision, and solutions for nonintrusive eye-controlled interaction (video-based head and gaze tracking). The paper reviews the system's key components and outlines various applications implemented for user testing. Preliminary results show that most of the users are impressed by a 3-D graphic user interface and the possibility to communicate with a computer by simply looking at the object of interest. On the other hand, the results emphasize the need for a more intelligent interface agent to avoid misinterpretation of the user's eye-controlled input and to reset undesired activities     

Collaborative virtual learning environments: design and evaluation

E-learning systems have gone through a radical change from the initial text-based environments to more stimulating multimedia systems. Such systems are Collaborative Virtual Environments, which could be used in order to support collaborative e-learning scenarios. The main aim of this paper is to aid educational designers in selecting, designing and evaluating three dimensional collaborative virtual environments in order to gain the pedagogical benefits of Computer Supported Collaborative Learning. Therefore, this paper initially discusses the potential of three dimensional networked virtual environments for supporting collaborative learning. Furthermore, based on a two-step platform selection process this paper (a) presents and compares three dimensional multi-user virtual environments for supporting collaborative learning and (b) validates the most promising solution against a set of design principles for educational virtual environments. According to these principles, an educational environment has been implemented on top of the selected platform in order to support collaborative e-learning scenarios. The design of this environment is also presented. In addition, this paper presents the results of three small scale studies carried out in a tertiary education department, to assess the educational environment. This environment has been evaluated based on a hybrid evaluation methodology for uncovering usability problems, collecting further requirements for additional functionality to support collaborative virtual learning environments, and determining the appropriateness of different kinds of learning scenarios.

Educational virtual environments: design rationale and architecture

The use of collaborative virtual environments in e-learning is one of the most promising uses of virtual reality technology. While much research has been done in the area of networked virtual environments corresponding to the sharing of events, very little research has been done on specific services and functionality. However both the requirements and the kind of the offered services affect significantly the design of a system. In this paper we present the design and implementation of a platform suitable for educational virtual environments, which are collaborative virtual environments aiming at offering collaborative e-learning services to the users. Apart from the platform itself, we present the technological choices, and a new method for sharing virtual environments. Furthermore, we present an e-learning environment to support e-learning services using collaborative virtual environments from both the technical and functional point of view, along with the end-user evaluation results.