Immersive Whiteboard Collaborative System

An immersive whiteboard system is presented where users at multiple locations can communicate with each other. The system features a virtual environment with vivid avatars, stroke compression and streaming technology to effectively deliver stroke data across meeting participants, friendly human interaction and navigation, virtual and physical whiteboard. The whiteboard is both a physical platform for our input/output interfaces and a virtual screen for sharing common multimedia. It is this whiteboard correspondence that allows the user to physically write on the virtual whiteboard. In addition to drawing on the shared virtual board, the immersive whiteboard in our setup permits users to control the application menus, insert multimedia objects into the world, and navigate around the virtual environment. By integrating multimedia objects and avatar representations into an immersive environment, we provide the users with a more transparent medium so that they feel as if they are communicating and interacting face-to-face. The whiteboard efficiently pulls all the collaboration technologies together. The goal of this collaborative system is to provide a convenient environment for participants to interact with each other and support collaborative applications such as instant messaging, distance learning and conferencing.     

Human interactions and personal space in collaborative virtual environments

As humans start to spend more time in collaborative virtual environments (CVEs) it becomes important to study their interactions in such environments. One aspect of such interactions is personal space. To begin to address this, we have conducted empirical investigations in a non immersive virtual environment: an experiment to investigate the influence on personal space of avatar gender, and an observational study to further explore the existence of personal space. Experimental results give some evidence to suggest that avatar gender has an influence on personal space although the participants did not register high personal space invasion anxiety, contrary to what one might expect from personal space invasion in the physical world. The observational study suggests that personal space does exist in CVEs, as the users tend to maintain, in a similar way to the physical world, a distance when they are interacting with each other. Our studies provide an improved understanding of personal space in CVEs and the results can be used to further enhance the usability of these environments.     

The H-N2N framework: towards providing interperception in massive applications

We propose a framework with a flexible architecture that have been designed and implemented for collaborative interaction of users, to be applied in massive applications through the Web. We introduce the concept of interperception and use technologies as massive virtual environments and teleoperation for the creation of environments (mixing virtual and real ones) in order to promote accessibility and transparency in the interaction between people, and between people and animate devices (such as robots) through the Web. Experiments with massive games, with interactive applications in digital television, with users and robots interacting in virtual and real versions of museums and cultural centers are presented to validate our proposal.     

Hybrid client-server architecture and control techniques for collaborative product development using haptic interfaces

In this paper, a collaborative product development and prototyping framework is proposed by using distributed haptic interfaces along with deformable objects modeling. Collaborative Virtual Environment (CVE) is a promising technique for industrial product development and virtual prototyping. Network control problems such as network traffic and network delay in communication have greatly limited collaborative virtual environment applications. The problems become more difficult when high-update-rate haptic interfaces and computation intensive deformable objects modeling are integrated into CVEs for intuitive manipulation and enhanced realism. A hybrid network architecture is proposed to balance the computational burden of haptic rendering and deformable object simulation. Adaptive artificial time compensation is used to reduce the time discrepancy between the server and the client. Interpolation and extrapolation approaches are used to synchronize graphic and haptic data transmitted over the network. The proposed techniques can be used for collaborative product development, virtual assembly, remote product simulation and other collaborative virtual environments where both haptic interfaces and deformable object models are involved.     

VSculpt : a distributed virtual sculpting environment for collaborative design

A collaborative virtual sculpting system supports a team of geographically separated designers/engineers connected by networks to participate in designing three-dimensional (3D) virtual engineering tools or sculptures. It encourages international collaboration at a minimal cost. However, in order for the system to be useful, two factors need to be addressed: intuitiveness and real-time interaction. Although a lot of effort has been put into developing virtual sculpting environments, only limited work addresses collaborative virtual sculpting. This is because in order to support real-time collaborative virtual sculpting, many challenging issues need to be addressed. We propose a collaborative virtual sculpting framework, called VSculpt. Through adapting some techniques we developed earlier and integrating them with some techniques developed here, the proposed framework provides a real-time intuitive environment for collaborative design. In particular, it addresses issues on efficient rendering and transmission of deformable objects, intuitive object deformation using the CyberGlove and concurrent object deformation by multiple clients. We demonstrate and evaluate the performance of the proposed framework through a number of experiments.     

VR interactive dialog system with verbal and nonverbal communication

We have constructed a dialog environment between a human and a virtual agent. With commercial off-the-shelf VR technologies, special devices such as a data glove have to be used for the interaction, but it is difficult for anyone to manipulate objects on their own. If there is a helper who has direct access to objects in virtual space, we may ask them. The question, however, is how to communicate with the helper. The basic idea is to utilize speech and gesture recognition systems. We have already reported the above-mentioned result, although only the avatar can move a virtual object in the current system. The user cannot freely manipulate virtual objects. Therefore, in a new attempt, we constructed a communication channel between virtual space and the real world so that the virtual object could be manipulated. In order to develop the new system, we extended the existing system to an internet meeting system allowing users in different places to interact with each other by voice and by a pointing action with a finger. This work was presented in part and awarded as Young Author Award at the 13th International Symposium on Artificial Life and Robotics, Oita, Japan, January 31–February 2, 2008     

基于集合论思想的协同虚拟环境交互研究

交互技术是协同虚拟环境中实现用户间的多感知（视觉、听觉、触觉等）交流与协作的重要手段。目前,在协同虚拟环境中,由于对存在的交互种类缺乏系统、全面的研究方法,从而用户交互程度明显不足,严重影响了地理协同工作。将集合论思想引入到协同虚拟环境的交互中,首次系统、完备性地分析了协同虚拟环境中存在的交互类型,将复杂的交互研究简化为6类交互的组合。在此基础上,采用Agent技术构建了智能语义物体模型以辅助交互的实现,并应用VTK和Cal3D软件开发包,实现了协同虚拟环境中基于化身的多种交互。     

Exploring multi-user interactions with dynamic NFC-displays

Near Field Communication (NFC) is an emerging technology for touch-based mobile interactions with single- and multi-tagged objects. Although the latter may allow for simultaneous and collaborative interactions, most prototypes were not designed for multiple users and were only evaluated with single-user interactions. In this paper, we investigate the design, usability and user experience of multi-user interactions on dynamic NFC-displays. These interactive surfaces use a grid of NFC-tags for the direct manipulation of projected application user interfaces. In two user studies with three prototypes for multi-user interaction, we evaluated the performance of dynamic NFC-displays, interactions among users and the interplay between mobile devices and large displays.     

Creating genetic applications for informal science learning in multi-user virtual environments

A team of developers at the Cornell Theory Center (CTC), Cornell University's high-performance computing and interdisciplinary research center, has implemented a 3D multi-user virtual science museum, SciCentr. SciCentr is a virtual world based on Active Worlds client/server technology. Related to gaming technology, and thus attractive to youth, virtual environments can be applied to formal and informal science education in areas such as genetics, as well as to language learning and social studies. In this paper, we review the current state of our understanding of the impact of this technology on informal science learning. We attempt to identify some of the advantages of using this technology, design features of our implementation, and ideas for managing the social interaction among educational users. More specifically, we will discuss pilot user experiences focused on crop genetics in SciCentr's Plant Breeding Beds and on the related SciFair exhibits, both examples of collaborative experiences in our virtual worlds. Finally, we describe the next stage in interactive interface development for our Gene Bot feature.     

Real-Time Tracking of Visually Attended Objects in Virtual Environments and Its Application to LOD

This paper presents a real-time framework for computationally tracking objects visually attended by the user while navigating in interactive virtual environments. In addition to the conventional bottom-up (stimulus-driven) saliency map, the proposed framework uses top-down (goal-directed) contexts inferred from the user's spatial and temporal behaviors, and identifies the most plausibly attended objects among candidates in the object saliency map. The computational framework was implemented using GPU, exhibiting high computational performance adequate for interactive virtual environments. A user experiment was also conducted to evaluate the prediction accuracy of the tracking framework by comparing objects regarded as visually attended by the framework to actual human gaze collected with an eye tracker. The results indicated that the accuracy was in the level well supported by the theory of human cognition for visually identifying single and multiple attentive targets, especially owing to the addition of top-down contextual information. Finally, we demonstrate how the visual attention tracking framework can be applied to managing the level of details in virtual environments, without any hardware for head or eye tracking.     

See-through techniques for referential awareness in collaborative virtual reality

Multi-user virtual reality systems enable natural collaboration in shared virtual worlds. Users can talk to each other, gesture and point into the virtual scenery as if it were real. As in reality, referring to objects by pointing results often in a situation whereon objects are occluded from the other users' viewpoints. While in reality this problem can only be solved by adapting the viewing position, specialized individual views of the shared virtual scene enable various other solutions. As one such solution we propose show-through techniques to make sure that the objects one is pointing to can always be seen by others. We first study the impact of such augmented viewing techniques on the spatial understanding of the scene, the rapidity of mutual information exchange as well as the proxemic behavior of users. To this end we conducted a user study in a co-located stereoscopic multi-user setup. Our study revealed advantages for show-through techniques in terms of comfort, user acceptance and compliance to social protocols while spatial understanding and mutual information exchange is retained. Motivated by these results we further analyze whether show-through techniques may also be beneficial in distributed virtual environments. We investigated a distributed setup for two users, each participant having its own display screen and a minimalist avatar representation for each participant. In such a configuration there is a lack of mutual awareness, which hinders the understanding of each other's pointing gestures and decreases the relevance of social protocols in terms of proxemic behavior. Nevertheless, we found that show-through techniques can improve collaborative interaction tasks even in such situations.     

Sharing VLNET worlds on the Web

Virtual environments define a new interface for networked multimedia applications. The sense of “presence” in the virtual environment is an important requirement for collaborative activities involving multiple remote users working with social interactions. Using virtual actors within the shared environment is a supporting tool for presence. In this paper, we present a shared virtual life network with virtual humans that provides a natural interface for collaborative working and we describe the bridge we realized between this 3D shared world and the Web through a system of 3D snapshots.     

Embodiments, avatars, clones and agents for multi-user, multi-sensory virtual worlds

This paper explores the issue of user embodiment within collaborative virtual environments. By user embodiment we mean the provision of users with appropriate body images so as to represent them to others and also to themselves. By collaborative virtual environments we mean multi-user virtual reality systems which explicitly support cooperative work (although we argue that the results of our exploration may also be applied to other kinds of collaborative system). The main part of the paper identifies a list of embodiment design issues grouped by the general themes of personal representation, conveying activity, embodiment in heterogeneous systems, embodiment of agents, and ethical issues. These issues are illustrated with examples from our own DIVE and MASSIVE collaborative virtual environments. The paper also uses this set of issues as an analytical framework for comparing a number of other communication technologies.     

Enhancing synchronous collaboration by using interactive visualisation of modelled traces

This article addresses issues related to traces modelling for formally describing human interactions of people engaged in a synchronous collaborative learning activity. The objective is to propose models and tools for representing, transforming, sharing and visualising traces of users’ experiences. The traces here represent the users’ activities in their interactions with the learning platform. Our proposition is based on reflexive learning defined as the ability to interact with the situation, in order to meet one’s own limitations. This work takes place in the ITHACA project which aims at developing an on-line learning platform that uses interaction traces as knowledge sources on, and for, the learners’ learning as individuals or groups. In this paper, we propose a general framework for trace management and sharing, a generic model of synchronous collaborative activity based on the notion of interaction modes that we specialized for whiteboard sharing and text chatting, and a conceptual framework for modelling the exploitation of modelled traces, in particular for interactive visualisation on the user side. This article extends our previous work [1] on the instrumented prototypes, by presenting our theorisation of the interactive visualisation of modelled traces.     

Harmonic rendering for visual coherence on mobile outdoor AR environment

Rapid developments in augmented reality (AR) and related technologies have led to increasing interest in immersive content. AR environments are created by combining virtual 3D models with a real-world video background. It is important to merge these two worlds seamlessly if users are to enjoy AR applications, but, all too often, the illumination and shading of virtual objects is not consider the real world lighting condition or does not match that of nearby real objects. In addition, visual artifacts produced when blending real and virtual objects further limit realism. In this paper, we propose a harmonic rendering technique that minimizes the visual discrepancy between the real and virtual environments to maintain visual coherence in outdoor AR. To do this, we introduce a method of estimating and approximating the Sun’s position and the sunlight direction to estimate the real sunlight intensity, as this is the most significant illumination source in outdoor AR and it provides a more realistic lighting environment for such content, reducing the mismatch between real and virtual objects.

     

Gaze interaction with virtual on-line communities: levelling the playing field for disabled users

This paper introduces the concept of enabling gaze-based interaction for users with high-level motor disabilities to control an avatar in a first-person perspective on-line community. An example community, Second Life, is introduced that could offer disabled users the same virtual freedom as any other user, and so allow disabled users to be able-bodied (should they wish) within the virtual world. A survey of the control demands for Second Life and a subsequent preliminary experiment show that gaze control has inherent problems particularly for locomotion and camera movement. These problems result in a lack of effective gaze control of Second Life, such that control is not practical and show that disabled users who interact using gaze will have difficulties in controlling Second Life (and similar environments). This suggests that these users could once again become disabled in the virtual world by the difficulties in effectively controlling their avatars, and their ‘disability privacy’, or the right to control an avatar as effectively as an able bodied user, and so appear virtually able bodied, will be compromised. Methods for overcoming these difficulties such as the use of gaze aware on-screen assistive tools could overcome these problems, but games manufacturers must design inclusively, so that disabled users may have the right to disability privacy in their Second (virtual) Lives.     

A trajectory-preserving synchronization method for collaborative visualization

In the past decade, a lot of research work has been conducted to support collaborative visualization among remote users over the networks, allowing them to visualize and manipulate shared data for problem solving. There are many applications of collaborative visualization, such as oceanography, meteorology and medical science. To facilitate user interaction, a critical system requirement for collaborative visualization is to ensure that remote users will perceive a synchronized view of the shared data. Failing this requirement, the user's ability in performing the desirable collaborative tasks will be affected. In this paper, we propose a synchronization method to support collaborative visualization. It considers how interaction with dynamic objects is perceived by application participants under the existence of network latency, and remedies the motion trajectory of the dynamic objects. It also handles the false positive and false negative collision detection problems. The new method is particularly well designed for handling content changes due to unpredictable user interventions or object collisions. We demonstrate the effectiveness of our method through a number of experiments.     

Context-Aware Mixed Reality: A Learning-Based Framework for Semantic-Level Interaction

Mixed reality (MR) is a powerful interactive technology for new types of user experience. We present a semantic-based interactive MR framework that is beyond current geometry-based approaches, offering a step change in generating high-level context-aware interactions. Our key insight is that by building semantic understanding in MR, we can develop a system that not only greatly enhances user experience through object-specific behaviours, but also it paves the way for solving complex interaction design challenges. In this paper, our proposed framework generates semantic properties of the real-world environment through a dense scene reconstruction and deep image understanding scheme. We demonstrate our approach by developing a material-aware prototype system for context-aware physical interactions between the real and virtual objects. Quantitative and qualitative evaluation results show that the framework delivers accurate and consistent semantic information in an interactive MR environment, providing effective real-time semantic-level interactions.     

Interaction with a desktop virtual environment: a 2D view into a 3D world

With the development of computer software and hardware in the past few years, it has been possible to produce effective training virtual environments on everyday personal computers with little expert training required for users or designers. However, the development of the equipment that enables this has brought little coinciding research on what features to include when designing these environments. Despite these increased advances in PC capabilities for desktop virtual environments (VEs), there are still limitations on the number of objects that can be programmed to be interactive, usually due to restrictions on programming time and cost. As a result, it is often left to the programmer to decide which of the objects included to increase the realism of the environment will be interactive and which aesthetic. The work presented in this paper is an experiment that aims to establish a guide for environment designers to aid effective environment interaction development by identifying key elements in a VE design.

Two-handed tangible interaction techniques for composing augmented blocks

Modeling tools typically have their own interaction methods for combining virtual objects. For realistic composition in 3D space, many researchers from the fields of virtual and augmented reality have been trying to develop intuitive interactive techniques using novel interfaces. However, many modeling applications require a long learning time for novice users because of unmanageable interfaces. In this paper, we propose two-handed tangible augmented reality interaction techniques that provide an easy-to-learn and natural combination method using simple augmented blocks. We have designed a novel interface called the cubical user interface, which has two tangible cubes that are tracked by marker tracking. Using the interface, we suggest two types of interactions based on familiar metaphors from real object assembly. The first, the screw-driving method, recognizes the user??s rotation gestures and allows them to screw virtual objects together. The second, the block-assembly method, adds objects based on their direction and position relative to predefined structures. We evaluate the proposed methods in detail with a user experiment that compares the different methods.