基于Internet的多用户共享虚拟环境框架的研究

多用户共享虚拟环境是指网络化的协同虚拟现实系统，在该环境中多个用户之间通过各自的三维图形表示实现在虚拟环境中的多人感知，用户之间的交互以及对虚拟对象的协同操纵，在综合分析现有系统以及Internet的开放性标准化需求基础上，提出了一种基于Internet的多用户共享虚拟环境解决方案，即多用户共享虚拟环境框架（multi-user shared virtual envitonment framework,MSVEF)，该框架描述了多用户虚拟环境的体系结构，定义了基于XML的相关交互协议，通过在原型系统中的初步应用，证明该框架可以支持不同网络平台的用户在同一个虚拟场景中的再现与交互。     

Gesture-based interaction via finger tracking for mobile augmented reality

The goal of this research is to explore new interaction metaphors for augmented reality on mobile phones, i.e. applications where users look at the live image of the device’s video camera and 3D virtual objects enrich the scene that they see. Common interaction concepts for such applications are often limited to pure 2D pointing and clicking on the device’s touch screen. Such an interaction with virtual objects is not only restrictive but also difficult, for example, due to the small form factor. In this article, we investigate the potential of finger tracking for gesture-based interaction. We present two experiments evaluating canonical operations such as translation, rotation, and scaling of virtual objects with respect to performance (time and accuracy) and engagement (subjective user feedback). Our results indicate a high entertainment value, but low accuracy if objects are manipulated in midair, suggesting great possibilities for leisure applications but limited usage for serious tasks.     

Design evaluation of information appliances using augmented reality-based tangible interaction

In this paper, we propose an approach to tangible augmented reality (AR) based design evaluation of information appliances, which not only exploits the use of tangible objects without hardwired connections to provide better visual immersion and support more tangible interaction, but also facilitates the adoption of a simple and low cost AR environment setup to improve user experience and performance. To enhance the visual immersion, we develop a solution for resolving hand occlusion in which skin color information is exploited with the use of the tangible objects to detect the hand regions properly. To improve the tangible interaction with the sense of touch, we introduce the use of product- and fixture-type objects, which provides the feelings of holding the product in his or her hands and touching buttons with his or her index fingertip in the AR setup. To improve user experience and performance in view of hardware configuration, we devise to adopt a simple and cost-effective AR setup that properly meets guidelines such as viewing size and distance, working posture, viewpoint matching, and camera movement. From experimental results, we found that the AR setup is good to improve the user experience and performance in design evaluation of handheld information appliances. We also found that the tangible interaction combined with the hand occlusion solver in the AR setup is very useful to improve tangible interaction and immersive visualization of virtual products while making the user experience the shapes and functions of the products well and comfortably.     

Interactive 3D content insertion in images for multimedia applications

This article addresses the problem of creating interactive mixed reality applications where virtual objects interact in images of real world scenarios. This is relevant to create games and architectural or space planning applications that interact with visual elements in the images such as walls, floors and empty spaces. These scenarios are intended to be captured by the users with regular cameras or using previously taken photographs. Introducing virtual objects in photographs presents several challenges, such as pose estimation and the creation of a visually correct interaction between virtual objects and the boundaries of the scene. The two main research questions addressed in this article include, the study of the feasibility of creating interactive augmented reality (AR) applications where virtual objects interact in a real world scenario using the image detected high-level features and, also, verifying if untrained users are capable and motivated enough to perform AR initialization steps. The proposed system detects the scene automatically from an image with additional features obtained using basic annotations from the user. This operation is significantly simple to accommodate the needs of non-expert users. The system analyzes one or more photos captured by the user and detects high-level features such as vanishing points, floor and scene orientation. Using these features it will be possible to create mixed and augmented reality applications where the user interactively introduces virtual objects that blend with the picture in real time and respond to the physical environment. To validate the solution several system tests are described and compared using available external image datasets.     

MagicMeeting: A Collaborative Tangible Augmented Reality System

We describe an augmented reality (AR) system that allows multiple participants to interact with 2D and 3D data using tangible user interfaces. The system features face-to-face communication, collaborative viewing and manipulation of 3D models, and seamless access to 2D desktop applications within the shared 3D space. All virtual content, including 3D models and 2D desktop windows, is attached to tracked physical objects in order to leverage the efficiencies of natural two-handed manipulation. The presence of 2D desktop space within 3D facilitates data exchange between the two realms, enables control of 3D information by 2D applications, and generally increases productivity by providing access to familiar tools. We present a general concept for a collaborative tangible AR system, including a comprehensive set of interaction techniques, a distributed hardware setup, and a component-based software architecture that can be flexibly configured using XML. We show the validity of our concept with an implementation of an application scenario from the automotive industry.     

Augmented Reality with Back-Projection Systems using Transflective Surfaces

In this paper, we introduce the concept of Extended VR (extending viewing space and interaction space of back-projection VR systems), by describing the use of a hand-held semi-transparent mirror to support augmented reality tasks with back-projection systems. This setup overcomes the problem of occlusion of virtual objects by real ones linked with such display systems. The presented approach allows an intuitive and effective application of immersive or semi-immersive virtual reality tasks and interaction techniques to an augmented surrounding space. Thereby, we use the tracked mirror as an interactive image-plane that merges the reflected graphics, which are displayed on the projection plane, with the transmitted image of the real environment. In our implementation, we also address traditional augmented reality problems, such as real-object registration and virtual-object occlusion. The presentation is complemented by a hypothesis of conceivable further setups that apply transflective surfaces to support an Extended VR environment.     

Photorealistic rendering of mixed reality scenes

Photo‐realistic rendering of virtual objects into real scenes is one of the most important research problems in computer graphics. Methods for capture and rendering of mixed reality scenes are driven by a large number of applications, ranging from augmented reality to visual effects and product visualization. Recent developments in computer graphics, computer vision, and imaging technology have enabled a wide range of new mixed reality techniques including methods for advanced image based lighting, capturing spatially varying lighting conditions, and algorithms for seamlessly rendering virtual objects directly into photographs without explicit measurements of the scene lighting. This report gives an overview of the state‐of‐the‐art in this field, and presents a categorization and comparison of current methods. Our in‐depth survey provides a tool for understanding the advantages and disadvantages of each method, and gives an overview of which technique is best suited to a specific problem.     

虚拟EAST模型及数据管理系统

EAST装置结构复杂并且含有很多功能各异的子系统,虚拟EAST系统旨在建立一个EAST虚拟现实场景,用户在虚拟EAST场景中通过交互实现虚拟漫游和信息获取.虚拟EAST系统在预处理阶段会产生各种格式和类型的装置模型文件以方便不同系统的开发需要,因此需要高效的管理和使用这些模型.另外,与实验相关的参数信息、诊断信息和电磁测量的相关信息都统一存储在MySQL数据库中,虚拟EAST系统的开发成员共同使用和维护模型及数据信息,因此需要建立一个统一的入口方便查看和管理装置模型和数据,以促进虚拟EAST系统的开发人员之间的相互协作,避免因不规范操作造成的数据信息错误.     

Real-time 3D human capture system for mixed-reality art and entertainment

A real-time system for capturing humans in 3D and placing them into a mixed reality environment is presented in this paper. Nine cameras surrounding her capture the subject. Looking through a head-mounted-display with a camera in front pointing at a marker, the user can see the 3D image of this subject overlaid onto a mixed reality scene. The 3D images of the subject viewed from this viewpoint are constructed using a robust and fast shape-from-silhouette algorithm. The paper also presents several techniques to produce good quality and speed up the whole system. The frame rate of our system is around 25 fps using only standard Intel processor-based personal computers. Besides a remote live 3D conferencing and collaborating system, we also describe an application of the system in art and entertainment, named Magic Land, which is a mixed reality environment where captured avatars of human and 3D computer generated virtual animations can form an interactive story and play with each other. This system demonstrates many technologies in human computer interaction: mixed reality, tangible interaction, and 3D communication. The result of the user study not only emphasizes the benefits, but also addresses some issues of these technologies.     

Immersive Whiteboard Collaborative System

An immersive whiteboard system is presented where users at multiple locations can communicate with each other. The system features a virtual environment with vivid avatars, stroke compression and streaming technology to effectively deliver stroke data across meeting participants, friendly human interaction and navigation, virtual and physical whiteboard. The whiteboard is both a physical platform for our input/output interfaces and a virtual screen for sharing common multimedia. It is this whiteboard correspondence that allows the user to physically write on the virtual whiteboard. In addition to drawing on the shared virtual board, the immersive whiteboard in our setup permits users to control the application menus, insert multimedia objects into the world, and navigate around the virtual environment. By integrating multimedia objects and avatar representations into an immersive environment, we provide the users with a more transparent medium so that they feel as if they are communicating and interacting face-to-face. The whiteboard efficiently pulls all the collaboration technologies together. The goal of this collaborative system is to provide a convenient environment for participants to interact with each other and support collaborative applications such as instant messaging, distance learning and conferencing.     

Interactive virtual relighting of real scenes

Computer augmented reality (CAR) is a rapidly emerging field which enables users to mix real and virtual worlds. Our goal is to provide interactive tools to perform common illumination, i.e., light interactions between real and virtual objects, including shadows and relighting (real and virtual light source modification). In particular, we concentrate on virtually modifying real light source intensities and inserting virtual lights and objects into a real scene; such changes can be very useful for virtual lighting design and prototyping. To achieve this, we present a three-step method. We first reconstruct a simplified representation of real scene geometry using semiautomatic vision-based techniques. With the simplified geometry, and by adapting recent hierarchical radiosity algorithms, we construct an approximation of real scene light exchanges. We next perform a preprocessing step, based on the radiosity system, to create unoccluded illumination textures. These replace the original scene textures which contained real light effects such as shadows from real lights. This texture is then modulated by a ratio of the radiosity (which can be changed) over a display factor which corresponds to the radiosity for which occlusion has been ignored. Since our goal is to achieve a convincing relighting effect, rather than an accurate solution, we present a heuristic correction process which results in visually plausible renderings. Finally, we perform an interactive process to compute new illumination with modified real and virtual light intensities     

Two-handed tangible interaction techniques for composing augmented blocks

Modeling tools typically have their own interaction methods for combining virtual objects. For realistic composition in 3D space, many researchers from the fields of virtual and augmented reality have been trying to develop intuitive interactive techniques using novel interfaces. However, many modeling applications require a long learning time for novice users because of unmanageable interfaces. In this paper, we propose two-handed tangible augmented reality interaction techniques that provide an easy-to-learn and natural combination method using simple augmented blocks. We have designed a novel interface called the cubical user interface, which has two tangible cubes that are tracked by marker tracking. Using the interface, we suggest two types of interactions based on familiar metaphors from real object assembly. The first, the screw-driving method, recognizes the user??s rotation gestures and allows them to screw virtual objects together. The second, the block-assembly method, adds objects based on their direction and position relative to predefined structures. We evaluate the proposed methods in detail with a user experiment that compares the different methods.     

Context-Aware Mixed Reality: A Learning-Based Framework for Semantic-Level Interaction

Mixed reality (MR) is a powerful interactive technology for new types of user experience. We present a semantic-based interactive MR framework that is beyond current geometry-based approaches, offering a step change in generating high-level context-aware interactions. Our key insight is that by building semantic understanding in MR, we can develop a system that not only greatly enhances user experience through object-specific behaviours, but also it paves the way for solving complex interaction design challenges. In this paper, our proposed framework generates semantic properties of the real-world environment through a dense scene reconstruction and deep image understanding scheme. We demonstrate our approach by developing a material-aware prototype system for context-aware physical interactions between the real and virtual objects. Quantitative and qualitative evaluation results show that the framework delivers accurate and consistent semantic information in an interactive MR environment, providing effective real-time semantic-level interactions.     

虚拟环境建模及实时性改善方法

提出了虚拟现实软件实现方法的概念,利用交互方式的,比较,阐明了虚拟现实与仿真的区别,通过计算机三维实体造型和真实感环境泻染技术,实现了虚拟环境建模,通过融入增强现实的思想,把握虚实结合尺度,将基于图象的方法与基于建模的相结合,从而改善了眯拟环境下交互的实时性。     

A Multi-User 3-D Virtual Environment with Interactive Collaboration and Shared Whiteboard Technologies

A multi-user 3-D virtual environment allows remote participants to have a transparent communication as if they are communicating face-to-face. The sense of presence in such an environment can be established by representing each participant with a vivid human-like character called an avatar. We review several immersive technologies, including directional sound, eye gaze, hand gestures, lip synchronization and facial expressions, that facilitates multimodal interaction among participants in the virtual environment using speech processing and animation techniques. Interactive collaboration can be further encouraged with the ability to share and manipulate 3-D objects in the virtual environment. A shared whiteboard makes it easy for participants in the virtual environment to convey their ideas graphically. We survey various kinds of capture devices used for providing the input for the shared whiteboard. Efficient storage of the whiteboard session and precise archival at a later time bring up interesting research topics in information retrieval.     

水彩画风格实时增强现实技术

在传统的AR系统中,虚拟物体和真实场景在视觉上存在较为明显的差异,达不到虚拟物体和真实场景无缝结合的要求。将增强现实技术与NPR有机地结合起来,减小这种视觉差异,研究并实现了水彩画风格的增强现实系统。     

虚拟拉伸实验的设计和实现

利用虚拟现实技术开发了虚拟拉伸实验,阐述了该虚拟实验的功能特点、结构模式以及设计原理,分析了其实现流程及方法,包括三维场景建模、虚拟场景动态生成以及交互式虚拟场景的开发。提出了利用可视化编程语言VB开发一个实现复杂交互动作功能的平台,该平台提供了友好界面,使用户能够直观地组成虚拟三维实验场景,利用键盘或鼠标来控制三维试验机在虚拟场景中运动,并根据用户的指令实现从静态界面对虚拟场景的调用及虚拟场景中试验仪器、工具及其事件的交互,从而控制虚拟模型的动作进行实验。     

Self adaptive animation based on user perspective

In this paper we present a new character animation technique in which the animation adapts itself based on the change in the user’s perspective, so that when the user moves and their point of viewing the animation changes, then the character animation adapts itself in response to that change. The resulting animation, generated in real-time, is a blend of key animations provided a priori by the animator. The blending is done with the help of efficient dual-quaternion transformation blending. The user’s point of view is tracked using either computer vision techniques or a simple user-controlled input modality, such as mouse-based input. This tracked point of view is then used to suitably select the blend of animations. We show a way to author and use such animations in both virtual as well as augmented reality scenarios and demonstrate that it significantly heightens the sense of presence for the users when they interact with such self adaptive animations of virtual characters.     

Virtual world explorations by using topological and semantic knowledge

This paper is dedicated to virtual world exploration techniques. Automatic camera control is important in many fields as computational geometry, visual servoing, robot motion, graph drawing, etc. The paper introduces a high-level camera controlling approach in virtual environments. The proposed method is related to real-time 3D scene exploration and is made of two steps. In the first step, a set of good viewpoints is chosen to give the user a maximum knowledge of the scene. The second step uses the viewpoints to compute a camera path between them. Finally, we define a notion of semantic distance between objects of the scene to improve the approach.     

一个基于局域网的微机虚拟漫游演示系统

介绍了一个基于局域网的微机虚拟漫游演示系统的基本组成和软件开发要点。系统采用头盔显示器和位置跟踪器作为与虚拟环境进行交互的设备,自行研制开发了场景编辑器、场景漫游器、位置跟踪等全部应用软件,使参与者能以接近自然的方式在虚拟场景中漫游。