Shared space: An augmented reality approach for computer supported collaborative work

Virtual Reality (VR) appears a natural medium for three-dimensional computer supported collaborative work (CSCW). However the current trend in CSCW is to adapt the computer interface to work with the user's traditional tools, rather than separating the user from the real world as does immersive VR. One solution is through Augmented Reality, the overlaying of virtual objects on the real world. In this paper we describe the Shared Space concept—the application of Augmented Reality for three-dimensional CSCW. This combines the advantages of Virtual Reality with current CSCW approaches. We describe a collaborative experiment based on this concept and present preliminary results which show that this approach may be better for some applications.     

增强现实技术综述

增强现实技术是将计算机渲染生成的虚拟场景与真实世界中的场景无缝融合起来的一种技术,它通过视频显示设备将虚实融合的场景呈现给用户,使人们与计算机之间的交互更加的自然,同时具有广泛的应用前景,因此成为近年来的一个研究热点。随着跟踪注册技术的进步、计算机性能的飞速发展、深度摄像机的普及,以及Light Field投影技术在增强现实中的应用,增强现实技术逐渐成为下一代人机交互的发展方向。该文章首先概述了增强现实的主要研究内容和发展情况,并详细介绍了增强现实的关键技术、开发工具,然后分类概述了增强现实应用案例。     

Experiments with Face-To-Face Collaborative AR Interfaces

We describe a design approach, Tangible Augmented Reality, for developing face-to-face collaborative Augmented Reality (AR) interfaces. Tangible Augmented Reality combines Augmented Reality techniques with Tangible User Interface elements to create interfaces in which users can interact with spatial data as easily as real objects. Tangible AR interfaces remove the separation between the real and virtual worlds, and so enhance natural face-to-face communication. We present several examples of Tangible AR interfaces and results from a user study that compares communication in a collaborative AR interface to more traditional approaches. We find that in a collaborative AR interface people use behaviours that are more similar to unmediated face-to-face collaboration than in a projection screen interface.     

An empirical evaluation of two natural hand interaction systems in augmented reality

Human-computer interaction based on hand gesture tracking is not uncommon in Augmented Reality. In fact, the most recent optical Augmented Reality devices include this type of natural interaction. However, due to hardware and system limitations, these devices, more often than not, settle for semi-natural interaction techniques, which may not always be appropriate for some of the tasks needed in Augmented Reality applications. For this reason, we compare two different optical Augmented Reality setups equipped with hand tracking. The first one is based on a Microsoft HoloLens (released in 2016) and the other one is based on a Magic Leap One (released more than two years later). Both devices offer similar solutions for the visualization and registration problems but differ in the hand tracking approach, since the former uses a metaphoric hand-gesture tracking and the latter relies on an isomorphic approach. We raise seven research questions regarding these two setups, which we answer after performing two task-based experiments using virtual elements, of different sizes, that are moved using natural hand interaction. The questions deal with the accuracy and performance achieved with these setups and also with user preference, recommendation and perceived usefulness. For this purpose, we collect both subjective and objective data about the completion of these tasks. Our initial hypothesis was that there would be differences, in favor of the isomorphic and newer setup, in the use of hand interaction. However, the results surprisingly show that there are very small objective differences between these setups, and the isomorphic approach is not significantly better in terms of accuracy and mistakes, although it allows a faster completion of one of the tasks. In addition, no remarkable statistically significant differences can be found between the two setups in the subjective datasets gathered through a specific questionnaire. We also analyze the opinions of the participants in terms of usefulness, preference and recommendation. The results show that, although the Magic Leap-based system gets more support, the differences are not statistically significant.

     

Augmented reality technologies,systems and applications

This paper surveys the current state-of-the-art of technology, systems and applications in Augmented Reality. It describes work performed by many different research groups, the purpose behind each new Augmented Reality system, and the difficulties and problems encountered when building some Augmented Reality applications. It surveys mobile augmented reality systems challenges and requirements for successful mobile systems. This paper summarizes the current applications of Augmented Reality and speculates on future applications and where current research will lead Augmented Reality’s development. Challenges augmented reality is facing in each of these applications to go from the laboratories to the industry, as well as the future challenges we can forecast are also discussed in this paper. Section 1 gives an introduction to what Augmented Reality is and the motivations for developing this technology. Section 2 discusses Augmented Reality Technologies with computer vision methods, AR devices, interfaces and systems, and visualization tools. The mobile and wireless systems for Augmented Reality are discussed in Section 3. Four classes of current applications that have been explored are described in Section 4. These applications were chosen as they are the most famous type of applications encountered when researching AR apps. The future of augmented reality and the challenges they will be facing are discussed in Section 5.     

实景空间中虚拟对象嵌入技术研究

实景空间是以实景图像为素材构造出的具有三维操纵能力的虚拟空间,增强现实技术可以把计算机产生的虚拟物体或其他信息合成到用户看到的真实场景中。该文主要讨论如何在实景空间中嵌入计算机生成的虚拟对象。首先介绍了实景空间的概念和模型,然后着重探讨了虚拟对象与实景空间的合成方法,主要解决合成中的几何一致性和光照一致性问题。     

混合跟踪技术在增强现实系统中的应用

跟踪技术是增强现实的一个重要研究方向,它致力于解决虚拟信息与真实环境的精确配准问题,构建稳定、高效的增强现实系统。通过分析增强现实系统的跟踪需求,指出单一跟踪存在的不足,说明采用混合跟踪技术的必要性,论述了基于视觉一惯性的混合跟踪技术及其在增强现实中的应用,实现了一种基于VisTracker视觉-惯性混合跟踪器的增强现实实验系统,并给出调试结果。     

Designing and implementing interactive and realistic augmented reality experiences

In this paper, we propose an approach for supporting the design and implementation of interactive and realistic Augmented Reality (AR). Despite the advances in AR technology, most software applications still fail to support AR experiences where virtual objects appear as merged into the real setting. To alleviate this situation, we propose to combine the use of model-based AR techniques with the advantages of current game engines to develop AR scenes in which the virtual objects collide, are occluded, project shadows and, in general, are integrated into the augmented environment more realistically. To evaluate the feasibility of the proposed approach, we extended an existing game platform named GREP to enhance it with AR capacities. The realism of the AR experiences produced with the software was assessed in an event in which more than 100 people played two AR games simultaneously.

     

A geospatial technique for detecting distance and Reflection Angle between real and virtual objects

This paper presents a geospatial collision detection technique consisting of two methods: Find Object Distance (FOD) and Find Reflection Angle (FRA). We show how the geospatial collision detection technique using a computer vision system detects a computer generated virtual object and a real object manipulated by a human user and how the virtual object can be reflected on a real floor after being detected by a real object. In the geospatial collision detection technique, the FOD method detects the real and virtual objects, and the FRA method predicts the next moving directions of virtual objects. We demonstrate the two methods by implementing a floor based Augmented Reality (AR) game, Ting Ting, which is played by bouncing fire-shaped virtual objects projected on a floor using bamboo-shaped real objects. The results reveal that the FOD and the FRA methods of the geospatial collision detection technique enable the smooth interaction between a real object manipulated by a human user and a virtual object controlled by a computer. The proposed technique is expected to be used in various AR applications as a low cost interactive collision detection engine such as in educational materials, interactive contents including games, and entertainment equipments. Keywords: Augmented reality, collision detection, computer vision, game, human computer interaction, image processing, interfaces.     

基于增强现实的虚拟实景空间的研究与实现

增强现实可以把计算机产生的虚拟物体或其他信息合成到用户看到的真实场景中．本文将增强现实技术与基于图像的绘制技术有机的结合起来，研究并实现了一个基于增强现实技术的虚拟实景空间系统．首先介绍了虚拟实景空间的构造，然后着重探讨了虚拟物体与实景空间的合成方法，主要解决合成中的几何一致性和光照一致性问题。     

Representing information – Classifying the Augmented Reality presentation space

Augmented Reality has a wide-ranging presentation space. In addition to presenting virtual information in a 3D space, such information can also be placed in relation to physical objects, locations or events.Decomposing this presentation space – or more exactly, the principles of how information is represented in Augmented Reality – into unique and independent dimensions provides a fundamental spectrum of options. First, this decomposition facilitates a fine-grained analysis of effects on human understanding. Second, multiple factors, given by multiple differences between different presentation systems with respect to more than one such principle, can be determined and properly addressed. Third, this decomposition facilitates a determination of new fields of research by identifying not-yet-used concepts.Since the beginning of Augmented Reality research, a growing number of applications have emerged that exploit various ways to represent information. This paper resumes this development and presents a set of independent dimensions covering representation principles of virtual information related to a physical environment: the temporality of virtual information, dimensionality, the frame of reference, mounting/registration and the type of reference. The suitability of the devised dimensions is tested by categorizing a wide variety of AR applications. The categorized data is analyzed for the most-often and less-frequently used combinations of classes. In particular, the classes that have not yet been used exhibit the potential to allow future work that investigates new options for information presentation.     

一种新型的交互式增强现实系统

增强现实是随着虚拟现实技术的发展而衍生小来的。它抒计算机产生的虚拟物体合成到真实环境当中。本文采用ARToolKit系统作为开发环境,Open Dynamic Engille（ODE）为动力学引擎以及利用Open Scene Graph（OSG）图形引擎开发包,将增强现实技术与碰撞检测技术相结合实现了一种新型的交互式增强现实系统。     

数字媒体未来发展趋势——数字媒体与增强现实技术

数字反哺现实:增强现实的概念;列举增强现实技术在科技方面的应用;增强现实的前沿技术成果;增强现实在艺术领域的应用;游戏:游戏领域基于增强现实技术的新发展。媒体融合的增强现实技术:增强现实技术与媒体融合成果的展望;增强现实技术已经发展到媒体融合阶段     

Visual Gaze Labeling for Augmented Reality Studies

Augmented Reality (AR) provides new ways for situated visualization and human-computer interaction in physical environments. Current evaluation procedures for AR applications rely primarily on questionnaires and interviews, providing qualitative means to assess usability and task solution strategies. Eye tracking extends these existing evaluation methodologies by providing indicators for visual attention to virtual and real elements in the environment. However, the analysis of viewing behavior, especially the comparison of multiple participants, is difficult to achieve in AR. Specifically, the definition of areas of interest (AOIs), which is often a prerequisite for such analysis, is cumbersome and tedious with existing approaches. To address this issue, we present a new visualization approach to define AOIs, label fixations, and investigate the resulting annotated scanpaths. Our approach utilizes automatic annotation of gaze on virtual objects and an image-based approach that also considers spatial context for the manual annotation of objects in the real world. Our results show, that with our approach, eye tracking data from AR scenes can be annotated and analyzed flexibly with respect to data aspects and annotation strategies.     

Covering photo-realistic properties of outdoor components with the effects of sky color in mixed reality

Outdoor rendering is an attractive topic in computer graphics. In this paper our main concern is to reveal the interaction between sky color and virtual objects in Mixed Reality environments. Although registration and tracking are two of the main issues in building effective Augmented Reality (AR) systems the creation of more realistic virtual objects indistinguishable from their real-world counterparts is our target which is currently the ultimate goal in AR. Two classes of sky color generation are employed to reveal the outdoor-element interaction. Virtual Sky Modelling (VSM) based on the Perez Model is capable of generating the sky color in a specific location, date and time. The second technique is to generate a virtual model based on the real image of the sky which is called in this paper Real Sky Modelling (RSM). Subsequently, preprocessing of the sky color bleeding is based on the radiosity technique to give the sky color effect to the virtual objects as well as the real ones. Through designing a test AR set-up and applying software and hardware the goal of a robust generation of realistic virtual objects with effect of sky color is achieved.     

The MATRIS project: real-time markerless camera tracking for Augmented Reality and broadcast applications

The term “Augmented Reality” covers a wide range of applications, from the overlay of virtual graphics on a real scene using a head-mounted display for use in areas such as industrial maintenance, through to the insertion of real-time virtual graphics in TV programmes. Fundamental to all these applications is the need to be able to accurately track the motion of the camera, so that the graphics may be rendered so as to appear rigidly locked to the real world. To overcome the limitations of existing tracking systems, the MATRIS project has developed a real-time system for measuring the movement of a camera, which uses image analysis to track naturally occurring features in the scene, and data from an inertial sensor. No additional sensors, special markers, or camera mounts are required. This paper gives an overview of the system, provides the context for the other articles in this journal and presents some results.     

In-Situ interactive image-based model building for Augmented Reality from a handheld device

Three-dimensional models of objects and their creation process are central for a variety of applications in Augmented Reality. In this article, we present a system that is designed for in-situ modeling using interactive techniques for two generic versions of handheld devices equipped with cameras. The system allows online building of 3D wireframe models through a combination of user interaction and automated methods. In particular, we concentrate in rigorous evaluation of the two devices and interaction methods in the context of 3D feature selection. We present the key components of our system, discuss our findings and results and identify design recommendations.     

Handling occlusions in video‐based augmented reality using depth information

Augmented Reality (AR) composes virtual objects with real scenes in a mixed environment where human–computer interaction has more semantic meanings. To seamlessly merge virtual objects with real scenes, correct occlusion handling is a significant challenge. We present an approach to separate occluded objects in multiple layers by utilizing depth, color, and neighborhood information. Scene depth is obtained by stereo cameras and two Gaussian local kernels are used to represent color, spatial smoothness. These three cues are intelligently fused in a probability framework, where the occlusion information can be safely estimated. We apply our method to handle occlusions in video‐based AR where virtual objects are simply overlapped on real scenes. Experiment results show the approach can correctly register virtual and real objects in different depth layers, and provide a spatial‐awareness interaction environment. Copyright © 2009 John Wiley & Sons, Ltd.     

面向虚拟化身的人脸表情模拟技术

为了实现基于增强现实的电子沙盘环境中的异地可视化交互功能,提出了一种面 向虚拟化身的三维表情模拟技术。首先,使用 RGB 摄像头跟踪异地作业人员的表情,基于约 束局部模型(CLM)提取人脸特征点数据后传输到本地;然后,采用基于径向基函数的插值算法 计算虚拟化身面部网格点的坐标,驱动模型模拟出与异地作业人员相同的表情;最后,为了提 高变形算法的精度和效率,提出一种基于贪心算法与人脸肌群分布的插值控制点选取和分区域 插值方法。实验结果表明,该算法能够满足实际应用对实时性和真实感的需求。     

Extended overview techniques for outdoor augmented reality

In this paper, we explore techniques that aim to improve site understanding for outdoor Augmented Reality (AR) applications. While the first person perspective in AR is a direct way of filtering and zooming on a portion of the data set, it severely narrows overview of the situation, particularly over large areas. We present two interactive techniques to overcome this problem: multi-view AR and variable perspective view. We describe in details the conceptual, visualization and interaction aspects of these techniques and their evaluation through a comparative user study. The results we have obtained strengthen the validity of our approach and the applicability of our methods to a large range of application domains.