Optically based direct manipulation for augmented reality

Augmented reality (AR) constitutes a very powerful three-dimensional user interface for many “hands-on” application scenarios. To fully exploit the AR paradigm, the computer must not only augment the real world, but also accept feedback from it. In this paper, we present an optical approach for collecting such feedback by analyzing video sequences to track users and the objects they work with. Our system can be set up in any room after quickly placing a few known optical targets in the scene. We present two demonstration scenarios to illustrate the overall concept and potential of our approach and then discuss the research issues involved.     

An augmented reality interface for visualizing and interacting with virtual content

In this paper, a novel AR interface is proposed that provides generic solutions to the tasks involved in augmenting simultaneously different types of virtual information and processing of tracking data for natural interaction. Participants within the system can experience a real-time mixture of 3D objects, static video, images, textual information and 3D sound with the real environment. The user-friendly AR interface can achieve maximum interaction using simple but effective forms of collaboration based on the combinations of human–computer interaction techniques. To prove the feasibility of the interface, the use of indoor AR techniques are employed to construct innovative applications and demonstrate examples from heritage to learning systems. Finally, an initial evaluation of the AR interface including some initial results is presented.     

Real-time advanced spinal surgery via visible patient model and augmented reality system

This paper presents an advanced augmented reality system for spinal surgery assistance, and develops entry-point guidance prior to vertebroplasty spinal surgery. Based on image-based marker detection and tracking, the proposed camera-projector system superimposes pre-operative 3-D images onto patients. The patients’ preoperative 3-D image model is registered by projecting it onto the patient such that the synthetic 3-D model merges with the real patient image, enabling the surgeon to see through the patients’ anatomy. The proposed method is much simpler than heavy and computationally challenging navigation systems, and also reduces radiation exposure. The system is experimentally tested on a preoperative 3D model, dummy patient model and animal cadaver model. The feasibility and accuracy of the proposed system is verified on three patients undergoing spinal surgery in the operating theater. The results of these clinical trials are extremely promising, with surgeons reporting favorably on the reduced time of finding a suitable entry point and reduced radiation dose to patients.     

Marker-less registration based on template tracking for augmented reality

Accurate 3D registration is a key issue in the Augmented Reality (AR) applications, particularly where are no markers placed manually. In this paper, an efficient markerless registration algorithm is presented for both outdoor and indoor AR system. This algorithm first calculates the correspondences among frames using fixed region tracking, and then estimates the motion parameters on projective transformation following the homography of the tracked region. To achieve the illumination insensitive tracking, the illumination parameters are solved jointly with motion parameters in each step. Based on the perspective motion parameters of the tracked region, the 3D registration, the camera’s pose and position, can be calculated with calibrated intrinsic parameters. A marker-less AR system is described using this algorithm, and the system architecture and working flow are also proposed. Experimental results with comparison quantitatively demonstrate the correctness of the theoretical analysis and the robustness of the registration algorithm.

A haptic pedal for surgery assistance

The research and development of mechatronic aids for surgery is a persistent challenge in the field of robotic surgery. This paper presents a new haptic pedal conceived to assist surgeons in the operating room by transmitting real-time surgical information through the foot. An effective human–robot interaction system for medical practice must exchange appropriate information with the operator as quickly and accurately as possible. Moreover, information must flow through the appropriate sensory modalities for a natural and simple interaction. However, users of current robotic systems might experience cognitive overload and be increasingly overwhelmed by data streams from multiple modalities. A new haptic channel is thus explored to complement and improve existing systems. A preliminary set of experiments has been carried out to evaluate the performance of the proposed system in a virtual surgical drilling task. The results of the experiments show the effectiveness of the haptic pedal in providing surgical information through the foot.     

Lessons learned on the way to industrial augmented reality applications, a retrospective on ARVIKA

This paper presents and discusses the results of the project ARVIKA. After a short introduction and presentation of the project goals, the augmented reality system is described. It consists in a Plug-In for internet-browser in which all the software components have been integrated. The advantages and disadvantages of the adopted approaches, as well as the lessons learned from the project are presented in details. Finally, the open issues are discussed and future work to be achieved until a possible implementation of augmented reality in an industrial context is proposed.     

An interactive virtual guide for the AR based visit of archaeological sites

One of the most interesting research lines about avatars is the design and the implementation of a synthetic behaviour able to drive avatar's actions according to an adaptive interaction paradigm. This aspect, indeed, is of fundamental importance to many advanced applications involving avatars and humans. This study presents a novel framework exploiting augmented reality to visualize a synthetic 3D virtual guide inside an exhibit or a public gallery, to assist visitors wearing a Head Mounted Display during their visit and providing them with both visual and informative enhancements not available in a standard tour. The Human–avatar interaction is approached through a model based on timed automata to address the conversational issues and to improve the quality of interaction by means of an effective synchronization. A usability study conducted on an application of this research to the “avatar assisted tour” of a roman villa, confirms the efficacy of the approach.     

An augmented reality human–computer interface for object localization in a cognitive vision system

The European Cognitive Vision project VAMPIRE uses mobile AR-kits to interact with a visual active memory for teaching and retrieval purposes. This paper describes concept and technical realization of the used mobile AR-kits and discusses interactive learning and retrieval in office environments, and the active memory infrastructure. The focus is on 3D interaction for pointing in a scene coordinate system. This is achieved by 3D augmented pointing, which combines inside-out tracking for head pose recovery and 3D stereo human–computer interaction. Experimental evaluation shows that the accuracy of this 3D cursor is within a few centimeters, which is sufficient to point at an object in an office. Finally, an application of the cursor in VAMPIRE is presented, where in addition to the mobile system, at least one stationary active camera is used to obtain different views of an object. There are many potential applications, for example an improved view-based object recognition.     

Treating cockroach phobia using a serious game on a mobile phone and augmented reality exposure: A single case study

In vivo exposure has proved its efficacy in the treatment of specific phobias; however, not all patients benefit from it. Communication and information technologies such as Virtual Reality (VR) and Augmented Reality (AR) have improved exposure treatment adherence and acceptance. Serious games (SG) could also be used in order to facilitate exposure treatment. A line of research on SG is emerging which focuses on health issues. We have developed a SG for the treatment of cockroach phobia that uses a mobile phone as the application device. This work examines results of an N = 1 study about whether the use of this mobile game can facilitate treatment of this specific phobia preparing her for the AR exposure. A 25-year-old woman with cockroach phobia participated in the study. Results showed that the use of the mobile game reduced her level of fear and avoidance before a “one-session” AR exposure treatment was applied, following the guidelines by Öst. The participant found very helpful the use of the SG before the AR exposure session and she was willing to use it after the AR exposure session as a homework assignment. Although the results of this study are preliminary, SG appears to be a line of research of high interest in clinical psychology for the treatment of specific phobias.     

基于增强现实的虚拟城市规划关键技术研究

增强现实是虚拟现实技术的重要分支,已经被成功的应用到众多领域.探讨了增强现实应用于虚拟城市规划的可行性,并给出了一种新颖的基于增强现实技术的城市规划方法.它可以满足城市规划对虚实交互、人机交互的要求,充分体现出增强现实技术在城市规划应用中的优势.介绍了系统结构与工作流程,着重分析了所采用的基于标识的虚实注册方法,并给出了运行结果.     

Effects of using a force feedback haptic augmented simulation on the attitudes of the gifted students towards studying chemical bonds in virtual reality environment

The aim of this study is to identify the effects of force feedback haptic applications developed in virtual reality environments (VREs), which is an important field of study in computer science and engineering, on gifted students’ attitudes towards chemistry education in learning process. A 3D 6 DOF (Degree of Freedom) haptik device (Phantom Omni?) has been used to develop the algorithm in this study. It can be used to transmit force and motion using a haptic device. Visual C++ was choosen as the software development environment. OpenGL? and Haptic Device Application Programming Interface have been used for rendering graphics. At the 3D image creation state Wrap 1200?, which is a kind of head-mounted display, has been chosen. The sample of this study consists of 52 students identified as gifted and are attending 6th and 7th grades at the Istanbul Science &; Art Center in Istanbul. The experimental group studied chemical bonds using an application developed by using a force feedback haptic device in VRE and the control group studied it by traditional teaching methods. The study reveals that there is a relation between using force feedback haptic applications which are developed in VREs and gifted students’ attitudes towards educational programs.     

Multi-user mixed reality system ‘Gulliver’s World’: a case study on collaborative edutainment at the intersection of material and virtual worlds

This case study is about ‘Gulliver’s World’, a multi-user mixed reality environment that functions simultaneously as interactive edutainment platform and learning environment as well as flexible infrastructure for the expansion of mixed reality environments via innovations in technology and media art. As an exhibition project, the installation is characterized by a nonlinear exhibition concept that activates interaction between individual users and different modes of virtual reality as well as collaboration among the users themselves. At seven workstations, people of all age groups range along the Reality–Virtuality continuum while collaboratively creating 3D worlds. Results of these creation activities are interactive worlds at the nexus of theatre, digital film production and game environment. As a research project, ‘Gulliver’s World’ features multilevel infrastructure with exemplary content in which the latest insights and models to emerge from HCI research as well as concepts of mixed reality and virtual environments and their supporting technology are brought together and developed further.