An augmented reality interface for visualizing and interacting with virtual content

In this paper, a novel AR interface is proposed that provides generic solutions to the tasks involved in augmenting simultaneously different types of virtual information and processing of tracking data for natural interaction. Participants within the system can experience a real-time mixture of 3D objects, static video, images, textual information and 3D sound with the real environment. The user-friendly AR interface can achieve maximum interaction using simple but effective forms of collaboration based on the combinations of human–computer interaction techniques. To prove the feasibility of the interface, the use of indoor AR techniques are employed to construct innovative applications and demonstrate examples from heritage to learning systems. Finally, an initial evaluation of the AR interface including some initial results is presented.     

An application of augmented reality to thickness inspection

An experiment was conducted to evaluate the use of augmented reality (AR) for thickness inspection of manufactured parts. The inspection task consisted of determining the thickness of flat-bottomed parts via measurements, taken at predetermined locations and in a specified sequence, using a vernier height gage. Inspectors wore a head-mounted AR display, which superimposed information over the surface of the part indicating where measurements were to be taken and in what sequence. Experiments were performed to compare the AR-aided inspection method to traditional computer-aided and manual inspection methods. Results indicated that while all three methods were similar in terms of accuracy, AR-aided inspection was 2–3 times faster than the other methods. Additionally, while inspection time increased with part shape complexity for the manual and computer-aided methods, it was virtually unaffected by part shape complexity for AR-aided inspection. These results indicate the potential use of augmented reality to improve the performance of manual inspection tasks substantially when task-specific information must be conveyed to the operator. © 1999 John Wiley & Sons, Inc.     

Multimedia augmented reality information system for museum guidance

This paper introduces the design and implementation of an augmented reality-based guidance system used for museum guidance. The objective of this study is to deliver the new museum guidance with intuitive and user-friendly interactions between man and machine based on computer vision technique in ubiquitous computing environment. In our design, a camera is used to capture hand motions and images of a printed guide, which are analyzed instantaneously to parse the designated actions the guidance system promptly responds with. The system can display and illustrate contextual information about 3D models of museum artifacts and multimedia materials to users in real time to compliment the museum experience. Moreover, the performance of the approach we proposed herein has been compared with that of the other alternatives with which the guidance system has also been installed for similar exhibitions. As a result, our proposed approach outperforms the other applications. The advantage of the proposed approach includes the exclusion of common tactile peripheral devices, for example, keyboard and mouse, which accordingly reduces hardware-related costs and reduces the risk of contamination in a public environment.     

A spatially augmented reality sketching interface for architectural daylighting design

We present an application of interactive global illumination and spatially augmented reality to architectural daylight modeling that allows designers to explore alternative designs and new technologies for improving the sustainability of their buildings. Images of a model in the real world, captured by a camera above the scene, are processed to construct a virtual 3D model. To achieve interactive rendering rates, we use a hybrid rendering technique, leveraging radiosity to simulate the interreflectance between diffuse patches and shadow volumes to generate per-pixel direct illumination. The rendered images are then projected on the real model by four calibrated projectors to help users study the daylighting illumination. The virtual heliodon is a physical design environment in which multiple designers, a designer and a client, or a teacher and students can gather to experience animated visualizations of the natural illumination within a proposed design by controlling the time of day, season, and climate. Furthermore, participants may interactively redesign the geometry and materials of the space by manipulating physical design elements and see the updated lighting simulation.     

An augmented reality human–computer interface for object localization in a cognitive vision system

The European Cognitive Vision project VAMPIRE uses mobile AR-kits to interact with a visual active memory for teaching and retrieval purposes. This paper describes concept and technical realization of the used mobile AR-kits and discusses interactive learning and retrieval in office environments, and the active memory infrastructure. The focus is on 3D interaction for pointing in a scene coordinate system. This is achieved by 3D augmented pointing, which combines inside-out tracking for head pose recovery and 3D stereo human–computer interaction. Experimental evaluation shows that the accuracy of this 3D cursor is within a few centimeters, which is sufficient to point at an object in an office. Finally, an application of the cursor in VAMPIRE is presented, where in addition to the mobile system, at least one stationary active camera is used to obtain different views of an object. There are many potential applications, for example an improved view-based object recognition.     

Using augmented reality to treat phobias

Virtual reality (VR) is useful for treating several psychological problems, including phobias such as fear of flying, agoraphobia, claustrophobia, and phobia to insects and small animals. We believe that augmented reality (AR) could also be used to treat some psychological disorders. AR and VR share some advantages over traditional treatments. However, AR gives a greater feeling of presence (the sensation of being there) and reality judgment (judging an experience as real) than VR because the environment and the elements the patient uses to interact with the application are real. Moreover, in AR users see their own hands, feet, and so on, whereas VR only simulates this experience. With these differences in mind, the question arises as to the kinds of psychological treatments AR and VR are most suited for. In our system, patients see their own hands, feet, and so on. They can touch the table that animals are crossing or seeing their feet while the animals are running on the floor. They can also hold a marker with a dead spider or cockroach or pick up a flyswatter, a can of insecticide, or a dustpan.     

Indirect augmented reality

Developing augmented reality (AR) applications for mobile devices and outdoor environments has historically required a number of technical trade-offs related to tracking. One approach is to rely on computer vision which provides very accurate tracking, but can be brittle, and limits the generality of the application. Another approach is to rely on sensor-based tracking which enables widespread use, but at the cost of generally poor tracking performance. In this paper we present and evaluate a new approach, which we call Indirect AR, that enables perfect alignment of virtual content in a much greater number of application scenarios.To achieve this improved performance we replace the live camera view used in video see through AR with a previously captured panoramic image. By doing this we improve the perceived quality of the tracking while still maintaining a similar overall experience. There are some limitations of this technique, however, related to the use of panoramas. We evaluate these boundaries conditions on both a performance and experiential basis through two user studies. The result of these studies indicates that users preferred Indirect AR over traditional AR in most conditions, and when conditions do degrade to the point the experience changes, Indirect AR can still be a very useful tool in many outdoor application scenarios.     

Calibration-free augmented reality

Camera calibration and the acquisition of Euclidean 3D measurements have so far been considered necessary requirements for overlaying three-dimensional graphical objects with live video. We describe a new approach to video-based augmented reality that avoids both requirements: it does not use any metric information about the calibration parameters of the camera or the 3D locations and dimensions of the environment's objects. The only requirement is the ability to track across frames at least four fiducial points that are specified by the user during system initialization and whose world coordinates are unknown. Our approach is based on the following observation: given a set of four or more noncoplanar 3D points, the projection of all points in the set can be computed as a linear combination of the projections of just four of the points. We exploit this observation by: tracking regions and color fiducial points at frame rate; and representing virtual objects in a non-Euclidean, affine frame of reference that allows their projection to be computed as a linear combination of the projection of the fiducial points. Experimental results on two augmented reality systems, one monitor-based and one head-mounted, demonstrate that the approach is readily implementable, imposes minimal computational and hardware requirements, and generates real-time and accurate video overlays even when the camera parameters vary dynamically     

Hand gesture guided robot-assisted surgery based on a direct augmented reality interface

Radiofrequency (RF) ablation is a good alternative to hepatic resection for treatment of liver tumors. However, accurate needle insertion requires precise hand-eye coordination and is also affected by the difficulty of RF needle navigation. This paper proposes a cooperative surgical robot system, guided by hand gestures and supported by an augmented reality (AR)-based surgical field, for robot-assisted percutaneous treatment. It establishes a robot-assisted natural AR guidance mechanism that incorporates the advantages of the following three aspects: AR visual guidance information, surgeon's experiences and accuracy of robotic surgery. A projector-based AR environment is directly overlaid on a patient to display preoperative and intraoperative information, while a mobile surgical robot system implements specified RF needle insertion plans. Natural hand gestures are used as an intuitive and robust method to interact with both the AR system and surgical robot. The proposed system was evaluated on a mannequin model. Experimental results demonstrated that hand gesture guidance was able to effectively guide the surgical robot, and the robot-assisted implementation was found to improve the accuracy of needle insertion. This human–robot cooperative mechanism is a promising approach for precise transcutaneous ablation therapy.     

An augmented reality system to guide radio‐frequency tumour ablation

Radio‐frequency ablation is a difficult operative task that requires a precise needle positioning in the centre of the pathology. This article presents an augmented reality system for hepatic therapy guidance that superimposes in real‐time 3D reconstructions (from CT acquisition) and a virtual model of the needle on external views of a patient. The superimposition of reconstructed models is performed with a 3D/2D registration based on radio‐opaque markers stuck on to the patient's skin. The characteristics of the problem (accuracy, robustness and time processing) led us to develop automatic procedures to extract and match the markers and to track the needle in real time. Experimental studies confirmed that our algorithms are robust and reliable. Preliminary experiments conducted on a human abdomen phantom showed that our system is highly accurate (needle positioning error within 3 mm) and enables the surgeon to reach a target in less than 1 minute on average. Our next step will be to perform an in vivo evaluation. Copyright © 2005 John Wiley & Sons, Ltd.     

RETRACTED ARTICLE: Multimodal interface interaction design model based on dynamic augmented reality

New channels, new carriers continue to produce new data types, how to deal with complex and volatile multi-source data under IoT big data environment has become an important issue. Information perception is the basic content of the development of things, but also an important means to achieve information perception. Information interaction is the basic condition for the realization of Internet of things, but also the basic technology of interconnection. The information sensing of the Internet of Things is mainly the source of information provided by it, which is also the application base of the Internet of Things itself. To effectively process the IoT data, this paper proposes the multimodal intelligent data analyzing paradigm for Internet of things systems. Our proposed multiple game theory based data analyzing paradigm will guarantee the optimal combination of the mentioned model to construct the effect and robust analyzing pattern. We construct the optimal combination of Bayesian model, Decision Tree, Random Forest, Logistic Regression, SVM, Adaboost, Markov and Neural Network under the multiple game framework. Several new ideas are proposed to enhance the traditional Nash Equilibrium. Performance of the proposed method is validated through the simulation compared with the other state-of-the-art algorithms.

     

Handling occlusions in video‐based augmented reality using depth information

Augmented Reality (AR) composes virtual objects with real scenes in a mixed environment where human–computer interaction has more semantic meanings. To seamlessly merge virtual objects with real scenes, correct occlusion handling is a significant challenge. We present an approach to separate occluded objects in multiple layers by utilizing depth, color, and neighborhood information. Scene depth is obtained by stereo cameras and two Gaussian local kernels are used to represent color, spatial smoothness. These three cues are intelligently fused in a probability framework, where the occlusion information can be safely estimated. We apply our method to handle occlusions in video‐based AR where virtual objects are simply overlapped on real scenes. Experiment results show the approach can correctly register virtual and real objects in different depth layers, and provide a spatial‐awareness interaction environment. Copyright © 2009 John Wiley & Sons, Ltd.     

An augmented reality training platform for assembly and maintenance skills

Training technicians to acquire new maintenance and assembly skills is important for various industries. Because maintenance and assembly tasks can be very complex, training technicians to efficiently perform new skills is challenging. Training of this type can be supported by Augmented Reality, a powerful industrial training technology that directly links instructions on how to perform the service tasks to the machine parts that require processing. Because of the increasing complexity of maintenance tasks, it is not sufficient to train the technicians in task execution. Instead, technicians must be trained in the underlying skills—sensorimotor and cognitive—that are necessary for the efficient acquisition and performance of new maintenance operations.These facts illustrate the need for efficient training systems for maintenance and assembly skills that accelerate the technicians’ acquisition of new maintenance procedures. Furthermore, these systems should improve the adjustment of the training process for new training scenarios and enable the reuse of worthwhile existing training material. In this context, we have developed a novel concept and platform for multimodal Augmented Reality-based training of maintenance and assembly skills, which includes sub-skill training and the evaluation of the training system. Because procedural skills are considered as the most important skills for maintenance and assembly operations, we focus on these skills and the appropriate methods for improving them.     

Virtual worlds,augmented reality,blended reality

The internet could be considered just another collection of digital tools, but it has had a more profound impact on how we live, work and play than most people would have predicted. The internet makes everywhere local. For the exchange of ideas and data, it does not matter where you are on the planet. For an industrialized society, or for one forming to emulate existing ones, this creates an interesting situation. If people can work and communicate at any distance the current organization of corporations, cities and even countries changes. However, the legacy set of tools available to communicate at distance miss many of the vital elements we use as humans. Telephones, email and web pages filter out much of who we are. It is very easy, therefore, to dismiss any new technology as more of the same and to stick with the legacy tools. We lose the notion of space, location, size, physical presence and nonverbal communication in the “traditional” internet tools. This tension between what we currently do and what we need to do acts as a flashpoint that causes new inventions to emerge.The internet helps spread these inventions, and acts as the expression of them. These evolutionary steps in our communication are not restricted to the keyboard and screen, so not only challenge our organizations they also challenge our ideas of what the norm is for computing. Looking at the components of some already exciting technologies and combining them with a changing society becoming well versed in freedom the internet can offer, leads us to a very close future, etc. One with a blended reality where digital and physical mixes in combinations that suit us as people. The future is already here we just have to join it.     

System satisfaction survey for the App to integrate search and augmented reality with geographical information technology

Microsystem Technologies - Digital humanities are unfamiliar words in Taiwan. With the popularity of digital technology and mobile computing, Digital Human Geography in both theory and practice...     

Webizing mobile augmented reality content

This paper presents a content structure for building mobile augmented reality (AR) applications in HTML5 to achieve a clean separation of the mobile AR content and the application logic for scaling as on the Web. We propose that the content structure contains the physical world as well as virtual assets for mobile AR applications as document object model (DOM) elements and that their behaviour and user interactions are controlled through DOM events by representing objects and places with a uniform resource identifier. Our content structure enables mobile AR applications to be seamlessly developed as normal HTML documents under the current Web eco-system.     

Collaborative visualization in augmented reality

The authors discuss Studierstube, a low-cost augmented reality system. The system features true stereoscopy, 3D interaction, individual viewpoints and customized views for multiple users, and unhindered natural collaboration     

Calibration-free augmented reality in perspective

This paper deals with video-based augmented reality and proposes an algorithm for augmenting a real video sequence with views of graphics objects without metric calibration of the video camera by representing the motion of the video camera in projective space. A virtual camera, by which views of graphics objects are generated, is attached to a real camera by specifying image locations of the world coordinate system of the virtual world. The virtual camera is decomposed into calibration and motion components in order to make full use of graphics tools. The projective motion of the real camera recovered from image matches has the function of transferring the virtual camera and makes the virtual camera move according to the motion of the real camera. The virtual camera also follows the change of the internal parameters of the real camera. This paper shows the theoretical and experimental results of our application of nonmetric vision to augmented reality     

移动增强现实框架系统

在已有的移动增强现实( MAR)系统中存在开发困难,虚拟现实( AR)效果观察缺乏通用性的问题,提出了一种MAR系统的通用框架,该框架包括三个部分：为智能移动设备用户所使用的通用MAR观察器、为MAR应用提供网络服务的MAR服务器,以及为开发人员提供了MAR应用定制器。该框架结合高通的Vuforia SDK针对平面图像实现了MAR的应用定制和通用AR效果浏览。框架为开发人员提供了无代码开发功能,为移动用户提供了遍览一类AR应用的通用观察器。实验结果表明该MAR框架系统降低了MAR应用的开发门槛与时间费用,大大提高了开发效率。