Comparing augmented reality visualization methods for assembly procedures

Virtual Reality - Assembly processes require now more than ever a systematic way to improve efficiency complying with increasing product demand. Several industrial scenarios have been using...     

Managing complex augmented reality models.

Mobile augmented reality requires georeferenced data to present world-registered overlays. To cover a wide area and all artifacts and activities, a database containing this information must be created, stored, maintained, delivered, and finally used by the client application. We present a data model and a family of techniques to address these needs.     

Indirect augmented reality

Developing augmented reality (AR) applications for mobile devices and outdoor environments has historically required a number of technical trade-offs related to tracking. One approach is to rely on computer vision which provides very accurate tracking, but can be brittle, and limits the generality of the application. Another approach is to rely on sensor-based tracking which enables widespread use, but at the cost of generally poor tracking performance. In this paper we present and evaluate a new approach, which we call Indirect AR, that enables perfect alignment of virtual content in a much greater number of application scenarios.To achieve this improved performance we replace the live camera view used in video see through AR with a previously captured panoramic image. By doing this we improve the perceived quality of the tracking while still maintaining a similar overall experience. There are some limitations of this technique, however, related to the use of panoramas. We evaluate these boundaries conditions on both a performance and experiential basis through two user studies. The result of these studies indicates that users preferred Indirect AR over traditional AR in most conditions, and when conditions do degrade to the point the experience changes, Indirect AR can still be a very useful tool in many outdoor application scenarios.     

Calibration-free augmented reality

Camera calibration and the acquisition of Euclidean 3D measurements have so far been considered necessary requirements for overlaying three-dimensional graphical objects with live video. We describe a new approach to video-based augmented reality that avoids both requirements: it does not use any metric information about the calibration parameters of the camera or the 3D locations and dimensions of the environment's objects. The only requirement is the ability to track across frames at least four fiducial points that are specified by the user during system initialization and whose world coordinates are unknown. Our approach is based on the following observation: given a set of four or more noncoplanar 3D points, the projection of all points in the set can be computed as a linear combination of the projections of just four of the points. We exploit this observation by: tracking regions and color fiducial points at frame rate; and representing virtual objects in a non-Euclidean, affine frame of reference that allows their projection to be computed as a linear combination of the projection of the fiducial points. Experimental results on two augmented reality systems, one monitor-based and one head-mounted, demonstrate that the approach is readily implementable, imposes minimal computational and hardware requirements, and generates real-time and accurate video overlays even when the camera parameters vary dynamically     

Designing augmented reality for the classroom

Augmented reality (AR) has recently received a lot of attention in education. Multiple AR systems for learning have been developed and tested through empirical studies often conducted in lab settings. While lab studies can be insightful, they leave out the complexity of a classroom environment. We developed three AR learning environments that have been used in genuine classroom contexts, some of them being now part of classroom regular practices. These systems and the learning activities they provide have been co-designed with teachers, for their own classrooms, through multiple cycles of prototyping and testing. We present here the features that emerged from these co-design cycles and abstract them into design principles.     

Mobile augmented reality for environmental monitoring

In response to dramatic changes in the environment, and supported by advances in wireless networking, pervasive sensor networks have become a common tool for environmental monitoring. However, tools for on-site visualization and interactive exploration of environmental data are still inadequate for domain experts. Current solutions are generally limited to tabular data, basic 2D plots, or standard 2D GIS tools designed for the desktop and not adapted to mobile use. In this paper, we introduce a novel augmented reality platform for 3D mobile visualization of environmental data. Following a user-centered design approach, we analyze processes, tasks, and requirements of on-site visualization tools for environmental experts. We present our multilayer infrastructure and the mobile augmented reality platform that leverages visualization of georeferenced sensor measurement and simulation data in a seamless integrated view of the environment.     

Information filtering for mobile augmented reality

The use of augmented reality (AR) techniques can revolutionize the way people interact with unfamiliar environments. By tracking the user's position and orientation, complicated spatial information can be registered against the real world. My colleagues and I are researching the problem of developing mobile AR systems to be worn by individual users operating in large, complicated environments such as cities. However, an urban environment is extremely complicated. It is populated by large numbers of buildings, each of which can have numerous facts stored about it. Therefore, it is easy for a user to experience information overload. This problem is illustrated. To minimize problems of information overload, we have begun to develop algorithms for information filtering. These tools automatically restrict the amount of information displayed     

Natural feature tracking for augmented reality

Natural scene features stabilize and extend the tracking range of augmented reality (AR) pose-tracking systems. We develop robust computer vision methods to detect and track natural features in video images. Point and region features are automatically and adaptively selected for properties that lead to robust tracking. A multistage tracking algorithm produces accurate motion estimates, and the entire system operates in a closed-loop that stabilizes its performance and accuracy. We present demonstrations of the benefits of using tracked natural features for AR applications that illustrate direct scene annotation, pose stabilization, and extendible tracking range. Our system represents a step toward integrating vision with graphics to produce robust wide-area augmented realities     

Real-time coherent stylization for augmented reality

This paper presents a new stylized augmented reality (AR) framework which can generate line drawing and abstracted shading styles. In comparison with the state-of-art work, our framework can significantly improve both the visual immersion of a single frame and the temporal coherence of augmented video streams in real time. In our framework, we first render virtual objects over the input camera images and then uniformly process the combined contents with stylization techniques. For generating line drawing stylization, we first propose a specially designed shading method to render the virtual objects, and then use an adapted Flow-based anisotropic Difference-of-Gaussion (FDoG) filter to yield the high-quality line drawing effect. For generating the abstracted stylization, a focus-guided diffusion filter and a soft color quantization operator are sequentially applied to the augmented image, and then the processed result is combined with the detected edges to produce the final abstraction effect. The presented algorithms are all sympathetic to highly parallel processing, allowing a real-time performance on contemporary graphics hardware.     

Live three-dimensional content for augmented reality

We describe an augmented reality system for superimposing three-dimensional (3-D) live content onto two-dimensional fiducial markers in the scene. In each frame, the Euclidean transformation between the marker and the camera is estimated. The equivalent virtual view of the live model is then generated and rendered into the scene at interactive speeds. The 3-D structure of the model is calculated using a fast shape-from-silhouette algorithm based on the outputs of 15 cameras surrounding the subject. The novel view is generated by projecting rays through each pixel of the desired image and intersecting them with the 3-D structure. Pixel color is estimated by taking a weighted sum of the colors of the projections of this 3-D point in nearby real camera images. Using this system, we capture live human models and present them via the augmented reality interface at a remote location. We can generate 384/spl times/288 pixel images of the models at 25 fps, with a latency of <100 ms. The result gives the strong impression that the model is a real 3-D part of the scene.     

Accurate overlaying for mobile augmented reality

Mobile augmented reality requires accurate alignment of virtual information with objects visible in the real world. We describe a system for mobile communications to be developed to meet these strict alignment criteria using a combination of computer vision, inertial tracking and low-latency rendering techniques. A prototype low-power and low-latency renderer using an off-the-shelf 3D card is discussed.     

Virtual and augmented reality 2020

With roots going back to Ivan Sutherland's research in the 1960s, virtual reality (VR) reached a plateau in the early 1990s when the promise was demonstrable in universities and research laboratories. Although we all knew we had a long way to go, hype overtook the field, leading to impossible expectations. Fortunately, a few years later the Internet became the latest hot topic, leaving VR researchers free to go back to work. Almost a decade later, VR has gone from “almost there” to “barely there”. That's a big step! Perhaps by 2020 we will be “pretty well there”! In this article, I venture a few remarks about progress over the next two decades. However, the safest prediction is that unanticipated science and technology advances will make this forecast seem overly conservative     

State-of-the-art virtual reality hardware for computer-aided design

Virtual reality is an exciting new field, which may offer significant improvements in the efficiency of human-computer interface for a wide variety of computer applications. This paper serves to identify technology developments in virtual reality hardware devices that may allow for development of a more efficient user interface for computer-aided design packages for mechanical design functions. Presented is a survey of the current state of the art in the rapidly changing field of virtual reality hardware devices. The devices are broken down into classes that are related to user interface features that may improve the efficiency of computer-aided design functions. Classes of virtual reality hardware devices are identified needing future research and development to implement the most effective virtual reality computer-aided design interface.     

Pose estimation for augmented reality applications using genetic algorithm.

This paper describes a genetic algorithm that tackles the pose-estimation problem in computer vision. Our genetic algorithm can find the rotation and translation of an object accurately when the three-dimensional structure of the object is given. In our implementation, each chromosome encodes both the pose and the indexes to the selected point features of the object. Instead of only searching for the pose as in the existing work, our algorithm, at the same time, searches for a set containing the most reliable feature points in the process. This mismatch filtering strategy successfully makes the algorithm more robust under the presence of point mismatches and outliers in the images. Our algorithm has been tested with both synthetic and real data with good results. The accuracy of the recovered pose is compared to the existing algorithms. Our approach outperformed the Lowe's method and the other two genetic algorithms under the presence of point mismatches and outliers. In addition, it has been used to estimate the pose of a real object. It is shown that the proposed method is applicable to augmented reality applications.     

Inverse lighting and photorealistic rendering for augmented reality

We present a practical and robust photorealistic rendering pipeline for augmented reality. We solve the real world lighting conditions from observations of a diffuse sphere or a rotated marker. The solution method is based on l ₁-regularized least squares minimization, yielding a sparse set of light sources readily usable with most rendering methods. The framework also supports the use of more complex light source representations. Once the lighting conditions are solved, we render the image using modern real-time rendering methods such as shadow maps with variable softness, ambient occlusion, advanced BRDF’s and approximate reflections and refractions. Finally, we perform post-processing on the resulting images in order to match the various aberrations and defects typically found in the underlying real-world video.     

Tracking for augmented reality on wearable computers

Wearable computers afford a degree of mobility that makes tracking for augmented reality difficult. This paper presents a novel object-centric tracking architecture for presenting augmented reality media in spatial relationships to objects, regardless of the objects' positions or motions in the world. The advance this system provides is the ability to sense and integrate new features into its tracking database, thereby extending the tracking region automatically. A lazy evaluation of the structure from motion problem uses images obtained from a single calibrated moving camera and applies recursive filtering to identify and estimate the 3D positions of new features. We evaluate the performance of two filters; a classic Extended Kalman Filter (EKF) and a filter based on a Recursive-Average of Covariances (RAC). Some implementation issues and results are discussed in conclusion.     

Hybrid tracking for outdoor augmented reality applications

We've developed a fully mobile, wearable AR system that combines a vision-based tracker (primarily software algorithms) that uses natural landmarks, with an inertial tracker (custom hardware and firmware) based on silicon micromachined accelerometers and gyroscopes. Unlike other vision-based and hybrid systems, both components recover the full 6 DOF pose. Fusing the two tracking subsystems gives us the benefits of both technologies, while the sensors' complementary nature helps overcome sensor-specific deficiencies. Our system is tailored to affordable, lightweight, energy-efficient mobile AR applications for urban environments, especially the historic centers of European cities.     

保持视觉稳定性的增强现实注册算法

提出了一种保持视觉稳定性的增强现实注册算法,特征点在视频中的初匹配基于随机树检索统计得到,初步匹配的特征点采用半几何限制结合单应矩阵迭代的方法进行错误匹配排除,随后采用光流法跟踪特征点,最后根据共面POSIT算法计算出物体世界坐标系到摄像机坐标系的变换矩阵。由于采用了特征点跟踪方法,在一定程度上有效的防止了虚拟物体在视觉上的抖动。     

Orientation tracking for outdoor augmented reality registration

Our work stems from a program focused on developing tracking technologies for wide-area augmented realities in unprepared outdoor environments. Other participants in the Defense Advanced Research Projects Agency (Darpa) funded Geospatial Registration of Information for Dismounted Soldiers (Grids) program included University of North Carolina at Chapel Hill and Raytheon. We describe a hybrid orientation tracking system combining inertial sensors and computer vision. We exploit the complementary nature of these two sensing technologies to compensate for their respective weaknesses. Our multiple-sensor fusion is novel in augmented reality tracking systems, and the results demonstrate its utility     

Optically based direct manipulation for augmented reality

Augmented reality (AR) constitutes a very powerful three-dimensional user interface for many “hands-on” application scenarios. To fully exploit the AR paradigm, the computer must not only augment the real world, but also accept feedback from it. In this paper, we present an optical approach for collecting such feedback by analyzing video sequences to track users and the objects they work with. Our system can be set up in any room after quickly placing a few known optical targets in the scene. We present two demonstration scenarios to illustrate the overall concept and potential of our approach and then discuss the research issues involved.