Online tracking of outdoor lighting variations for augmented reality with moving cameras

In augmented reality, one of key tasks to achieve a convincing visual appearance consistency between virtual objects and video scenes is to have a coherent illumination along the whole sequence. As outdoor illumination is largely dependent on the weather, the lighting condition may change from frame to frame. In this paper, we propose a full image-based approach for online tracking of outdoor illumination variations from videos captured with moving cameras. Our key idea is to estimate the relative intensities of sunlight and skylight via a sparse set of planar feature-points extracted from each frame. To address the inevitable feature misalignments, a set of constraints are introduced to select the most reliable ones. Exploiting the spatial and temporal coherence of illumination, the relative intensities of sunlight and skylight are finally estimated by using an optimization process. We validate our technique on a set of real-life videos and show that the results with our estimations are visually coherent along the video sequences.     

基于图像的点云建模及其真实感绘制

从视觉凸壳的理论出发,提出了一种基于图像的真实物体的点云建模和绘制方法。在建模方面,首先从不同视点采集目标物体的图像,然后对采样图像所形成的视觉凸壳进行均匀的点采样。同时,利用等间隔索引表来组织每幅采样图像的轮廓边,从而提高了建模效率。在绘制方面,首先分析目标物体在各种不同光照条件下的图像,并得到点云模型的离散反射属性,然后通过优化的插值方法实现真实感的绘制效果。实验表明,该方法的建模速度快,绘制结果具有很强的真实感。     

Algorithm for photorealistic rendering of transparent optically active isotropic crystals

Programming and Computer Software - This paper is devoted to the problem of photorealistic rendering of optically active crystals. Optical activity (rotation) is turning of the polarization plane...     

A low-cost photorealistic CG dataset rendering pipeline for facial landmark localization

Multimedia Tools and Applications - Face analysis has been a hot research field in computer vision for decades. The dataset is of vital importance for modern machine learning methods. The paper...     

使用GPU加速渲染真实感水波效果的研究

为了提高水面波动效果模拟的实时性并保持真实感,提出一种基于可编程图形处理器(GPU)的向量代数运算模型,并使用该模型求解水波动方程以加速水波仿真过程.给出了模型的数据结构定义和数据操作设计,对二维水波微分方程进行离散化处理,将其表达为矩阵矢量相乘的形式,再采取共轭梯度法进行求解,以获得代表水面的高度图.通过高度图信息来生成法线,从而获得水面反射和折射效果.实验结果表明,该算法充分发挥了GPU的高速性能优势来求解复杂的水面波动方程,能有效提高真实感水波纹模拟的效率.     

植物叶片表面质感建模与真实感绘制研究进展

植物表观的真实感建模是计算机图形学领域的重要研究内容,叶子作为植物最重要的器官,尤其受到广泛的关注。近年来,随着计算机硬件技术和图形算法的快速发展以及对植物叶子生理机理研究的不断深入,植物叶片表观质感建模和真实感绘制的研究取得了很多成果。植物叶片表面光学特性的采集与建模技术是其中的研究热点和难点。从植物叶片真实感质感模型的定义出发,介绍了近年来国内外在植物叶片表面质感建模和真实感绘制方面取得的最新研究进展,并给出详细的分析和总结。最后对该领域研究存在的问题和未来发展方向提出了一些看法。     

Indirect augmented reality

Developing augmented reality (AR) applications for mobile devices and outdoor environments has historically required a number of technical trade-offs related to tracking. One approach is to rely on computer vision which provides very accurate tracking, but can be brittle, and limits the generality of the application. Another approach is to rely on sensor-based tracking which enables widespread use, but at the cost of generally poor tracking performance. In this paper we present and evaluate a new approach, which we call Indirect AR, that enables perfect alignment of virtual content in a much greater number of application scenarios.To achieve this improved performance we replace the live camera view used in video see through AR with a previously captured panoramic image. By doing this we improve the perceived quality of the tracking while still maintaining a similar overall experience. There are some limitations of this technique, however, related to the use of panoramas. We evaluate these boundaries conditions on both a performance and experiential basis through two user studies. The result of these studies indicates that users preferred Indirect AR over traditional AR in most conditions, and when conditions do degrade to the point the experience changes, Indirect AR can still be a very useful tool in many outdoor application scenarios.     

Calibration-free augmented reality

Camera calibration and the acquisition of Euclidean 3D measurements have so far been considered necessary requirements for overlaying three-dimensional graphical objects with live video. We describe a new approach to video-based augmented reality that avoids both requirements: it does not use any metric information about the calibration parameters of the camera or the 3D locations and dimensions of the environment's objects. The only requirement is the ability to track across frames at least four fiducial points that are specified by the user during system initialization and whose world coordinates are unknown. Our approach is based on the following observation: given a set of four or more noncoplanar 3D points, the projection of all points in the set can be computed as a linear combination of the projections of just four of the points. We exploit this observation by: tracking regions and color fiducial points at frame rate; and representing virtual objects in a non-Euclidean, affine frame of reference that allows their projection to be computed as a linear combination of the projection of the fiducial points. Experimental results on two augmented reality systems, one monitor-based and one head-mounted, demonstrate that the approach is readily implementable, imposes minimal computational and hardware requirements, and generates real-time and accurate video overlays even when the camera parameters vary dynamically     

Designing augmented reality for the classroom

Augmented reality (AR) has recently received a lot of attention in education. Multiple AR systems for learning have been developed and tested through empirical studies often conducted in lab settings. While lab studies can be insightful, they leave out the complexity of a classroom environment. We developed three AR learning environments that have been used in genuine classroom contexts, some of them being now part of classroom regular practices. These systems and the learning activities they provide have been co-designed with teachers, for their own classrooms, through multiple cycles of prototyping and testing. We present here the features that emerged from these co-design cycles and abstract them into design principles.     

Mobile augmented reality for environmental monitoring

In response to dramatic changes in the environment, and supported by advances in wireless networking, pervasive sensor networks have become a common tool for environmental monitoring. However, tools for on-site visualization and interactive exploration of environmental data are still inadequate for domain experts. Current solutions are generally limited to tabular data, basic 2D plots, or standard 2D GIS tools designed for the desktop and not adapted to mobile use. In this paper, we introduce a novel augmented reality platform for 3D mobile visualization of environmental data. Following a user-centered design approach, we analyze processes, tasks, and requirements of on-site visualization tools for environmental experts. We present our multilayer infrastructure and the mobile augmented reality platform that leverages visualization of georeferenced sensor measurement and simulation data in a seamless integrated view of the environment.     

Real-time coherent stylization for augmented reality

This paper presents a new stylized augmented reality (AR) framework which can generate line drawing and abstracted shading styles. In comparison with the state-of-art work, our framework can significantly improve both the visual immersion of a single frame and the temporal coherence of augmented video streams in real time. In our framework, we first render virtual objects over the input camera images and then uniformly process the combined contents with stylization techniques. For generating line drawing stylization, we first propose a specially designed shading method to render the virtual objects, and then use an adapted Flow-based anisotropic Difference-of-Gaussion (FDoG) filter to yield the high-quality line drawing effect. For generating the abstracted stylization, a focus-guided diffusion filter and a soft color quantization operator are sequentially applied to the augmented image, and then the processed result is combined with the detected edges to produce the final abstraction effect. The presented algorithms are all sympathetic to highly parallel processing, allowing a real-time performance on contemporary graphics hardware.     

Natural feature tracking for augmented reality

Natural scene features stabilize and extend the tracking range of augmented reality (AR) pose-tracking systems. We develop robust computer vision methods to detect and track natural features in video images. Point and region features are automatically and adaptively selected for properties that lead to robust tracking. A multistage tracking algorithm produces accurate motion estimates, and the entire system operates in a closed-loop that stabilizes its performance and accuracy. We present demonstrations of the benefits of using tracked natural features for AR applications that illustrate direct scene annotation, pose stabilization, and extendible tracking range. Our system represents a step toward integrating vision with graphics to produce robust wide-area augmented realities     

Information filtering for mobile augmented reality

The use of augmented reality (AR) techniques can revolutionize the way people interact with unfamiliar environments. By tracking the user's position and orientation, complicated spatial information can be registered against the real world. My colleagues and I are researching the problem of developing mobile AR systems to be worn by individual users operating in large, complicated environments such as cities. However, an urban environment is extremely complicated. It is populated by large numbers of buildings, each of which can have numerous facts stored about it. Therefore, it is easy for a user to experience information overload. This problem is illustrated. To minimize problems of information overload, we have begun to develop algorithms for information filtering. These tools automatically restrict the amount of information displayed     

Accurate overlaying for mobile augmented reality

Mobile augmented reality requires accurate alignment of virtual information with objects visible in the real world. We describe a system for mobile communications to be developed to meet these strict alignment criteria using a combination of computer vision, inertial tracking and low-latency rendering techniques. A prototype low-power and low-latency renderer using an off-the-shelf 3D card is discussed.     

Live three-dimensional content for augmented reality

We describe an augmented reality system for superimposing three-dimensional (3-D) live content onto two-dimensional fiducial markers in the scene. In each frame, the Euclidean transformation between the marker and the camera is estimated. The equivalent virtual view of the live model is then generated and rendered into the scene at interactive speeds. The 3-D structure of the model is calculated using a fast shape-from-silhouette algorithm based on the outputs of 15 cameras surrounding the subject. The novel view is generated by projecting rays through each pixel of the desired image and intersecting them with the 3-D structure. Pixel color is estimated by taking a weighted sum of the colors of the projections of this 3-D point in nearby real camera images. Using this system, we capture live human models and present them via the augmented reality interface at a remote location. We can generate 384/spl times/288 pixel images of the models at 25 fps, with a latency of <100 ms. The result gives the strong impression that the model is a real 3-D part of the scene.     

Virtual and augmented reality 2020

With roots going back to Ivan Sutherland's research in the 1960s, virtual reality (VR) reached a plateau in the early 1990s when the promise was demonstrable in universities and research laboratories. Although we all knew we had a long way to go, hype overtook the field, leading to impossible expectations. Fortunately, a few years later the Internet became the latest hot topic, leaving VR researchers free to go back to work. Almost a decade later, VR has gone from “almost there” to “barely there”. That's a big step! Perhaps by 2020 we will be “pretty well there”! In this article, I venture a few remarks about progress over the next two decades. However, the safest prediction is that unanticipated science and technology advances will make this forecast seem overly conservative     

Tracking for augmented reality on wearable computers

Wearable computers afford a degree of mobility that makes tracking for augmented reality difficult. This paper presents a novel object-centric tracking architecture for presenting augmented reality media in spatial relationships to objects, regardless of the objects' positions or motions in the world. The advance this system provides is the ability to sense and integrate new features into its tracking database, thereby extending the tracking region automatically. A lazy evaluation of the structure from motion problem uses images obtained from a single calibrated moving camera and applies recursive filtering to identify and estimate the 3D positions of new features. We evaluate the performance of two filters; a classic Extended Kalman Filter (EKF) and a filter based on a Recursive-Average of Covariances (RAC). Some implementation issues and results are discussed in conclusion.     

保持视觉稳定性的增强现实注册算法

提出了一种保持视觉稳定性的增强现实注册算法,特征点在视频中的初匹配基于随机树检索统计得到,初步匹配的特征点采用半几何限制结合单应矩阵迭代的方法进行错误匹配排除,随后采用光流法跟踪特征点,最后根据共面POSIT算法计算出物体世界坐标系到摄像机坐标系的变换矩阵。由于采用了特征点跟踪方法,在一定程度上有效的防止了虚拟物体在视觉上的抖动。     

Hybrid tracking for outdoor augmented reality applications

We've developed a fully mobile, wearable AR system that combines a vision-based tracker (primarily software algorithms) that uses natural landmarks, with an inertial tracker (custom hardware and firmware) based on silicon micromachined accelerometers and gyroscopes. Unlike other vision-based and hybrid systems, both components recover the full 6 DOF pose. Fusing the two tracking subsystems gives us the benefits of both technologies, while the sensors' complementary nature helps overcome sensor-specific deficiencies. Our system is tailored to affordable, lightweight, energy-efficient mobile AR applications for urban environments, especially the historic centers of European cities.