Foveated Real‐Time Ray Tracing for Head‐Mounted Displays

Head‐mounted displays with dense pixel arrays used for virtual reality applications require high frame rates and low latency rendering. This forms a challenging use case for any rendering approach. In addition to its ability of generating realistic images, ray tracing offers a number of distinct advantages, but has been held back mainly by its performance. In this paper, we present an approach that significantly improves image generation performance of ray tracing. This is done by combining foveated rendering based on eye tracking with reprojection rendering using previous frames in order to drastically reduce the number of new image samples per frame. To reproject samples a coarse geometry is reconstructed from a G‐Buffer. Possible errors introduced by this reprojection as well as parts that are critical to the perception are scheduled for resampling. Additionally, a coarse color buffer is used to provide an initial image, refined smoothly by more samples were needed. Evaluations and user tests show that our method achieves real‐time frame rates, while visual differences compared to fully rendered images are hardly perceivable. As a result, we can ray trace non‐trivial static scenes for the Oculus DK2 HMD at 1182 × 1464 per eye within the the VSync limits without perceived visual differences.     

基于光学映射虚物体的并行绘制

尽管当前基于全局光照模型的图形绘制方法可以渲染出高质量的图象 ,但因为其计算量巨大 ,难以适用于诸如建筑物漫游、虚拟现实等对绘制速度有严格要求的场合 .为此引入光学映射虚物体的概念 ,利用构建在联网PC机上的集群系统 ,并行创建反射和折射虚物体 ,然后利用集群中各节点的图形加速硬件 ,像处理实际三维物体一样绘制这些虚物体 ,可以快速地绘制出反射 /折射效果的图象 .实验结果证明 ,该方法利用 CU P的计算能力和图形硬件的加速特性实现了真实感图形的高性能绘制 ,特别适用于诸如建筑物漫游、计算机动画和虚拟现实等要求交互式绘制的场合     

真实感图形绘制技术研究

随着虚拟现实技术的发展,对各种图形绘制技术提出了越来越高的要求。好的图形绘制技术是虚拟现实、动态仿真、实时可视化、高品质动画等越来越多的图像、图形应用技术的实现基础。本文主要介绍了基于图像的绘制技术、基于点的绘制技术,以及基于图形与图像的混合绘制技术在原理和应用。     

基于游戏引擎的三维虚拟漫游系统实现

三维虚拟环境漫游是虚拟现实技术应用的基本方向之一,具有重要的研究意义与应用价值。基于成熟的游戏引擎技术,提出一个具有较强通用性和可扩展性的三维虚拟漫游系统开发框架,并从3D场景快速构建、3D场景实时绘制、高级视觉特效生成和基于物理的场景模拟等四个方面详细介绍了该框架的关键技术实现。通过对系统原型的测试表明,采用该开发框架所构建的三维虚拟漫游系统可在一般的普通主流PC机上达到较好的性能和表现效果。     

基于图象绘制的虚拟现实系统环境

虚拟现实的一个重要目标是使用计算机生成逼真的视觉世界,使用户可以对虚拟世界的客体进行交互式考察．虚拟现实的实现有两种方法．传统上,用三维图形学的方法实行建模和绘制．此方法需要繁琐的建模和昂贵的专用绘制硬件,而且图形绘制的质量和场景的复杂性受到很大的限制．基于图象的绘制是实现虚拟现实系统的一种新方法．它克服了三维图形方法的缺点．本文在总结已有技术的基础上,提出一个实现基于图象绘制的虚拟现实系统模型,并据此模型实现一个实验系统及相应的图象工具．     

基于图形与图像的混合绘制技术

随着虚拟现实技术的发展,对基于图形与图像的混合绘制技术提出了越来越高的要求。基于图形与图像的混合绘制技术是虚拟现实、动态仿真、实时可视化、高品质动画等越来越多的图像、图形应用技术的实现基础。该文对各种基于图形与图像的混合绘制技术进行分类综述、研究和展望。     

Efficient rendering of animated characters through optimized per‐joint impostors

In this paper, we present a new impostor‐based representation for 3D animated characters supporting real‐time rendering of thousands of agents. We maximize rendering performance by using a collection of pre‐computed impostors sampled from a discrete set of view directions. Our approach differs from previous work on view‐dependent impostors in that we use per‐joint rather than per‐character impostors. Our characters are animated by applying the joint rotations directly to the impostors, instead of choosing a single impostor for the whole character from a set of pre‐defined poses. This offers more flexibility in terms of animation clips, as our representation supports any arbitrary pose, and thus, the agent behavior is not constrained to a small collection of pre‐defined clips. Because our impostors are intended to be valid for any pose, a key issue is to define a proper boundary for each impostor to minimize image artifacts while animating the agents. We pose this problem as a variational optimization problem and provide an efficient algorithm for computing a discrete solution as a pre‐process. To the best of our knowledge, this is the first time a crowd rendering algorithm encompassing image‐based performance, small graphics processing unit footprint, and animation independence is proposed. Copyright © 2012 John Wiley & Sons, Ltd.     

三维模板跟踪的基准合成数据集构建及算法评估

三维模板跟踪旨在将预先构建的三维CAD模型与输入图像中的相应目标进行精确配准,在增强现实、机器人等领域具有重要的应用,也是计算机视觉领域的关键问题之一.近年来,三维模板跟踪的准确率和稳定性都得到了持续提升,但仅有少量的工作关注三维模板跟踪数据集的构建.随着深度学习的普及,各领域中大规模数据集的构建越来越被重视,为算法的...     

虚拟现实应用中的并行渲染技术

虚拟现实应用要求图形系统具备实时渲染复杂、精细场景的能力。基于PC机群的并行渲染系统具有性价比高、扩展性好的特点,适合虚拟现实应用。该文针对虚拟现实应用设计并实现了一个保留模式的sort—first并行实时渲染系统,对影响渲染性能的关键因素进行研究,给出具体实现步骤、试验验证数据和结论。应用表明,该方法能够实现系统的高效并行计算及虚拟现实应用中的复杂场景实时处理。     

Time‐critical rendering algorithm with incorporation of LoD,visibility culling and object impostor

Given enough CPU time, present graphics technology can render near‐photorealistic images. However, for real‐time graphics applications such as virtual reality systems, developers must make explicit programming decisions, trading off rendering quality for interactive update rates. In this papar we present a new algorithm for rendering complex 3D models at near‐interactive rates that can be used in virtual environments composed of static or dynamic scenes. The algorithm integretes the techniques of level of details (LoD), visibility computation and object impostor. The method is more suitable for very dynamic scenes with high depth complexity. We introduce a new criterion to identify the occluder and the occludee: the object that can be replaced by its LoD model and the one that can be replaced by its impostor. The efficiency of our algorithm is then illustrated by experimental results. Copyright © 2003 John Wiley & Sons, Ltd.     

High‐resolution 360 Video Foveated Stitching for Real‐time VR

In virtual reality (VR) applications, the contents are usually generated by creating a 360° Video panorama of a real‐world scene. Although many capture devices are being released, getting high‐resolution panoramas and displaying a virtual world in real‐time remains challenging due to its computationally demanding nature. In this paper, we propose a real‐time 360° Video foveated stitching framework, that renders the entire scene in different level of detail, aiming to create a high‐resolution panoramic Video in real‐time that can be streamed directly to the client. Our foveated stitching algorithm takes Videos from multiple cameras as input, combined with measurements of human visual attention (i.e. the acuity map and the saliency map), can greatly reduce the number of pixels to be processed. We further parallelize the algorithm using GPU to achieve a responsive interface and validate our results via a user study. Our system accelerates graphics computation by a factor of 6 on a Google Cardboard display.     

三维虚拟场景绘制加速技术综述

加速三维图形的绘制对三维虚拟场景的交互式生成有重要作用。该文结合三维图形绘制流水线，概括了提高绘制性能的途径，综述了可见性处理、细节层次方法、基于图象的绘制以及大型几何数据库的操纵等主要的绘制加速技术。     

基于点采样的大规模三维模型显示

大规模三维模型的实时显示一直是许多虚拟现实应用追求的目标之一,也是计算机图形学领域的研究热点之一。文中采用基于采样点的显示方法,把具有显示质量优势的传统三角形与显示速度快、简化表示能力强的采样点,通过八叉树有机组织起来;通过对基本算法做出有效的改进,并添加计算分解和重用、有效的数据管理等模块,以高效的实现方式完成了30,000个独立运动的虚拟人场景的实时显示。文中的方法也可处理其它场景,能直接应用于紧急疏散或影视、游戏等领域中大规模三维模型的显示,具有很好的应用前景。     

基于图象渲染的球面虚拟空间生成及实时漫游算法研究

在基于图象渲染（Image Based Rendering）技术的虚拟现实系统中,球面虚拟空间与柱面虚拟空间相比,具有不受上下观察范围限制等优点。特别是用于表现三维室内场景,球面虚拟空间比柱面虚拟空间优越得多。本文提出了球面虚拟空间生成,以及球面虚拟空间的实时漫游算法。     

可扩展的实时自然景物模拟算法

针对传统的粒子系统实时仿真存在只能针对单一自然景物模拟、计算耗时、图像不真实、算法复杂等问题,提出了一种基于粒子系统和图形处理器（GPU）加速通用可扩展的自然景物模拟算法。在该算法中,粒子的物理运动计算过程和渲染阶段完全由CPU转移至GPU,可以增加粒子数量和提高渲染速度;同时,在渲染过程中,可以较好地利用硬件支持的粒子图技术来改善渲染中粒子的外表,选择不同纹理,从而能够较方便地模拟不同的自然景物。最后,在GPU上实现了雪花、喷泉、烟花、瀑布等模拟,算法充分利用了GPU的多通道并行处理性和可编程性,提高了自然景物模拟的实时性,可运用于虚拟现实系统。     

RayCaching: Amortized Isosurface Rendering for Virtual Reality

Real-time virtual reality requires efficient rendering methods to deal with high- resolution stereoscopic displays and low latency head-tracking. Our proposed RayCaching method renders isosurfaces of large volume datasets by amortizing raycasting over several frames and caching primary rays as small bricks that can be efficiently rasterized. An occupancy map in form of a clipmap provides level of detail and ensures that only bricks corresponding to visible points on the isosurface are being cached and rendered. Hard shadows and ambient occlusion from secondary rays are also accumulated and stored in the cache. Our method supports real-time isosurface rendering with dynamic isovalue and allows stereoscopic visualization and exploration of large volume datasets at framerates suitable for virtual reality applications.     

Presence and interaction in mixed reality environments

In this paper, we present a simple and robust mixed reality (MR) framework that allows for real-time interaction with virtual humans in mixed reality environments under consistent illumination. We will look at three crucial parts of this system: interaction, animation and global illumination of virtual humans for an integrated and enhanced presence. The interaction system comprises of a dialogue module, which is interfaced with a speech recognition and synthesis system. Next to speech output, the dialogue system generates face and body motions, which are in turn managed by the virtual human animation layer. Our fast animation engine can handle various types of motions, such as normal key-frame animations, or motions that are generated on-the-fly by adapting previously recorded clips. Real-time idle motions are an example of the latter category. All these different motions are generated and blended on-line, resulting in a flexible and realistic animation. Our robust rendering method operates in accordance with the previous animation layer, based on an extended for virtual humans precomputed radiance transfer (PRT) illumination model, resulting in a realistic rendition of such interactive virtual characters in mixed reality environments. Finally, we present a scenario that illustrates the interplay and application of our methods, glued under a unique framework for presence and interaction in MR.     

卡通风格增强现实系统

增强现实技术将计算机生成的虚拟物体叠加到真实场景中.在传统的AR系统中,虚拟物体和真实场景视觉上存在较为明显的差异,达不到虚拟物体和真实场景无缝结合的要求.本文将增强现实技术与非真实感的绘制技术有机的结合起来,减小这种视觉差异,研究并实现了卡通风格的增强现实系统.     

宽幅度摩擦力控制触觉再现系统

触觉再现能够表达虚拟物体的表面特征,提高虚拟现实系统的真实感,因而成为人机交互领域内的研究重点。本文将空气压膜效应与电磁激励相结合,提出一种新颖的摩擦力控制触觉再现方法,并研制一种宽幅度摩擦力控制触觉再现系统,其中电磁激励用于增加摩擦力控制范围。经过一系列触觉感知实验验证了本系统的有效性,实验结果表明本文提出的方法相对于传统的摩擦力控制方法可以实现更宽泛的摩擦力系数控制。     

GPU‐Based Spherical Light Field Rendering with Per‐Fragment Depth Correction

Image‐based rendering techniques are a powerful alternative to traditional polygon‐based computer graphics. This paper presents a novel light field rendering technique which performs per‐pixel depth correction of rays for high‐quality reconstruction. Our technique stores combined RGB and depth values in a parabolic 2D texture for every light field sample acquired at discrete positions on a uniform spherical setup. Image synthesis is implemented on the GPU as a fragment program which extracts the correct image information from adjacent cameras for each fragment by applying per‐pixel depth correction of rays. We show that the presented image‐based rendering technique provides a significant improvement compared to previous approaches. We explain two different rendering implementations which make use of a uniform parametrisation to minimise disparity problems and ensure full six degrees of freedom for virtual view synthesis. While one rendering algorithm implements an iterative refinement approach for rendering light fields with per pixel depth correction, the other approach employs a raycaster, which provides superior rendering quality at moderate frame rates. GPU based per‐fragment depth correction of rays, used in both implementations, helps reducing ghosting artifacts to a non‐noticeable amount and provides a rendering technique that performs without exhaustive pre‐processing for 3D object reconstruction and without real‐time ray‐object intersection calculations at rendering time.