Index rendering: hardware-efficient architecture for 3-D graphics in multimedia system

Real-time three-dimensional (3D) graphics is emerging rapidly in multimedia applications, but it suffers from requirements for huge computation, high bandwidth, and large buffer. In order to achieve hardware efficiency for 3D graphics rendering, we propose a novel approach named index rendering. The basic concept of index rendering is to realize a 3D rendering pipeline by using asynchronous multi-dataflows. Triangle information can be divided into several parts with each part capable of being transferred independently and asynchronously. Finally, all data are converged by the index to generate the final image. The index rendering approach can eliminate unnecessary operations in the traditional 3D graphics pipeline, the unnecessary operations are caused by the invisible pixels and triangles in the 3D scene. Previous work, deferred shading, eliminates the operations relating to invisible pixels, but it requires huge tradeoffs in bandwidth and buffer size. With index rendering, we can eliminate operations on both invisible pixels and triangles with fewer tradeoffs as compared with the deferred shading approach. The simulation and analysis results show that the index rendering approach can reduce 10%-70% of lighting operations when using the flat and Gouraud shading process and decrease 30%-95% when using Phong shading. Furthermore, it saves 70% of buffer size and 50%-70% of bandwidth compared with the deferred shading approach. The result also indicates that this approach of index rendering is especially suitable for low-cost portable rendering devices. Hence, index rendering is a hardware-efficient architecture for 3D graphics, and it makes rendering hardware more easily integrated into multimedia systems, especially system-on-a-chip (SOC) designs.     

协同系统中三维图形渲染的分布式架构

在三维协同系统中,三维图形的渲染效率和传输速度影响到了系统的整体效率。本文采用分布式架构保证三维图形在协同系统中的渲染和传输效率。把渲染好的三维图形以拍照的形式生成图片在网络中传输,打破了传统三维图形渲染对浏览器的限制以及对移动终端消耗巨大资源的局限。服务器端通过建立多线程TCP通信,既保证了用户交互操作的可靠性,同时又避免了多个用户同时操作三维图形时出现的操作二义性。最终实现用户与三维图形的实时交互,从而提高用户体验。      

Bandwidth reduction for remote navigation systems through view prediction and progressive transmission

Remote navigation systems, such as exploration of virtual 3D worlds or remote surgery, usually require higher bandwidth than the Internet connection commonly available at home. In this paper, we explore a set of techniques to reduce the bandwidth required by these applications. Our system consists of a client PC equipped with a graphics card, and a remote high-end server. The server hosts the remote environment and does the actual rendering of the scenes for several clients, and the new image is passed to them. This scheme is suitable when the data has a copyright or when its size may exceed the rendering capabilities of the client. The general scheme is the following: each time the position changes, the new view is predicted by both the client and the server and the difference information between the predicted view and the correct one is sent to the client. To reduce bandwidth we can improve the prediction method, and the transmission system. We present here two groups of techniques: First, a set of lossless methods which achieve reductions of up to a 9:1 ratio. These are a combination of a two-level forward warping, that takes advantage of spatial coherence, and a masking method, which allows to transmit only the information that really needs to be updated. Second, a set of lossy methods suitable for very low bandwidth environments which involve both progressive transmission and image reuse. They consider relevant parameters such as the number of pixels, the amount of information they provide, and their colour deviation in order to create a strategy for prioritizing the information transmission. This system allows to improve up to an additional 4:1 ratio. The quality of the generated images is very high, and often indistinguishable from the correct ones.     

Multistreaming of 3-D Scenes With Optimized Transmission and Rendering Scalability

Three-dimensional (3D) graphic scenes require considerable network bandwidth to be transmitted and computing power to be rendered on a user's terminal. Toward high-quality display in real time, we propose a sender-driven mechanism for streaming 3D scenes in a resource-constrained environment. In doing so, objects are encoded into multiresolutions to provide transmission and rendering scalability, and a weighted distortion metric is developed to measure the quality of a scene rendered with multiresolution objects, modeling objects' unequal importance regarding display. To preserve the manipulation independency of multiple objects in data delivery while provide preferential treatment for different objects as well as different layers of each object, transmission of the objects is performed over multiple streams in a partially sequenced and partially reliable fashion. A rate-distortion optimization framework is developed, which determines an optimal level of reliability for every chunk of data in each stream, taking into account the rendering importance of the object, the distortion-rate performance of the data chunks, and the statistics of the network link. Compared with heuristical methods, simulation results show that the proposed framework maximizes the display quality of the scene while minimizing the amount of data that needs to be processed by the client's rendering engine     

Lighting and Occlusion in a Wave-Based Framework

We present novel methods to enhance Computer Generated Holography (CGH) by introducing a complex‐valued wave‐based occlusion handling method. This offers a very intuitive and efficient interface to introduce optical elements featuring physically‐based light interaction exhibiting depth‐of‐field, diffraction, and glare effects. Fur‐thermore, an efficient and flexible evaluation of lit objects on a full‐parallax hologram leads to more convincing images. Previous illumination methods for CGH are not able to change the illumination settings of rendered holo‐grams. In this paper we propose a novel method for real‐time lighting of rendered holograms in order to change the appearance of a previously captured holographic scene. These functionalities are features of a bigger wave‐based rendering framework which can be combined with 2D framebuffer graphics. We present an algorithm which uses graphics hardware to accelerate the rendering.     

利用遥感图象生成地形三维实景图

本文介绍了利用遥感形象作为纹理图象，采用纹理映射技术生成地形三维实景图的原理和算法。主要包括：遥感图象上特征点（目标点）坐标的量算；遥感图象与相应地面间透视变换关系的确定；影象灰度的重采样；实景图的显示。最后给出了实验结果。实验表明，这种方法所生成的地形三维实景图，所含信息量大，真实性好，现势性强。     

Streaming of Complex 3D Scenes for Remote Walkthroughs

We describe a new 3D scene streaming approach for remote walkthroughs. In a remote walkthrough, a user on a client machine interactively navigates through a scene that resides on a remote server. Our approach allows a user to walk through a remote 3D scene, without ever having to download the entire scene from the server. Our algorithm achieves this by selectively transmitting only small parts of the scene and lower quality representations of objects, based on the user's viewing parameters and the available connection bandwidth. An online optimization algorithm selects which object representations to send, based on the integral of a benefit measure along the predicted path of movement. The rendering quality at the client depends on the available bandwidth, but practical navigation of the scene is possible even when bandwidth is low.     

GPU在复杂场景的阴影绘制中的应用

通过有效利用图形硬件的图形处理单元（GPU）的运算能力和可编程性，将人量计算从CPU分离出来。在GPU上采用顶点和片元程序进行阴影计算，从而加速复杂场景阴影绘制。选择图像空间阴影算法进行GPU加速绘制。用Cg图形编程语言和OpenGL实现了算法的绘制过程，能够满足通用的复杂3D场景应用的需要，达到满意的实时绘制效果。     

Web-based visualization of spatial objects in 3DGIS

Adaptive rendering large and complex spatial data has become an important research issue in a 3DGIS application. In order to transmit the data to the client efficiently,this paper proposes a node-layer data model to manage the 3D scene. Because the large spatial data and limited network bandwidth are the main bottlenecks of web-based 3DGIS,a client/server architecture including progressive transmission methods and multiresolution representations,together with the spatial index,are developed to improve the p...     

A virtual reality system using the concentric mosaic: construction, rendering, and data compression

This paper proposes a new image-based rendering (IBR) technique called "concentric mosaic" for virtual reality applications. IBR using the plenoptic function is an efficient technique for rendering new views of a scene from a collection of sample images previously captured. It provides much better image quality and lower computational requirement for rendering than conventional three-dimensional (3-D) model-building approaches. The concentric mosaic is a 3-D plenoptic function with viewpoints constrained on a plane. Compared with other more sophisticated four-dimensional plenoptic functions such as the light field and the lumigraph, the file size of a concentric mosaic is much smaller. In contrast to a panorama, the concentric mosaic allows users to move freely in a circular region and observe significant parallax and lighting changes without recovering the geometric and photometric scene models. The rendering of concentric mosaics is very efficient, and involves the reordering and interpolating of previously captured slit images in the concentric mosaic. It typically consists of hundreds of high-resolution images which consume a significant amount of storage and bandwidth for transmission. An MPEG-like compression algorithm is therefore proposed in this paper taking into account the access patterns and redundancy of the mosaic images. The compression algorithms of two equivalent representations of the concentric mosaic, namely the multiperspective panoramas and the normal setup sequence, are investigated. A multiresolution representation of concentric mosaics using a nonlinear filter bank is also proposed.     

可微绘制技术研究进展

可微绘制技术是当前虚拟现实、计算机图形学与计算机视觉领域研究的热点,其目标是改造计算机图形学中以光栅化或光线跟踪算法为主的真实感绘制流程,支持梯度信息回传以计算由输出图像的变化导致的输入几何、材质属性变化,通过与优化及深度学习技术等相结合支持从数据中学习绘制模型和逆向推理,是可微学习技术在计算机图形学绘制技术中的应用的具体体现,在增强/虚拟现实内容生成、三维重建、表观采集建模和逆向光学设计等领域中有广泛的应用前景。本文对可微绘制当前的发展状况进行调研,重点对该技术在真实感绘制、3维重建和表观采集建模中的研究和应用情况进行综述,并对可微绘制技术发展趋势进行展望,以期推动可微技术在学术界和产业界的进一步发展。     

基于点采样的大规模三维模型显示

大规模三维模型的实时显示一直是许多虚拟现实应用追求的目标之一,也是计算机图形学领域的研究热点之一。文中采用基于采样点的显示方法,把具有显示质量优势的传统三角形与显示速度快、简化表示能力强的采样点,通过八叉树有机组织起来;通过对基本算法做出有效的改进,并添加计算分解和重用、有效的数据管理等模块,以高效的实现方式完成了30,000个独立运动的虚拟人场景的实时显示。文中的方法也可处理其它场景,能直接应用于紧急疏散或影视、游戏等领域中大规模三维模型的显示,具有很好的应用前景。     

Compression Domain Volume Rendering for Distributed Environments

This paper describes a method for volume data compression and rendering which bases on wavelet splats. The underlying concept is especially designed for distributed and networked applications, where we assume a remote server to maintain large scale volume data sets, being inspected, browsed through and rendered interactively by a local client. Therefore, we encode the server’s volume data using a newly designed wavelet based volume compression method. A local client can render the volumes immediately from the compression domain by using wavelet footprints, a method proposed earlier. In addition, our setup features full progression, where the rendered image is refined progressively as data comes in. Furthermore, framerate constraints are considered by controlling the quality of the image both locally and globally depending on the current network bandwidth or computational capabilities of the client. As a very important aspect of our setup, the client does not need to provide storage for the volume data and can be implemented in terms of a network application. The underlying framework enables to exploit all advantageous properties of the wavelet transform and forms a basis for both sophisticated lossy compression and rendering. Although coming along with simple illumination and constant exponential decay, the rendering method is especially suited for fast interactive inspection of large data sets and can be supported easily by graphics hardware.     

Interactive, internet delivery of visualization via structured prerendered multiresolution imagery

We present a novel approach for latency-tolerant delivery of visualization and rendering results where client-side frame rate display performance is independent of source dataset size, image size, visualization technique or rendering complexity. Our approach delivers pre-rendered, multiresolution images to a remote user as they navigate through different viewpoints, visualization or rendering parameters. We employ demand-driven tiled, multiresolution image streaming and prefetching to efficiently utilize available bandwidth while providing the maximum resolution user can perceive from a given viewpoint. Since image data is the only input to our system, our approach is generally applicable to all visualization and graphics rendering applications capable of generating image files in an ordered fashion. In our implementation, a normal web server provides on-demand images to a remote custom client application, which uses client-pull to obtain and cache only those images required to fulfill the interaction needs. The main contributions of this work are: (1) an architecture for latency-tolerant, remote delivery of precomputed imagery suitable for use with any visualization or rendering application capable of producing images in an ordered fashion; (2) a performance study showing the impact of diverse network environments and different tunable system parameters on end-to-end system performance in terms of deliverable frames per second.     

A Comparison of Pixel Complexity in Composition Techniques for Sort-Last Rendering

Sort-last rendering is a method of parallelizing compute-intensive computer graphics; specifically, the primitives that describe a scene are first allocated to a set of renderers, and the rendered images are then composited to give the final image. This paper surveys six such techniques and compares their performance with active pixels, i.e., pixels that are covered by at least one primitive. Active pixels offer a uniform way of accounting for the time, space, and bandwidth costs in sort-last rendering. The comparison highlights the strengths of each technique. For example, tree composition has minimum work, binary-swap (hypercube) has minimum composition latency, direct pixel forwarding (mesh) has minimum bandwidth latency, and snooping (bus) has minimum bandwidth volume; binary-swap's bandwidth and composition latencies decrease, whereas the bandwidth volumes for direct pixel forwarding and snooping are constant, when the number of renderers increase; etc.     

基于GPU的快速Level Set图像分割

水平集(1evel set)图像分割方法是图像分割中的一个重要方法，但是该算法的计算量大，往往不能达到实时处理的要求。给出了利用新一代的可编程图形处理器(GPU)实现level set的加速算法。首先介绍了如何在GPU上利用片元渲染程序进行网格化的线性运算和有限差分PDE计算，把level set方法的离散化算子映射到GPU上。由于以数据流处理方式的GPU的存储访问快，具有并行运算能力，同时level set算法演化的显示不再需要把数据从CPU传到GPU，因此较大地提高了算法速度与交互显示。文中实现并测试了一个与初始化状态独立的二维level set的算子用于图像分割，并对其运算结果和性能进行了比较，结果表明该方法具有更快的速度。     

基于图形硬件的纹理图像编码与实时绘制算法

真实感绘制对于细节的要求越来越高,应用程序通常采用多幅或大幅分辨率很高的纹理图像,有限的内存空间就成了一个制约的瓶颈.针对纹理图像的特点和可编程图形硬件的特殊要求,该文提出了一种新的面向绘制的编码算法--增量式纹理编码算法及相应的解压绘制算法,有效地解决了纹理存储容量和真实感之间的矛盾,并利用可编程图形硬件实现了实时解压绘制.该算法在图像压缩编码过程中,动态添加码表内容,只有当已有码表内容不能表示当前图像区域时,才增加码表内容.这种方法不仅能够对于自相似性较强的纹理图像取得很高的压缩比,而且由于码表的动态更新特性,可以对图像序列进行流式编码.在绘制纹理时,该算法充分利用了现有可编程图像硬件的特性,实现了实时解压绘制.文中分别对于静态图像和动态图像序列进行了实验,结果显示,此方法能灵活有效地对各类纹理图像进行编码.     

面向移动终端的三维模型简化与碰撞检测方法研究

移动终端三维场景的绘制与漫游由于其庞大的模型数据量和复杂的外观形态使得实现清晰的场景快速绘制十分困难。为了加快模型的绘制,提出了一种面向移动终端的三维模型的简化与碰撞检测方法,以优化绘制过程。该方法通过二次测量误差半边折叠算法来简化三维模型,并利用八叉树技术对不能显示在屏幕中的场景进行剔除,从而实现三维场景的快速读取、组织和绘制。针对移动设备的屏幕尺寸以及计算能力等限制,实现了适用于移动平台的碰撞检测算法,减少了计算量。实验结果表明,该方法能有效地简化模型,并提高绘制效率,同时减少碰撞检测的计算时间,因而可应用于三维场景的快速逼真绘制。     

渲染器与Web服务器耦合实现远程体 渲染的交互优化

目的 为使用户随时随地获得需要较大计算和存储资源的交互真实感体渲染服务,设计并实现了一种面向Web的远程真实感体渲染方法。方法 计算量较大的实时体渲染任务由远端渲染服务器中的GPU加速完成并通过WebSocket将渲染图像发送至客户端;客户端浏览器只需负责接收显示渲染图像并监听发送用户交互事件。提出了一种输出系统耦合算法用以连接输出图像速率较大的渲染服务器和发送图像速率较慢的Web服务器。算法能根据Web服务器发送图像的情况动态调整每次图像输出的迭代计算次数,改变渲染服务器输出图像的时间间隔以达到与Web服务器发送速度相平衡,同时保持渲染服务器持续工作。结果 实验比较了在不同网络传输条件下,采用输出系统耦合算法与直接连接渲染器和Web服务器,渲染4个不同数据集所需的完成时间及帧率等性能评价指标。在局域网和广域网环境下,本文方法分别最多只需17 s和14 s即可完成整个渲染过程,而采用直接连接渲染器和Web服务器的方法则分别至少需要31 s和60 s才能完成整个渲染过程。实验结果表明采用输出系统耦合算法在不同网络条件下均可较大地缩短整个渲染过程所需时间,使用户在较短时间内获得高质量渲染图像。结论 本文渲染器与Web服务器耦合实现远程体渲染交互优化的方法可让用户使用与计算能力无关的桌面或移动设备通过网络透明使用高性能渲染系统;系统采用的输出系统耦合算法能够根据网络承载能力自适应调整渲染器输出速度,使用户在不同的网络环境中均可以较快的速度获得高质量渲染图像。     

基于BRDF 和GPU 并行计算的全局光照实时渲染

基于光线追踪,将屏幕图像像素分解为投射光线与场景对象交点面片辐射亮度和 纹理贴图的合成,每个面片的辐射亮度计算基于双向反射分布函数(BRDF)基的线性组合,并通 过图形处理器(GPU)处理核心并行绘制进行加速,最后与并行计算的纹理映射结果进行合成。 提出了一种基于BRDF 和GPU 并行计算的全局光照实时渲染算法,利用GPU 并行加速,在提 高绘制效率的前提下,实现动态交互材质的全局光照实时渲染。重点研究：对象表面对光线的 多次反射用BRDF 基的线性组合来表示,将非线性问题转换为线性问题,从而提高绘制效率; 利用GPU 并行加速,分别计算对象表面光辐射能量和纹理映射及其线性组合,进一步提高计算 效率满足实时绘制需求。