A SIMD-efficient 14 instruction shader program for high-throughput microtriangle rasterization

This paper shows that breaking the barrier of 1 triangle/clock rasterization rate for microtriangles in modern GPU architectures in an efficient way is possible. The fixed throughput of the special purpose culling and triangle setup stages of the classic pipeline limits the GPU scalability to rasterize many triangles in parallel when these cover very few pixels. In contrast, the shader core counts and increasing GFLOPs in modern GPUs clearly suggests parallelizing this computation entirely across multiple shader threads, making use of the powerful wide-ALU instructions. In this paper, we present a very efficient SIMD-like rasterization code targeted at very small triangles that scales very well with the number of shader cores and has higher performance than traditional edge equation based algorithms. We have extended the ATTILA GPU shader ISA (del Barrioet al. in IEEE International Symposium on Performance Analysis of Systems and Software, pp. 231–241, 2006) with two fixed point instructions to meet the rasterization precision requirement. This paper also introduces a novel subpixel Bounding Box size optimization that adjusts the bounds much more finely, which is critical for small triangles, and doubles the 2×2-pixel stamp test efficiency. The proposed shader rasterization program can run on top of the original pixel shader program in such a way that selected fragments are rasterized, attribute interpolated and pixel shaded in the same pass. Our results show that our technique yields better performance than a classic rasterizer at 8 or more shader cores, with speedups as high as 4× for 16 shader cores.     

Hardware‐Assisted Projected Tetrahedra

We present a flexible and highly efficient hardware‐assisted volume renderer grounded on the original Projected Tetrahedra (PT) algorithm. Unlike recent similar approaches, our method is exclusively based on the rasterization of simple geometric primitives and takes full advantage of graphics hardware. Both vertex and geometry shaders are used to compute the tetrahedral projection, while the volume ray integral is evaluated in a fragment shader; hence, volume rendering is performed entirely on the GPU within a single pass through the pipeline. We apply a CUDA‐based visibility ordering achieving rendering and sorting performance of over 6 M Tet/s for unstructured datasets. Furthermore, as each tetrahedron is processed independently, we employ a data‐parallel solution which is neither bound by GPU memory size nor does it rely on auxiliary volume information. In addition, iso‐surfaces can be readily extracted during the rendering process, and time‐varying data are handled without extra burden.     

Sample-based cameras for feed forward reflection rendering

This paper presents sample-based cameras for rendering high quality reflections on convex reflectors at interactive rates. The method supports change of view, moving objects and reflectors, higher order reflections, view-dependent lighting of reflected objects, and reflector surface properties. In order to render reflections with the feed forward graphics pipeline, one has to project reflected vertices. A sample-based camera is a collection of BSP trees of pinhole cameras that jointly approximate the projection function. It is constructed from the reflected rays defined by the desired view and the scene reflectors. A scene point is projected by invoking only the cameras that contain it in their frustums. Reflections are rendered by projecting the scene geometry and then rasterizing in hardware     

光线追踪的OpenCL加速实现研究

目前GPU计算能力让kD-Tree划分实时场景光线追踪并行算法的执行变得更具有可行性。图像处理器(GPU)高效应用于多边形的渲染,GPU内部单元的可编程性已经让其广泛应用于多边形渲染以外的领域。本文详细描述使用OpenCL的kD-Tree遍历算法,对运算占主要部分的相交测试作出改进,同时提高了GPU计算能力与存储器的利用率,从而提升了光线追踪算法效率。     

Visibility Driven Rasterization

We present a new visibility driven rasterization scheme that significantly increases the rendering performance of modern graphic subsystems. Instead of rasterizing, texturing, lighting, and depth-testing each individual pixel, we introduce a two-level visibility mask within the rasterization stage which facilitates the removal of groups of pixels and triangles from rasterization and subsequent pipeline stages.
Local visibility information is stored within the visibility mask that is updated several times during the generation of a frame. The update can easily be accomplished by extending already available (in hardware) occlusion culling mechanisms (i.e. those of HP and SGI), where it is possible to integrate the additional functionality without any additional delay cycles. In addition to these existing hardware based occlusion culling approaches—which cull only geometry contained in bounding volumes determined as occluded —we are able to significantly accelerate the rendering of the geometry determined as visible . However, our approach does not specifically rely on such occlusion culling hardware.The proposed new rasterization scheme is well suited for hardware implementation, can easily be integrated into low-cost rasterizers, and its scalability can vary upon available chip real estate. Only incremental modifications of modern graphics subsystems are required to achieve a significant improvement in rendering performance.     

支持Shader的Direct3D9应用程序透明并行化

根据图形处理器的最新可编程单元Vertex Shader和Pixel Shader的体系结构和单机Direct3D9应用程序的执行流程,提出支持Shader的Direct3D9应用程序在图形集群的透明并行化策略.图形集群的节点划分为资源分配和资源绘制节点,资源分配节点通过截取绘制接口将应用程序实时转换为6类绘制资源,包括命令流、Vertex Shader、Pixel Shader、顶点流、索引流和纹理流.资源绘制节点根据绘制资源的描述信息和资源数据重构出Direct3D9的绘制命令.图形集群中的所有绘制节点都保留全部的绘制资源,并且通过计算基于多流模式场景数据在屏幕空间的包围盒进行绘制任务划分.实验证明,使用,这种策略完全可以实现支持Shader的Direct3D9应用程序透明并行化.相对于单机绘制,基于图形集群的并行图形绘制不仅提高绘制性能而且得到较高绘制加速比.     

Fast indirect illumination using Layered Depth Images

We present a novel hybrid rendering method for diffuse and glossy indirect illumination. A scene is rendered using standard rasterization on a GPU. In a shader, secondary ray queries are used to sample incident light and to compute indirect lighting. We observe that it is more important to cast many rays than to have precise results for each ray. Thus, we approximate secondary rays by intersecting them with precomputed layered depth images of the scene. We achieve interactive to real-time frame rates including indirect diffuse and glossy effects.     

Automatic camera control in virtual environments augmented using multiple sparse videos

Automated virtual camera control has been widely used in animation and interactive virtual environments. We have developed a multiple sparse camera based free view video system prototype that allows users to control the position and orientation of a virtual camera, enabling the observation of a real scene in three dimensions (3D) from any desired viewpoint. Automatic camera control can be activated to follow selected objects by the user. Our method combines a simple geometric model of the scene composed of planes (virtual environment), augmented with visual information from the cameras and pre-computed tracking information of moving targets to generate novel perspective corrected 3D views of the virtual camera and moving objects. To achieve real-time rendering performance, view-dependent textured mapped billboards are used to render the moving objects at their correct locations and foreground masks are used to remove the moving objects from the projected video streams. The current prototype runs on a PC with a common graphics card and can generate virtual 2D views from three cameras of resolution 768×576 with several moving objects at about 11 fps.     

利用反向投影实现的实时软阴影映射算法

基于阴影映射算法,提出一种利用反向投影实现的实时软阴影的新算法。算法对每个光源都产生对应的阴影图,使用阴影图作为对场景的离散化表示,引入可见因子来计算场景点的亮度信息,并采用GPU片元着色、层次阴影图、自适应精度等方法加速阴影渲染。实验表明,算法非常适合于实时渲染复杂、动态的场景,可以很好地处理遮挡物的融合,并且很容易在可编程图形硬件上实现。     

Real-time motion blur using extruded triangles

Several algorithms have been introduced to render motion blur in real time by solving the visibility problem in the spatial-temporal domains. However, some algorithms render at interactive frame rates but have artifacts or noise. Therefore, we propose a new algorithm that renders real-time motion blur using extruded triangles. Our method uses two triangles in the previous frame and the current frame to make an extruded triangle then send it to rasterization. By using the standard rasterization, visibility determination is performed efficiently. To solve the occlusion between extruded triangles for a given pixel, we introduce a combination solution using a sorting in front-to-back order and bitwise operations in the spatial-temporal dimensions. This solution ensures that only non-occluded extruded triangles are shaded. We further improve performance of our algorithm using a coverage map.     

People counting via multiple views using a fast information fusion approach

Real-time estimates of a crowd size is a central task in civilian surveillance. In this paper we present a novel system counting people in a crowd scene with overlapping cameras. This system fuses all single view foreground information to localize each person present on the scene. The purpose of our fusion strategy is to use the foreground pixels of each single views to improve real-time objects association between each camera of the network. The foreground pixels are obtained by using an algorithm based on codebook. In this work, we aggregate the resulting silhouettes over cameras network, and compute a planar homography projection of each camera’s visual hull into ground plane. The visual hull is obtained by finding the convex hull of the foreground pixels. After the projection into the ground plane, we fuse the obtained polygons by using the geometric properties of the scene and on the quality of each camera detection. We also suggest a region-based approach tracking strategy which keeps track of people movements and of their identities along time, also enabling tolerance to occasional misdetections. This tracking strategy is implemented on the result of the views fusion and allows to estimate the crowd size dependently on each frame. Assessment of experiments using public datasets proposed for the evaluation of counting people system demonstrates the performance of our fusion approach. These results prove that the fusion strategy can run in real-time and is efficient for making data association. We also prove that the combination of our fusion approach and the proposed tracking improve the people counting.

     

Dynamic sampling rate: harnessing frame coherence in graphics applications for energy-efficient GPUs

In real-time rendering, a 3D scene is modelled with meshes of triangles that the GPU projects to the screen. They are discretized by sampling each triangle at regular space intervals to generate fragments which are then added texture and lighting effects by a shader program. Realistic scenes require detailed geometric models, complex shaders, high-resolution displays and high screen refreshing rates, which all come at a great compute time and energy cost. This cost is often dominated by the fragment shader, which runs for each sampled fragment. Conventional GPUs sample the triangles once per pixel; however, there are many screen regions containing low variation that produce identical fragments and could be sampled at lower than pixel-rate with no loss in quality. Additionally, as temporal frame coherence makes consecutive frames very similar, such variations are usually maintained from frame to frame. This work proposes Dynamic Sampling Rate (DSR), a novel hardware mechanism to reduce redundancy and improve the energy efficiency in graphics applications. DSR analyzes the spatial frequencies of the scene once it has been rendered. Then, it leverages the temporal coherence in consecutive frames to decide, for each region of the screen, the lowest sampling rate to employ in the next frame that maintains image quality. We evaluate the performance of a state-of-the-art mobile GPU architecture extended with DSR for a wide variety of applications. Experimental results show that DSR is able to remove most of the redundancy inherent in the color computations at fragment granularity, which brings average speedups of 1.68x and energy savings of 40%.

     

基于GPU的屏幕空间镜头水滴实时渲染

为了实时模拟真实性高的镜头水滴,提出一种在可编程图形硬件中实现的镜头水滴渲染新方法。首先三维场景被渲染到一张场景纹理,然后在GPU着色器中为镜头水滴产生不规则边缘接触曲线,使用该曲线快速构建水滴的曲面,最后采用曲面的表面信息并根据水滴的光学属性渲染出水滴的折射效果。采用该方法,可以实时地在屏幕上渲染出镜头水滴效果。使用GPU着色器进行渲染,可以在渲染出效果逼真的水滴的情况下,获得实时的帧率。采用光线折射物理方法渲染出的水滴的效果比直接使用纹理贴图方式获得的水滴的效果更逼真。     

Index rendering: hardware-efficient architecture for 3-D graphics in multimedia system

Real-time three-dimensional (3D) graphics is emerging rapidly in multimedia applications, but it suffers from requirements for huge computation, high bandwidth, and large buffer. In order to achieve hardware efficiency for 3D graphics rendering, we propose a novel approach named index rendering. The basic concept of index rendering is to realize a 3D rendering pipeline by using asynchronous multi-dataflows. Triangle information can be divided into several parts with each part capable of being transferred independently and asynchronously. Finally, all data are converged by the index to generate the final image. The index rendering approach can eliminate unnecessary operations in the traditional 3D graphics pipeline, the unnecessary operations are caused by the invisible pixels and triangles in the 3D scene. Previous work, deferred shading, eliminates the operations relating to invisible pixels, but it requires huge tradeoffs in bandwidth and buffer size. With index rendering, we can eliminate operations on both invisible pixels and triangles with fewer tradeoffs as compared with the deferred shading approach. The simulation and analysis results show that the index rendering approach can reduce 10%-70% of lighting operations when using the flat and Gouraud shading process and decrease 30%-95% when using Phong shading. Furthermore, it saves 70% of buffer size and 50%-70% of bandwidth compared with the deferred shading approach. The result also indicates that this approach of index rendering is especially suitable for low-cost portable rendering devices. Hence, index rendering is a hardware-efficient architecture for 3D graphics, and it makes rendering hardware more easily integrated into multimedia systems, especially system-on-a-chip (SOC) designs.     

实时绘制带斑纹的毛发

提出一种将reaction-diffusion过程纹理与Lengyel的毛发绘制算法相结合，实时生成带有斑纹的毛发的方法。利用vertexshader和pixel shader等硬件扩展功能对Lengyel方法进行了改进，不但提高了绘制速度，而且使得毛发的光照计算更为准确．     

Special relativistic visualization by local ray tracing

Special relativistic visualization offers the possibility of experiencing the optical effects of traveling near the speed of light, including apparent geometric distortions as well as Doppler and searchlight effects. Early high-quality computer graphics images of relativistic scenes were created using offline, computationally expensive CPU-side 4D ray tracing. Alternate approaches such as image-based rendering and polygon-distortion methods are able to achieve interactivity, but exhibit inferior visual quality due to sampling artifacts. In this paper, we introduce a hybrid rendering technique based on polygon distortion and local ray tracing that facilitates interactive high-quality visualization of multiple objects moving at relativistic speeds in arbitrary directions. The method starts by calculating tight image-space footprints for the apparent triangles of the 3D scene objects. The final image is generated using a single image-space ray tracing step incorporating Doppler and searchlight effects. Our implementation uses GPU shader programming and hardware texture filtering to achieve high rendering speed.     

Adaptive Ray‐bundle Tracing with Memory Usage Prediction: Efficient Global Illumination in Large Scenes

This paper proposes an adaptive rendering technique for ray‐bundle tracing. Ray‐bundle tracing can be done by per‐pixel linked‐list construction on a GPU rasterization pipeline. This rasterization based approach offers significant benefits for the efficient generation of light maps (e.g., hardware acceleration, tessellation, and recycling of shaders used in real‐time graphics). However, it is inapplicable to large and complex scenes due to the limited capacity of the GPU memory because it requires a high‐resolution frame buffer and high‐capacity node buffer for the linked‐lists. In addition, memory overflow can potentially occur on the per‐pixel linked‐list since the memory usage of the lists is usually unknown before the rendering process. We introduce an adaptive tiling technique with memory usage prediction. Our method uses an appropriately tiled frame buffer, thus eliminating almost all of the overflow risks thanks to our adaptive tile subdivision scheme. Using this technique, we are able to render high‐quality light maps of large and complex scenes which cannot be computed using previous ray‐bundle based methods.     

Automated camera planning to film robot operations

Automatic 3D animation generation techniques are becoming increasingly popular in different areas related to computer graphics such as video games and animated movies. They help automate the filmmaking process even by non professionals without or with minimal intervention of animators and computer graphics programmers. Based on specified cinematographic principles and filming rules, they plan the sequence of virtual cameras that the best render a 3D scene. In this paper, we present an approach for automatic movie generation using linear temporal logic to express these filming and cinematography rules. We consider the filming of a 3D scene as a sequence of shots satisfying given filming rules, conveying constraints on the desirable configuration (position, orientation, and zoom) of virtual cameras. The selection of camera configurations at different points in time is understood as a camera plan, which is computed using a temporal-logic based planning system (TLPlan) to obtain a 3D movie. The camera planner is used within an automated planning application for generating 3D tasks demonstrations involving a teleoperated robot arm on the the International Space Station (ISS). A typical task demonstration involves moving the robot arm from one configuration to another. The main challenge is to automatically plan the configurations of virtual cameras to film the arm in a manner that conveys the best awareness of the robot trajectory to the user. The robot trajectory is generated using a path-planner. The camera planner is then invoked to find a sequence of configurations of virtual cameras to film the trajectory.     

Non‐Linear Beam Tracing on a GPU

Beam tracing combines the flexibility of ray tracing and the speed of polygon rasterization. However, beam tracing so far only handles linear transformations; thus, it is only applicable to linear effects such as planar mirror reflections but not to non‐linear effects such as curved mirror reflection, refraction, caustics and shadows. In this paper, we introduce non‐linear beam tracing to render these non‐linear effects. Non‐linear beam tracing is highly challenging because commodity graphics hardware supports only linear vertex transformation and triangle rasterization. We overcome this difficulty by designing a non‐linear graphics pipeline and implementing it on top of a commodity GPU. This allows beams to be non‐linear where rays within the same beam do not have to be parallel or intersect at a single point. Using these non‐linear beams, real‐time GPU applications can render secondary rays via polygon streaming similar to how they render primary rays. A major strength of this methodology is that it naturally supports fully dynamic scenes without the need to pre‐store a scene database. Utilizing our approach, non‐linear ray tracing effects can be rendered in real‐time on a commodity GPU under a unified framework.     

Ray geometry in non-pinhole cameras: a survey

A pinhole camera collects rays passing through a common 3D point and its image resembles what would be seen by human eyes. In contrast, a non-pinhole (multi-perspective) camera combines rays collected by different viewpoints. Despite their incongruity of view, their images are able to preserve spatial coherence and can depict, within a single context, details of a scene that are simultaneously inaccessible from a single view, yet easily interpretable by a viewer. In this paper, we thoroughly discuss the design, modeling, and implementation of a broad class of non-pinhole cameras and their applications in computer graphics and vision. These include mathematical (conceptual) camera models such as the General Linear Cameras and real non-pinhole cameras such as catadioptric cameras and projectors. A unique component of this paper is a ray geometry analysis that uniformly models these non-pinhole cameras as manifolds of rays and ray constraints. We also model the thin lens as a ray transform and study how ray geometry is changed by the thin lens for studying distortions and defocusing. We hope to provide mathematical fundamentals to satisfy computer vision researchers as well as tools and algorithms to aid computer graphics and optical engineering researchers.