Efficient Caustic Rendering with Lightweight Photon Mapping

Robust and efficient rendering of complex lighting effects, such as caustics, remains a challenging task. While algorithms like vertex connection and merging can render such effects robustly, their significant overhead over a simple path tracer is not always justified and – as we show in this paper ‐ also not necessary. In current rendering solutions, caustics often require the user to enable a specialized algorithm, usually a photon mapper, and hand‐tune its parameters. But even with carefully chosen parameters, photon mapping may still trace many photons that the path tracer could sample well enough, or, even worse, that are not visible at all. Our goal is robust, yet lightweight, caustics rendering. To that end, we propose a technique to identify and focus computation on the photon paths that offer significant variance reduction over samples from a path tracer. We apply this technique in a rendering solution combining path tracing and photon mapping. The photon emission is automatically guided towards regions where the photons are useful, i.e., provide substantial variance reduction for the currently rendered image. Our method achieves better photon densities with fewer light paths (and thus photons) than emission guiding approaches based on visual importance. In addition, we automatically determine an appropriate number of photons for a given scene, and the algorithm gracefully degenerates to pure path tracing for scenes that do not benefit from photon mapping.     

Adaptive Caustic Maps Using Deferred Shading

Caustic maps provide an interactive image-space method to render caustics, the focusing of light via reflection and refraction. Unfortunately, caustic mapping suffers problems similar to shadow mapping: aliasing from poor sampling and map projection as well as temporal incoherency from frame-to-frame sampling variations. To reduce these problems, researchers have suggested methods ranging from caustic blurring to building a multiresolution caustic map. Yet these all require a fixed photon sampling, precluding the use of importance-based photon densities. This paper introduces adaptive caustic maps. Instead of densely sampling photons via a rasterization pass, we adaptively emit photons using a deferred shading pass. We describe deferred rendering for refractive surfaces, which speeds rendering of refractive geometry up to 25% and with adaptive sampling speeds caustic rendering up to 200%. These benefits are particularly noticable for complex geometry or using millions of photons. While developed for a GPU rasterizer, adaptive caustic map creation can be performed by any renderer that individually traces photons, e.g., a GPU ray tracer.     

Progressive Expectation‐Maximization for Hierarchical Volumetric Photon Mapping

State‐of‐the‐art density estimation methods for rendering participating media rely on a dense photon representation of the radiance distribution within a scene. A critical bottleneck of such kernel‐based approaches is the excessive number of photons that are required in practice to resolve fine illumination details, while controlling the amount of noise. In this paper, we propose a parametric density estimation technique that represents radiance using a hierarchical Gaussian mixture. We efficiently obtain the coefficients of this mixture using a progressive and accelerated form of the Expectation‐Maximization algorithm. After this step, we are able to create noise‐free renderings of high‐frequency illumination using only a few thousand Gaussian terms, where millions of photons are traditionally required. Temporal coherence is trivially supported within this framework, and the compact footprint is also useful in the context of real‐time visualization. We demonstrate a hierarchical ray tracing‐based implementation, as well as a fast splatting approach that can interactively render animated volume caustics.     

Unsupervised Image Reconstruction for Gradient-Domain Volumetric Rendering

Gradient-domain rendering can highly improve the convergence of light transport simulation using the smoothness in image space. These methods generate image gradients and solve an image reconstruction problem with rendered image and the gradient images. Recently, a previous work proposed a gradient-domain volumetric photon density estimation for homogeneous participating media. However, the image reconstruction relies on traditional L1 reconstruction, which leads to obvious artifacts when only a few rendering passes are performed. Deep learning based reconstruction methods have been exploited for surface rendering, but they are not suitable for volume density estimation. In this paper, we propose an unsupervised neural network for image reconstruction of gradient-domain volumetric photon density estimation, more specifically for volumetric photon mapping, using a variant of GradNet with an encoded shift connection and a separated auxiliary feature branch, which includes volume based auxiliary features such as transmittance and photon density. Our network smooths the images on global scale and preserves the high frequency details on a small scale. We demonstrate that our network produces a higher quality result, compared to previous work. Although we only considered volumetric photon mapping, it's straightforward to extend our method for other forms, like beam radiance estimation.     

Rendering Caustics on Non-Lambertian Surfaces

This paper presents a new technique for rendering caustics on non-Lambertian surfaces. The method is based on an extension of the photon map which removes previous restrictions limiting the usage to Lambertian surfaces. We add information about the incoming direction to the photons and this allows us to combine the photon map with arbitrary reflectance functions. By using a cone-filter we improve the quality of the radiance estimate in particular at discontinuities. Furthermore we introduce balancing of the photon map which not only reduces the memory requirements but also significantly reduces the rendering time. We have used the method to render caustics on surfaces with reflectance functions varying from Lambertian to glossy specular.     

GPU-based rendering of point-sampled water surfaces

Particle-based simulations are widely used to simulate fluids. We present a real-time rendering method for the results of particle-based simulations of water. Traditional approaches to visualize the results of particle-based simulations construct water surfaces that are usually represented by polygons. To construct water surfaces from the results of particle-based simulations, a density function is assigned to each particle and a density field is computed by accumulating the values of the density functions of all particles. However, the computation of the density field is time consuming. To address this problem, we propose an efficient calculation of density field using a graphics processing unit (GPU). We present a rendering method for water surfaces sampled by points. The use of the GPU permits efficient simulation of optical effects, such as refraction, reflection, and caustics.     

基于栅格的全局光子图重建

基于光子图的光子映射算法能产生高质量的照片级图像。对于光照复杂的 场景,光子图需要存储大量光子以提高生成图像的质量,这不仅占用大量的内存空间,而且 光照估计的时间长。论文提出基于栅格的全局光子图重建的算法,即在光子包围盒被栅格化 后,其非空栅格中一定比例的光子被用来重建新的光子图,并保证重建前后栅格内光子能量 和守恒,这使得重建前后光子图的光照估计的效果相近。通过增加特定栅格中的重建光子数 目,能有效减少由几何偏差引起的光照估计误差,增强直接聚焦（焦散）和间接聚焦光照的 绘制效果;并使用简单方法检测生成图像中少量噪声,增加少量采样即可有效减少相应的噪 声。全局光子图重建算法的计算成本低,并保持生成图像的视觉独立性。     

Photon Differential Splatting for Rendering Caustics

We present a photon splatting technique which reduces noise and blur in the rendering of caustics. Blurring of illumination edges is an inherent problem in photon splatting, as each photon is unaware of its neighbours when being splatted. This means that the splat size is usually based on heuristics rather than knowledge of the local flux density. We use photon differentials to determine the size and shape of the splats such that we achieve adaptive anisotropic flux density estimation in photon splatting. As compared to previous work that uses photon differentials, we present the first method where no photons or beams or differentials need to be stored in a map. We also present improvements in the theory of photon differentials, which give more accurate results and a faster implementation. Our technique has good potential for GPU acceleration, and we limit the number of parameters requiring user adjustment to an overall smoothing parameter and the number of photons to be traced.     

Simplified photon mapping for real-time caustics rendering

The objective of this paper is to adapt photon mapping for real-time simulation of caustics. High-performance algorithm adapted for the GPU and implemented on the basis of cross-platform OpenGL and OpenCL APIs is proposed. For effective rendering of caustics and generation of photon map OpenGL shaders are used. Voxel acceleration structure constructed directly on the GPU by means of OpenCL provides fast access to photon map. Performance estimation for different equipment is given.     

Irradiance regression for efficient final gathering in global illumination

Photon mapping is widely used for global illumination rendering because of its high computational efficiency. But its efficiency is still limited, mainly by the intensive sampling required in final gathering, a process that is critical for removing low frequency artifacts of density estimation. In this paper, we propose a method to predict the final gathering estimation with direct density estimation, thereby achieving high quality global illumination by photon mapping with high efficiency. We first sample the irradiance of a subset of shading points by both final gathering and direct radiance estimation. Then we use the samples as a training set to predict the final gathered irradiance of other shading points through regression. Consequently, we are able to achieve about three times overall speedup compared with straightforward final gathering in global illumination computation with the same rendering quality.     

光子映射在CUDA中的研究与实现

通过修改光子映射算法的实现过程,使得该算法能够通过CUDA完全运行在最新的GPU上,从而能够充分利用GPU强大的并行计算能力,加速光子映射的实现。光子映射在CUDA中的实现主要通过两个方面来完成：构建光子图和估计辐射能。同时为了提高对光子图中的光子信息的查找速度,采用了kd-tree结构来存储光子信息,使得可以通过KNN（K-Nearest Neighbor）快速搜索光子图。在所测试环境中,渲染速度是CPU中的近1O倍。     

A versatile optical model for hybrid rendering of volume data

In volume rendering, most optical models currently in use are based on the assumptions that a volumetric object is a collection of particles and that the macro behavior of particles, when they interact with light rays, can be predicted based on the behavior of each individual particle. However, such models are not capable of characterizing the collective optical effect of a collection of particles which dominates the appearance of the boundaries of dense objects. In this paper, we propose a generalized optical model that combines particle elements and surface elements together to characterize both the behavior of individual particles and the collective effect of particles. The framework based on a new model provides a more powerful and flexible tool for hybrid rendering of isosurfaces and transparent clouds of particles in a single scene. It also provides a more rational basis for shading, so the problem of normal-based shading in homogeneous regions encountered in conventional volume rendering can be easily avoided. The model can be seen as an extension to the classical model. It can be implemented easily, and most of the advanced numerical estimation methods previously developed specifically for the particle-based optical model, such as preintegration, can be applied to the new model to achieve high-quality rendering results.     

Anisotropic density estimation for photon mapping

Computational Visual Media - Photon mapping is a widely used technique for global illumination rendering. In the density estimation step of photon mapping, the indirect radiance at a shading point...     

Into the Blue: Better Caustics through Photon Relaxation

The photon mapping method is one of the most popular algorithms employed in computer graphics today. However, obtaining good results is dependent on several variables including kernel shape and bandwidth, as well as the properties of the initial photon distribution. While the photon density estimation problem has been the target of extensive research, most algorithms focus on new methods of optimising the kernel to minimise noise and bias. In this paper we break from convention and propose a new approach that directly redistributes the underlying photons. We show that by relaxing the initial distribution into one with a blue noise spectral signature we can dramatically reduce background noise, particularly in areas of uniform illumination. In addition, we propose an efficient heuristic to detect and preserve features and discontinuities. We then go on to demonstrate how reconfiguration also permits the use of very low bandwidth kernels, greatly improving render times whilst reducing bias.     

The photon pipeline revisited

With the development of real-time ray tracing in recent years, it is now very interesting to ask if real-time performance can be achieved for high-quality rendering algorithms based on ray tracing. In this paper, we propose a pipelined architecture to implement reverse photon mapping. Our architecture can use real-time ray tracing to generate photon points and camera points, so the main challenge is how to implement the gathering phase that computes the final image. Traditionally, the gathering phase of photon mapping has only allowed coarse-grain parallelism, and this situation has been a source of inefficiency, cache thrashing, and limited throughput. To avail fine-grain pipelining and data parallelism, we arrange computations so that photons can be processed independently, similar to the way that triangles are efficiently processed in traditional real-time graphics hardware. We employ several techniques to improve cache behavior and to reduce communication overhead. Simulations show that the bandwidth requirements of this architecture are within the capacity of current and future hardware, and this suggests that photon mapping may be a good choice for real-time performance in the future.     

Extending Backward Polygon Beam Tracing to Glossy Scattering Surfaces

Backward polygon beam tracing methods, that is beam tracing from the light source (L), are well suited to gather path coherency from specular (S) scattering surfaces. These methods are useful for modelling and efficiently simulating caustics on diffuse (D) surfaces; an effect due to LS⁺D transport paths. This paper generalizes backward polygon beam tracing to include a glossy (G) scattering surface. To this end the details of a beam tracing lumped model and implementation of L(S | G)D transport paths are presented. Although we limit the discussion to short transport paths, we show that backward beam tracing is faster than photon mapping by an order of magnitude for rendering caustics from glossy and specular surfaces.     

Deep‐learning the Latent Space of Light Transport

We suggest a method to directly deep‐learn light transport, i. e., the mapping from a 3D geometry‐illumination‐material configuration to a shaded 2D image. While many previous learning methods have employed 2D convolutional neural networks applied to images, we show for the first time that light transport can be learned directly in 3D. The benefit of 3D over 2D is, that the former can also correctly capture illumination effects related to occluded and/or semi‐transparent geometry. To learn 3D light transport, we represent the 3D scene as an unstructured 3D point cloud, which is later, during rendering, projected to the 2D output image. Thus, we suggest a two‐stage operator comprising a 3D network that first transforms the point cloud into a latent representation, which is later on projected to the 2D output image using a dedicated 3D‐2D network in a second step. We will show that our approach results in improved quality in terms of temporal coherence while retaining most of the computational efficiency of common 2D methods. As a consequence, the proposed two stage‐operator serves as a valuable extension to modern deferred shading approaches.     

Feature-Adaptive Rendering of Loop Subdivision Surfaces on Modern GPUs

We present a novel approach for real-time rendering Loop subdivision surfaces on modern graphics hardware. Our algorithm evaluates both positions and normals accurately, thus providing the true Loop subdivision surface. The core idea is to recursively refine irregular patches using a GPU compute kernel. All generated regular patches are then directly evaluated and rendered using tile hardware tessellation unit. Our approach handles triangular control meshes of arbitrary topologies and incorporates common subdivision surface features such as semi-sharp creases and hierarchical edits. While surface rendering is accurate up to machine precision, we also enforce a consistent bitwise evaluation of positions and normals at patch boundaries. This is particularly useful in the context of displacement mapping which strictly requires inatching surface normals. Furthermore, we incorporate efficient level-of-detail rendering where subdivision depth and tessellation density can be adjusted on-the-fly. Overall, our algorithm provides high-quality results at real-time frame rates, thus being ideally suited to interactive rendering applications such as video games or authoring tools.     

基于学习的图像超分辨重建方法综述

全面综述了基于学习的单帧图像超分辨重建技术的研究与发展。基于学习的单帧图像超分辨重建借助机器学习技术,通过学习低分辨与高分辨图像之间的映射关系估计低分辨图像中丢失的高频细节,以获得边缘清晰、纹理细节丰富的高质量图像。根据超分辨重建过程中实例样本使用方式和学习算法的不同,已有基于学习的超分辨重建方法可分为五种类型,包括基于[k]近邻学习的方法、基于流形学习的方法、基于字典学习的方法、基于实例多线性回归的方法和基于深度学习的方法。对每类方法的主要思想和具有代表性的方法进行了详细介绍,对六种具有代表性的基于学习的超分辨重建方法的重建结果进行了比较和分析。最后,对基于学习的超分辨重建技术的未来发展趋势进行了展望。