Rendering realistic spectral bokeh due to lens stops and aberrations

Creating bokeh effect in synthesized images can improve photorealism and emphasize interesting subjects. Therefore, we present a novel method for rendering realistic bokeh effects, especially chromatic effects, which are absent for existing methods. This new method refers to two key techniques: an accurate dispersive lens model and an efficient spectral rendering scheme. This lens model is implemented based on optical data of real lenses and considers wavelength dependency of physical lenses by introducing a sequential dispersive ray tracing algorithm inside this model. This spectral rendering scheme is proposed to support rendering of lens dispersion and integration between this new model and bidirectional ray tracing. The rendering experiments demonstrate that our method is able to simulate realistic spectral bokeh effects caused by lens stops and aberrations, especially chromatic aberration, and feature high rendering efficiency.     

散景效果的真实感绘制

针对现有方法绘制的散景效果真实感较差的问题,提出一种基于几何光学理论的散景效果真实感绘制方法.该方法以光线传播的折射定律为基础,利用序列光线追踪方法对相机镜头的光学成像特性进行精确建模;对相机镜头的内部结构进行精确模拟,包括孔径光阑和渐晕光阑,以绘制出由孔径形状和渐晕共同作用的散景效果;利用几何光学理论和序列光线追踪方法精确计算出出射光瞳的位置和大小,以辅助光线采样,提高光线追踪效率.绘制结果表明,利用该方法能够绘制出较为逼真的散景效果,正确模拟了孔径形状和渐晕对散景效果的影响,并具有较高的光线追踪效率.     

Efficient Monte Carlo rendering with realistic lenses

In this paper we present a novel approach to simulate image formation for a wide range of real world lenses in the Monte Carlo ray tracing framework. Our approach sidesteps the overhead of tracing rays through a system of lenses and requires no tabulation. To this end we first improve the precision of polynomial optics to closely match ground‐truth ray tracing. Second, we show how the Jacobian of the optical system enables efficient importance sampling, which is crucial for difficult paths such as sampling the aperture which is hidden behind lenses on both sides. Our results show that this yields converged images significantly faster than previous methods and accurately renders complex lens systems with negligible overhead compared to simple models, e.g. the thin lens model. We demonstrate the practicality of our method by incorporating it into a bidirectional path tracing framework and show how it can provide information needed for sophisticated light transport algorithms.     

Efficiently Simulating the Bokeh of Polygonal Apertures in a Post‐Process Depth of Field Shader

The effect of aperture shape on an image, known in photography as ‘bokeh’, is an important characteristic of depth of field in real‐world cameras. However, most real‐time depth of field techniques produce Gaussian bokeh rather than the circular or polygonal bokeh that is almost universal in real‐world cameras. ‘Scattering’ (i.e. point‐splatting) techniques provide a flexible way to model any aperture shape, but tend to have prohibitively slow performance, and require geometry‐shaders or significant engine changes to implement. This paper shows that simple post‐process ‘gathering’ depth of field shaders can be easily extended to simulate certain bokeh effects. Specifically we show that it is possible to efficiently model the bokeh effects of square, hexagonal and octagonal apertures using a novel separable filtering approach. Performance data from a video game engine test demonstrates that our shaders attain much better frame rates than a naive non‐separable approach.     

Stereo-based bokeh effects for photography

Bokeh, a sought-after photo rendering style of out-of-focus blur, typically aims at an esthetic quality which is not available to low-end consumer-grade cameras due to the lens design. We present a bokeh simulation method using stereo-vision techniques. We refine a depth map obtained by stereo matching, also using some minor user interaction. Overexposed regions are recovered according to depth information. A depth-aware bokeh effect is then applied with user-adjustable apertures sizes or shapes. We also simulate swirly bokeh, also known as cat-eye effect. Our method mainly aims at the visual quality of the bokeh effect rather than (so far) at time efficiency. Experiments show that our results are natural looking and that they can be comparable to bokeh effects achieved with expensive real-world bokeh-capable camera systems.     

Practical Real‐Time Lens‐Flare Rendering

We present a practical real‐time approach for rendering lens‐flare effects. While previous work employed costly ray tracing or complex polynomial expressions, we present a coarser, but also significantly faster solution. Our method is based on a first‐order approximation of the ray transfer in an optical system, which allows us to derive a matrix that maps lens flare‐producing light rays directly to the sensor. The resulting approach is easy to implement and produces physically‐plausible images at high framerates on standard off‐the‐shelf graphics hardware.     

Using Wavefront Tracing for the Visualization and Optimization of Progressive Lenses

Progressive addition lenses are a relatively new approach to compensate for defects of the human visual system. While traditional spectacles use rotationally symmetric lenses, progressive lenses require the specification of free-form surfaces. This poses difficult problems for the optimal design and its visual evaluation.
This paper presents two new techniques for the visualization of optical systems and the optimization of progressive lenses. Both are based on the same wavefront tracing approach to accurately evaluate the refraction properties of complex optical systems.
We use the results of wavefront tracing for continuously re-focusing the eye during rendering. Together with distribution ray tracing, this yields high-quality images that accurately simulate the visual quality of an optical system. The design of progressive lenses is difficult due to the trade-off between the desired properties of the lens and unavoidable optical errors, such as astigmatism and distortions. We use wavefront tracing to derive an accurate error functional describing the desired properties and the optical error across a lens. Minimizing this error yields optimal free-form lens surfaces.
While the basic approach is much more general, in this paper, we describe its application to the particular problem of designing and evaluating progressive lenses and demonstrate the benefits of the new approach with several example images.     

Sparse high‐degree polynomials for wide‐angle lenses

Rendering with accurate camera models greatly increases realism and improves the match of synthetic imagery to real‐life footage. Photographic lenses can be simulated by ray tracing, but the performance depends on the complexity of the lens system, and some operations required for modern algorithms, such as deterministic connections, can be difficult to achieve. We generalise the approach of polynomial optics, i.e. expressing the light field transformation from the sensor to the outer pupil using a polynomial, to work with extreme wide angle (fisheye) lenses and aspherical elements. We also show how sparse polynomials can be constructed from the large space of high‐degree terms (we tested up to degree 15). We achieve this using a variant of orthogonal matching pursuit instead of a Taylor series when computing the polynomials. We show two applications: photorealistic rendering using Monte Carlo methods, where we introduce a new aperture sampling technique that is suitable for light tracing, and an interactive preview method suitable for rendering with deep images.     

Decoupled Space and Time Sampling of Motion and Defocus Blur for Unified Rendering of Transparent and Opaque Objects

We propose a unified rendering approach that jointly handles motion and defocus blur for transparent and opaque objects at interactive frame rates. Our key idea is to create a sampled representation of all parts of the scene geometry that are potentially visible at any point in time for the duration of a frame in an initial rasterization step. We store the resulting temporally‐varying fragments (t‐fragments) in a bounding volume hierarchy which is rebuild every frame using a fast spatial median construction algorithm. This makes our approach suitable for interactive applications with dynamic scenes and animations. Next, we perform spatial sampling to determine all t‐fragments that intersect with a specific viewing ray at any point in time. Viewing rays are sampled according to the lens uv‐sampling for depth‐of‐field effects. In a final temporal sampling step, we evaluate the predetermined viewing ray/t‐fragment intersections for one or multiple points in time. This allows us to incorporate all standard shading effects including transparency. We describe the overall framework, present our GPU implementation, and evaluate our rendering approach with respect to scalability, quality, and performance.     

Polynomial Optics: A Construction Kit for Efficient Ray‐Tracing of Lens Systems

Simulation of light transport through lens systems plays an important role in graphics. While basic imaging properties can be conveniently derived from linear models (like ABCD matrices), these approximations fail to describe nonlinear effects and aberrations that arise in real optics. Such effects can be computed by proper ray tracing, for which, however, finding suitable sampling and filtering strategies is often not a trivial task. Inspired by aberration theory, which describes the deviation from the linear ray transfer in terms of wavefront distortions, we propose a ray‐space formulation for nonlinear effects. In particular, we approximate the analytical solution to the ray tracing problem by means of a Taylor expansion in the ray parameters. This representation enables a construction‐kit approach to complex optical systems in the spirit of matrix optics. It is also very simple to evaluate, which allows for efficient execution on CPU and GPU alike, including the computation of mixed derivatives of any order. We evaluate fidelity and performance of our polynomial model, and show applications in high‐quality offline rendering and at interactive frame rates.     

Interactive Lenses for Visualization: An Extended Survey

The elegance of using virtual interactive lenses to provide alternative visual representations for selected regions of interest is highly valued, especially in the realm of visualization. Today, more than 50 lens techniques are known in the closer context of visualization, far more in related fields. In this paper, we extend our previous survey on interactive lenses for visualization. We propose a definition and a conceptual model of lenses as extensions of the classic visualization pipeline. An extensive review of the literature covers lens techniques for different types of data and different user tasks and also includes the technologies employed to display lenses and to interact with them. We introduce a taxonomy of lenses for visualization and illustrate its utility by dissecting in detail a multi‐touch lens for exploring large graph layouts. As a conclusion of our review, we identify challenges and unsolved problems to be addressed in future research.     

An Eyeglass Simulator Using Conoid Tracing

This paper proposes a method for displaying images at the fovea of the retina taking visual acuity into account. Previous research has shown that a point light source projected onto the retina forms an ellipse, which can be computed with wavefront tracing from each point in space. We propose a novel concept using conoid tracing, with which we can acquire defocusing information several times faster than that acquired by previous methods. We also show that conoid tracing is more robust and produces higher quality results. In conoid tracing the ray is regarded as a conoid, a thin cone‐like shape with varying elliptical cross‐section. The viewing ray from the retina is traced as a conoid and evaluated at each sample location. Using the sampled and pre‐computed data for the spatial distribution of blurring, we implemented an interactive eyeglass simulator. This paper demonstrates some visualization results utilizing the interactivity of the simulator, which an eyeglass lens design company uses to evaluate the design of complex progressive lenses.     

Shape and topology optimization of acoustic lens system using phase field method

A layout optimization method for a two-dimensional acoustic lens system used in underwater imaging is presented. To this end, a shape and topology optimization is formulated for the design problem of a lens system for the first time. The layout of a lens system to be optimized includes the number of lenses, shape of lens surfaces, distances between lenses, and lens materials. A phase field function is employed to implicitly parameterize the boundaries of the lenses, which move according to design sensitivities during optimization. Multiple lenses with different materials are optimized using a single phase field function. Because the ratio of the acoustic wavelength with respect to lens dimensions is large, diffraction effects should be taken into account. Accordingly, the performance of a lens system should be analyzed using wave acoustics and not the ray tracing method. The optimization problem is formulated to remove the aberrations of coma and field curvature. The validity of the proposed optimization method is demonstrated by solving benchmark design problems including a lens system with a large field of view.     

NeuroLens: Data‐Driven Camera Lens Simulation Using Neural Networks

Rendering with full lens model can offer images with photorealistic lens effects, but it leads to high computational costs. This paper proposes a novel camera lens model, NeuroLens, to emulate the imaging of real camera lenses through a data‐driven approach. The mapping of image formation in a camera lens is formulated as imaging regression functions (IRFs), which map input rays to output rays. IRFs are approximated with neural networks, which compactly represent the imaging properties and support parallel evaluation on a graphics processing unit (GPU). To effectively represent spatially varying imaging properties of a camera lens, the input space spanned by incident rays is subdivided into multiple subspaces and each subspace is fitted with a separate IRF. To further raise the evaluation accuracy, a set of neural networks is trained for each IRF and the output is calculated as the average output of the set. The effectiveness of the NeuroLens is demonstrated by fitting a wide range of real camera lenses. Experimental results show that it provides higher imaging accuracy in comparison to state‐of‐the‐art camera lens models, while maintaining the high efficiency for processing camera rays.     

基于Navarro示意眼模型的视觉真实感绘制*

结合人眼光学建模和计算机图形学的真实感绘制技术,提出一种基于Navarro示意眼模型的人眼视觉真实感绘制方法.利用Navarro模型与传统的单透镜模型相比能够更精确地模拟人眼的特性,将Navarro模型引入视觉真实感绘制中;采用光线追踪方法,加入非球面折射面的计算,精确地模拟人眼的成像特性.实验结果表明此种方法能够更精...     

The Undistort Lens

Detail‐in‐context lens techniques can be useful for exploring visualizations of data spaces that are too large or have too much detail to fit in regular displays. For example, by bending the space in the right way we can bring together details from two separate areas for easy comparison while roughly keeping the context that situates each area within the global space. While these techniques can be powerful tools, they also introduce distortions that need to be understood, and often the tools have to be disabled in order to have access to the undistorted data. We introduce the undistort lens, a complement to existing distortion‐based techniques that provides a local and separate presentation of the original geometry without affecting any distortion‐based lenses currently used in the presentation. The undistort lens is designed to allow interactive access to the underlying undistorted data within the context of the distorted space, and to enable a better understanding of the distortions. The paper describes the implementation of a generic back‐mapping mechanism that enables the implementation of undistort lenses for arbitrary distortion based techniques, including those presented in the lens literature. We also provide a series of use‐case scenarios that demonstrate the situations in which the technique can complement existing lenses.     

Three-dimensional large-aperture lens antennas with gradient refractive index

We propose an accurate method for designing three-dimensional （3D） large-aperture metamaterial slab lens antennas with gradient refractive index （GRIN）. According to the geometric optics, Fermat principle, ray-tracing technique and impedance matching, the 3D GRIN slab lenses with large apertures are accurately designed and simulated. With the aid of the effective medium theory, an X-band and a Ku-band conical horn antennas loaded with the 3D GRIN slab lenses of 250-mm diameter are experimentally realized using the drilling-hole technique on the printed circuit boards （PCBs） as the unit cells of metamaterials. Compared to the traditional dielectric lens with the same aperture, the proposed antennas have very good performance with high directivity, and the gain is increased by 2 to 5 dB. Using the same method, we design and realize a huge-aperture GRIN lens in the X band with a diameter of 1000 mm, which is composed of nearly one millions of inhomogeneous unit cells of square-ring resonators and dielectric blocks with drilling holes. Due to the huge aperture size, the electromagnetic ray paths inside and outside of the GRIN lens are verified and optimized using the ray tracing technique. Measurement results show good performance of the proposed antenna with high directivity.     

Feed and loss effects in spherical lens antennas for satellite communications

The hemispherical lens antenna is a candidate for satellite communications‐on‐the‐move, offering good scan performance in a reduced height. A short focal length minimizes height but presents challenges in illuminating the lens. Aperture efficiency is dominated by both the primary feed and dielectric loss. Feed effects are investigated in a threefold approach: spherical wave theory, commercial solver, and measurements. Gain and loss in a 432 mm diameter polyethylene/polystyrene lens are also measured. Gain for a waveguide‐fed array of two lenses is 36.3, 38.8, and 41.1 dBi, respectively, at 12.5, 20, and 30 GHz. The performance of a proposed four‐element array of equivalent area is then estimated.     

Perspective correct occlusion‐capable augmented reality displays using cloaking optics constraints

Perspective‐correct occlusion‐capable augmented reality displays are generalized using an optical cloak constraint for ray transfer analysis or simulations; any ray entering the optical system exits at the height and angle as if it passed through empty space. We analyze several systems with two‐lens, three‐lens, and four‐lens looped groups in inline, folded, and looped configurations. We design and demonstrate a four‐lens folded optical cloak and a three‐lens inverted cloak with an erecting prism.