Evaluation of HDR tone‐mapping algorithms using a high‐dynamic‐range display to emulate real scenes

Abstract— Current HDR display technology approaches the dynamic‐range capabilities of the fully adapted human visual system. As such, this technology has potential for performing as a surrogate for real‐world scenes in the perceptual evaluation of high‐dynamic‐range (HDR) image‐reproduction algorithms that aim to map HDR scenes to the limited dynamic ranges available in typical display and print technology. Compared with direct image assessment in comparison with real‐world scenes, it is clear that use of HDR display technology has the benefit of simplicity in experimental design while maintaining the HDR of the original scene. To evaluate this potential application of HDR display technology, seven published versions of well‐known HDR tone‐mapping algorithms were benchmarked for perceptual rendering accuracy against each of four real‐world scenes constructed in the laboratory and against corresponding images on an HDR display. The results illustrate that visual assessments obtained from the HDR display and those obtained from real‐world scenes are in good agreement, validating the potential for HDR display technology as an evaluation tool in this context.     

Rendering of HDR images to improve brightness discrimination

The impression of quality of images can be enhanced on a high dynamic range (HDR) displays. Generally, a conventional 8‐bit image can be processed to an HDR image by inverse tone mapping operators. Among the operators, brightness discrimination mapping by applying brightness adaptation model attempted to mimic the human visual system. In this paper, we use a brightness adaptation model to derive a brightness discrimination mapping algorithm for HDR displays. The proposed algorithm maximizes a function, which represents the local and global brightness discrimination range by exploiting characteristics of the human visual system. Enhancement of details is verified by visualizing HDR images from dark to bright regions. Improvement of dynamic range is quantified by measuring increased discrimination ratio.     

基于灰度级映射函数建模的多曝光高动态图像重建

为解决传统多曝光图像融合的实时性和动态场景鬼影消除问题,提出了基于灰度级映射函数建模的多曝光高动态图像重建算法。对任意大小的低动态范围（Low dynamic range,LDR）图像序列,仅需拟合与灰阶数目相同个数而不是与相机分辨率个数相同的视觉适应的S形曲线,利用最佳成像值判别方法直接融合,提高了算法的融合效率,能够达到实时性图像融合要求。对动态场景的融合,设计灰度级映射关系恢复理想状态的多曝光图像,利用差分法检测运动目标区域,作鬼影消除处理,融合得到一幅能够反映真实场景信息且不受鬼影影响的高动态范围图像。     

High‐performance local‐dimming algorithm based on image characteristic and logarithmic function

With the emergence of high‐dynamic range (HDR) image and video, displaying an HDR image or video on a standard dynamic range displays with natural visual effect, clear details on local areas, and high‐contrast ratio has become an important issue. To achieve HDR, local‐dimming technique has been commonly practiced. In this paper, we investigate the problem of local dimming for LED‐backlight LC display to provide algorithm support for developing HDR displays. A novel local‐dimming algorithm is proposed to improve the contrast ratio, enhance the visual quality, and reduce the power consumption of the LCDs. The proposed algorithm mainly consists of two parts. The first part is a backlight luminance extraction method based on dynamic threshold and the maximum grayscale of an image block to improve the contrast ratio and reduce the power consumption. In the second one, a pixel compensation method based on logarithmic function is used to improve the visual quality and contrast ratio. At the same time, in order to better smooth the backlight diffusion at the edges of the backlight luminance signal to enhance the accuracy of the pixel compensation, we draw on the idea of BMA and improve it to establish the backlight diffusion model with different low‐pass‐filter templates for different types of blocks. Simulation and measured results show that the proposed algorithm outperforms the competing ones in contrast ratio, visual quality, and power‐saving ratio.     

Attention-based high dynamic range imaging

Many tone mapping algorithms have been proposed based on the studies in Human Visual System; however, they rarely addressed the effects of attention to contrast response. As attention plays an important role in human visual system, we proposed a local tone mapping method that respects both attention and adaptation effects. We adopt the High Dynamic Range (HDR) saliency map to compute an attention map, which predicts the attentive regions and nonattentative regions in an HDR image. The attention map is then used to locally adjust the contrast of the HDR image according to attention and adaptation models found in psychophysics. We applied our tone mapping approach to HDR images and videos and compared with the results generated by three state-of-the-art tone mapping algorithms. Our experiments show that our approach produces results with better image quality in terms of preserving details and chromaticity of visual saliency.     

Retinal HDR images: Intraocular glare and object size

Abstract— Starting from measured scene luminances, the retinal images of high‐dynamic‐range (HDR) test targets were calculated. These test displays contain 40 gray squares with a 50% average surround. In order to approximate a natural scene, the surround area was made up of half‐white and half‐black squares of different sizes. In this display, the spatial‐frequency distribution approximates a 1/f function of energy vs. spatial frequency. Images with 2.7 and 5.4 optical density ranges were compared. Although the target luminances are very different, after computing the retinal image according to the CIE scatter glare formula, it was found that the retinal ranges are very similar. Intraocular glare strongly restricts the range of the retinal image. Furthermore, uniform, equiluminant target patches are spatially transformed to different gradients with unequal retinal luminances. The usable dynamic range of the display correlates with the range on the retina. Observers report that appearances of white and black squares are constant and uniform, despite the fact that the retinal stimuli are variable and non‐uniform. Human vision uses complex spatial processing to calculate appearance from retinal arrays. Spatial image processing increases apparent contrast with increased white area in the surround. Post‐retinal spatial vision counteracts glare.     

一种高动态范围图像可视化算法*

提出的自适应HDR图像可视化算法中,输入图像被分解为基本层和细节层.该算法将整体明暗效果的显示看做整体问题,对表示亮度的基本层采用基于整体统计信息的直方图调整算法处理;可视细节信息的保持作为局部问题,算法采用自适应细节增强算法处理细节层.通过定义映射图对细节增强后的图像进行最终映射,将两方面结合起来得到最终结果.实验结果表明,该算法能对HDR图像进行较高视觉质量的显示.     

Tone Reproduction: A Perspective from Luminance-Driven Perceptual Grouping

We address the tone reproduction problem by integrating the local adaptation effect with the consistency in global contrast impression. Many previous works on tone reproduction have focused on investigating the local adaptation mechanism of human eyes to compress high-dynamic-range (HDR) luminance into a displayable range. Nevertheless, while the realization of local adaptation properties is not theoretically defined, exaggerating such effects often leads to unnatural visual impression of global contrast. We propose to perceptually decompose the luminance into a small number of regions that sequentially encode the overall impression of an HDR image. A piecewise tone mapping can then be constructed to region-wise perform HDR compressions, using local mappings constrained by the estimated global perception. Indeed, in our approach, the region information is used not only to practically approximate the local properties of luminance, but more importantly to retain the global impression. Besides, it is worth mentioning that the proposed algorithm is efficient, and mostly does not require excessive parameter fine-tuning. Our experimental results and comparisons indicate that the described framework gives a good balance in both preserving local details and maintaining global perceptual impression of HDR scenes. First online version published in October, 2005     

一种基于视觉特性的仿生图像增强算法

常见的基于人类视觉特性的图像增强算法由于是同时完成动态范围压缩和对比度增强,导致增强图像的整体对比度不高、边缘部分效果不佳.通过分析人类视觉系统的全局和局部自适应调节原理及人眼视网膜神经节细胞感受野的传输特性,提出一种仿生图像增强算法.为适应人类视觉系统对光强的主观感觉特性,对图像作全局亮度对数变换;并利用人眼的主观亮度感觉与实际光强的对数呈局部线性关系的特性,采用视网膜神经元感受野三高斯模型来调整亮度图像的局部对比度;最后利用线性变换恢复图像的彩色信息.实验结果表明,该算法的增强效果良好,特别是对于图像边界处,既能很好地增强边缘对比,又可有效地提升区域亮度对比和亮度梯度信息.     

高动态范围图像梯度压缩算法

高动态范围（HDR）图像是一种可以表示实际场景中亮度大范围变化的图像类型,图像中的像素值正比于场景中对应点的实际亮度值,因此,可以更好地表示场景中亮区和暗区的光学特性。为了在常规显示硬件上显示HDR图像,采用梯度压缩算法,在亮度图像梯度域上对大梯度进行衰减,压缩图像亮度的动态范围。实验结果表明,该算法能对HDR图像进行较高视觉质量的显示。     

High Dynamic Range Point Clouds for Real‐Time Relighting

Acquired 3D point clouds make possible quick modeling of virtual scenes from the real world. With modern 3D capture pipelines, each point sample often comes with additional attributes such as normal vector and color response. Although rendering and processing such data has been extensively studied, little attention has been devoted using the light transport hidden in the recorded per‐sample color response to relight virtual objects in visual effects (VFX) look‐dev or augmented reality (AR) scenarios. Typically, standard relighting environment exploits global environment maps together with a collection of local light probes to reflect the light mood of the real scene on the virtual object. We propose instead a unified spatial approximation of the radiance and visibility relationships present in the scene, in the form of a colored point cloud. To do so, our method relies on two core components: High Dynamic Range (HDR) expansion and real‐time Point‐Based Global Illumination (PBGI). First, since an acquired color point cloud typically comes in Low Dynamic Range (LDR) format, we boost it using a single HDR photo exemplar of the captured scene that can cover part of it. We perform this expansion efficiently by first expanding the dynamic range of a set of renderings of the point cloud and then projecting these renderings on the original cloud. At this stage, we propagate the expansion to the regions not covered by the renderings or with low‐quality dynamic range by solving a Poisson system. Then, at rendering time, we use the resulting HDR point cloud to relight virtual objects, providing a diffuse model of the indirect illumination propagated by the environment. To do so, we design a PBGI algorithm that exploits the GPU's geometry shader stage as well as a new mipmapping operator, tailored for G‐buffers, to achieve real‐time performances. As a result, our method can effectively relight virtual objects exhibiting diffuse and glossy physically‐based materials in real time. Furthermore, it accounts for the spatial embedding of the object within the 3D environment. We evaluate our approach on manufactured scenes to assess the error introduced at every step from the perfect ground truth. We also report experiments with real captured data, covering a range of capture technologies, from active scanning to multiview stereo reconstruction.     

高动态范围图像和色阶映射算子

图像传感器动态响应范围的局限使其在捕捉高动态范围场景时力不从心, 为了捕捉高动态范围图像(High dynamic range image, HDRI), 近年来出现了许多新型传感器和新方法, 本文将简要介绍这些研究进展; 同样由于动态响应范围的局限, 显示设备也不能胜任HDRI的显示, 必须利用色阶映射算子(Tone mapping operator, TMO)将图像的动态范围进行合理的压缩, TMO最终决定了图像显示的质量, 本文将众多的TMO归纳为全局算子和局部算子并进行了详细论述.     

Evaluation of tone mapping operators in night-time virtual worlds

The subjective quality of a virtual world depends on the quality of displayed images. In the present paper, we address a technical aspect of image quality in virtual environments. Due to the recent development of high dynamic range (HDR) imaging in computer graphics applications, tone mapping operators (TMO) are needed in the graphic pipeline, and their impact on the final image quality needs to be tested. Previous evaluations of such operators have emphasized the fact that the specific merit of a given operator may depend on both the scene and the application. The dynamic behavior of tone mapping operators was not tested before, and we have designed two psychophysical experiments in order to assess the relevance of various TMO for a specific class of virtual worlds, outdoor scenes at night and an interactive application, to explore an outdoor virtual world at night. In a first experiment, 5 HDR video clips were tone-mapped using 8 operators from the literature, resulting in 40 videos. These 40 videos were presented to 14 subjects, which were asked to rate their realism. However, the subject’s evaluation was not a direct comparison with the HDR videos. In a second experiment, 9 HDR photographs of urban scenes at night were tone-mapped with the same 8 operators. The resulting 72 photographs were presented to 13 subjects, at the location where the photographs were taken. The subjects were asked to rate the realism of each tone-mapped image, displayed on a laptop, with respect to the physical scene they experienced. The first experiment emphasized the importance of modeling the temporal visual adaptation for a night-time application.     

Visual Perception in Realistic Image Synthesis

Realism is often a primary goal in computer graphics imagery, and we strive to create images that are perceptually indistinguishable from an actual scene. Rendering systems can now closely approximate the physical distribution of light in an environment. However, physical accuracy does not guarantee that the displayed images will have authentic visual appearance. In recent years the emphasis in realistic image synthesis has begun to shift from the simulation of light in an environment to images that look as real as the physical environment they portray. In other words the computer image should be not only physically correct but also perceptually equivalent to the scene it represents. This implies aspects of the Human Visual System (HVS) must be considered if realism is required. Visual perception is employed in many different guises in graphics to achieve authenticity. Certain aspects of the visual system must be considered to identify the perceptual effects that a realistic rendering system must achieve in order to reproduce effectively a similar visual response to a real scene. This paper outlines the manner in which knowledge about visual perception is increasingly appearing in state‐of‐the‐art realistic image synthesis. After a brief overview of the HVS, this paper is organized into four sections, each exploring the use of perception in realistic image synthesis, each with slightly different emphasis and application. First, Tone Mapping Operators, which attempt to map the vast range of computed radiance values to the limited range of display values, are discussed. Then perception based image quality metrics, which aim to compare images on a perceptual rather than physical basis, are presented. These metrics can be used to evaluate, validate and compare imagery. Thirdly, perception driven rendering algorithms are described. These algorithms focus on embedding models of the HVS directly into global illumination computations in order to improve their efficiency. Finally, techniques for comparing computer graphics imagery against the real world scenes they represent are discussed.     

High‐resolution 360 Video Foveated Stitching for Real‐time VR

In virtual reality (VR) applications, the contents are usually generated by creating a 360° Video panorama of a real‐world scene. Although many capture devices are being released, getting high‐resolution panoramas and displaying a virtual world in real‐time remains challenging due to its computationally demanding nature. In this paper, we propose a real‐time 360° Video foveated stitching framework, that renders the entire scene in different level of detail, aiming to create a high‐resolution panoramic Video in real‐time that can be streamed directly to the client. Our foveated stitching algorithm takes Videos from multiple cameras as input, combined with measurements of human visual attention (i.e. the acuity map and the saliency map), can greatly reduce the number of pixels to be processed. We further parallelize the algorithm using GPU to achieve a responsive interface and validate our results via a user study. Our system accelerates graphics computation by a factor of 6 on a Google Cardboard display.     

基于亮度分区模糊融合的高动态范围成像算法

针对单幅图像生成高动态范围（HDR）图像进行直方图扩展时,造成的色彩失真、局部细节信息丢失的问题,提出了一种基于亮度分区融合的高动态范围图像成像算法。首先,提取正常曝光彩色图像的亮度分量,根据亮度阈值将亮度分成两个区间;然后,对两个区间的图像用改进的指数函数扩展其亮度范围,使得低亮度区域的亮度增加、范围扩大,高亮度区域的亮度减小、范围扩大,从而增大图像的整体对比度,保留色彩和细节信息;最后,将扩展后的图像和原始正常曝光的图像基于模糊逻辑的方法融合为高动态图像。分别从主观和客观两方面对所提算法进行了分析。实验结果表明,所提算法能够有效地扩展图像的亮度范围,并保持场景的颜色信息和细节信息,生成的图像视觉效果更佳。     

A study on user preference of high dynamic range over low dynamic range video

The increased interest in high dynamic range (HDR) video over existing low dynamic range (LDR) video during the past decade or so was primarily due to its inherent capability to capture, store and display the full range of real-world lighting visible to the human eye with increased precision. This has led to an inherent assumption that HDR video would be preferable by the end-user over LDR video due to the more immersive and realistic visual experience provided by HDR. This assumption has led to a considerable body of research into efficient capture, processing, storage and display of HDR video. Although this is beneficial for scientific research and industrial purposes, very little research has been conducted to test the veracity of this assumption. In this paper, we conduct two subjective studies by means of a ranking and a rating-based experiment where 60 participants in total, 30 in each experiment, were tasked to rank and rate several reference HDR video scenes along with three mapped LDR versions of each scene on an HDR display, in order of their viewing preference. Results suggest that given the option, end-users prefer the HDR representation of the scene over its LDR counterpart.     

Single‐shot High Dynamic Range Imaging Using Coded Electronic Shutter

Typical high dynamic range (HDR) imaging approaches based on multiple images have difficulties in handling moving objects and camera shakes, suffering from the ghosting effect and the loss of sharpness in the output HDR image. While there exist a variety of solutions for resolving such limitations, most of the existing algorithms are susceptible to complex motions, saturation, and occlusions. In this paper, we propose an HDR imaging approach using the coded electronic shutter which can capture a scene with row‐wise varying exposures in a single image. Our approach enables a direct extension of the dynamic range of the captured image without using multiple images, by photometrically calibrating rows with different exposures. Due to the concurrent capture of multiple exposures, misalignments of moving objects are naturally avoided with significant reduction in the ghosting effect. To handle the issues with under‐/over‐exposure, noise, and blurs, we present a coherent HDR imaging process where the problems are resolved one by one at each step. Experimental results with real photographs, captured using a coded electronic shutter, demonstrate that our method produces a high quality HDR images without the ghosting and blur artifacts.     

Segmentation and adaptive assimilation for detail-preserving display of high-dynamic range images

Realistic display of high-dynamic range images is a difficult problem. Previous methods for high-dynamic range image display suffer from halo artifacts or are computationally expensive. We present a novel method for computing local adaptation luminance that can be used with several different visual adaptation-based tone-reproduction operators for displaying visually accurate high-dynamic range images. The method uses fast image segmentation, grouping, and graph operations to generate local adaptation luminance. Results on several images show excellent dynamic range compression, while preserving detail without the presence of halo artifacts. With adaptive assimilation, the method can be configured to bring out a high-dynamic range appearance in the display image. The method is efficient in terms of processor and memory use.