ExpandNet: A Deep Convolutional Neural Network for High Dynamic Range Expansion from Low Dynamic Range Content

High dynamic range (HDR) imaging provides the capability of handling real world lighting as opposed to the traditional low dynamic range (LDR) which struggles to accurately represent images with higher dynamic range. However, most imaging content is still available only in LDR. This paper presents a method for generating HDR content from LDR content based on deep Convolutional Neural Networks (CNNs) termed ExpandNet. ExpandNet accepts LDR images as input and generates images with an expanded range in an end‐to‐end fashion. The model attempts to reconstruct missing information that was lost from the original signal due to quantization, clipping, tone mapping or gamma correction. The added information is reconstructed from learned features, as the network is trained in a supervised fashion using a dataset of HDR images. The approach is fully automatic and data driven; it does not require any heuristics or human expertise. ExpandNet uses a multiscale architecture which avoids the use of upsampling layers to improve image quality. The method performs well compared to expansion/inverse tone mapping operators quantitatively on multiple metrics, even for badly exposed inputs.     

Modeling a Generic Tone-mapping Operator

Although several new tone‐mapping operators are proposed each year, there is no reliable method to validate their performance or to tell how different they are from one another. In order to analyze and understand the behavior of tone‐mapping operators, we model their mechanisms by fitting a generic operator to an HDR image and its tone‐mapped LDR rendering. We demonstrate that the majority of both global and local tone‐mapping operators can be well approximated by computationally inexpensive image processing operations, such as a per‐pixel tone curve, a modulation transfer function and color saturation adjustment. The results produced by such a generic tone‐mapping algorithm are often visually indistinguishable from much more expensive algorithms, such as the bilateral filter. We show the usefulness of our generic tone‐mapper in backward‐compatible HDR image compression, the black‐box analysis of existing tone mapping algorithms and the synthesis of new algorithms that are combination of existing operators.     

Parallel implementation and optimization of high definition video real-time dehazing

In some warning applications, such as aircraft taking-off and landing, ship sailing, and traffic guidance in foggy weather, the high definition (HD) and rapid dehazing of images and videos is increasingly necessary. Existing technologies for the dehazing of videos or images have not completely exploited the parallel computing capacity of modern multi-core CPU and GPU, and leads to the long dehazing time or the low frame rate of video dehazing which cannot meet the real-time requirement. In this paper, we propose a parallel implementation and optimization method for the real-time dehazing of the high definition videos based on a single image haze removal algorithm. Our optimization takes full advantage of the modern CPU+GPU architecture, which increases the parallelism of the algorithm, and greatly reduces the computational complexity and the execution time. The optimized OpenCL parallel implementation is integrate into FFmpeg as an independent module. The experimental results show that for a single image, the performance of the optimized OpenCL algorithm is improved approximately 500% compared with the existing algorithm, and approximately 153% over the basic OpenCL algorithm. The 1080p (1920?×?1080) high definition hazy video can also processed at a real-time rate (more than 41 frames per second).     

Temporal Glare: Real-Time Dynamic Simulation of the Scattering in the Human Eye

Glare is a consequence of light scattered within the human eye when looking at bright light sources. This effect can be exploited for tone mapping since adding glare to the depiction of high-dynamic range (HDR) imagery on a low-dynamic range (LDR) medium can dramatically increase perceived contrast. Even though most, if not all, subjects report perceiving glare as a bright pattern that fluctuates in time, up to now it has only been modeled as a static phenomenon. We argue that the temporal properties of glare are a strong means to increase perceived brightness and to produce realistic and attractive renderings of bright light sources. Based on the anatomy of the human eye, we propose a model that enables real-time simulation of dynamic glare on a GPU. This allows an improved depiction of HDR images on LDR media for interactive applications like games, feature films, or even by adding movement to initially static HDR images. By conducting psychophysical studies, we validate that our method improves perceived brightness and that dynamic glare-renderings are often perceived as more attractive depending on the chosen scene.     

Revertible tone mapping of high dynamic range imagery: Integration to JPEG 2000

This paper presents a revertible tone mapping approach based on subband architecture where the dynamic range of the HDR (High Dynamic Range) image is decreased to LDR (Low Dynamic Range) to fit several types of applications. The LDR image can be later expanded to get back the original HDR content. One important benefit of the proposed approach is its backward compatibility with low dynamic (LDR) image applications since no extra information is needed to perform a very efficient HDR reconstruction. In order to improve the efficiency of our TM (Tone Mapping), we couple it with an optimisation procedure to minimize the reconstruction error. Subjective and objective comparisons with state-of-the-art methods have shown superior quality results of both tone mapped and reconstructed images. As a potential application, the integration of the proposed tone mapping to JPEG 2000 encoder achieved competitive performance compared to reference HDR image encoders.     

A real-time implementation of gradient domain high dynamic range compression using a local Poisson solver

This paper presents a real-time hardware implementation of a gradient domain dynamic range compression algorithm for high dynamic range (HDR) images. This technique works by calculating the gradients of the HDR image, manipulating those gradients, and reconstructing an output low dynamic range image that corresponds to the manipulated gradients. Reconstruction involves solving the Poisson equation. We propose a Poisson solver that utilizes only local information around each pixel along with special boundary conditions, and requires a small and fixed amount of hardware for any image size, with no need to buffer the entire image. The hardware implementation is described in VHDL and synthesized for a field programmable gate array (FPGA) device. The maximum operating frequency achieved is fast enough to process high dynamic range videos with one megapixel per frame at a rate of about 100 frames per second. The hardware is tested on standard HDR images from the Debevec library. The output images produced have good visual quality.     

基于CUDA的声辐射力弹性成像算法研究

声辐射力弹性成像是一种新的测量组织硬度的超声成像方法。不同于其他超声组织弹性成像方法,声辐射力弹性成像能够定量测量组织的弹性模量数值,并且具有对操作者经验依赖性低的特点。然而,由于成像算法数据处理量大,运算时间长,声辐射力弹性成像还无法进行准实时的二维成像。为了获得实时的二维声辐射力弹性图像,提出并实现了一种适合于在GPU上并行计算的声辐射力弹性成像算法。通过与运行在CPU上的原始声辐射力弹性成像算法进行对比,证明在GPU上实现的算法大幅度地提高了运算速度。在自制弹性仿体上,比较了基于GPU和CPU两种算法所成的二维弹性分布图像的质量,结果证明两者的图像质量没有明显差异。     

基于细节再现的高动态范围图像分层映射算法

针对当前映射算法中亮度的映射函数非适性而引起对比度过度压缩的问题,以及映射时亮度变化改变图像细节可见性的问题,提出了一种基于细节再现的高动态范围(HDR)图像分层映射算法。该算法采用视觉响应曲线作为基础层的映射函数,根据图像局部适应性亮度动态地映射亮度;同时,在Stevens效应的思想基础上依据映射前后亮度变化获得补偿系数,拉伸或压缩细节层。测试结果表明：该映射算法所得图像能正确再现更多的可见细节。     

High dynamic range image tone mapping and retexturing using fast trilateral filtering

Using fast trilateral filtering we present a novel tone mapping and retexturing method for high dynamic range (HDR) images. Our new trilateral filtering-based tone mapping is about seven to ten times faster than that in [3]. Firstly, a novel tone mapping algorithm for HDR images is presented. It is based on fast bilateral filtering and two newly developed filters: the quasi-Cauchy function kernel filter and the fourth degree Taylor polynomial kernel filter. Secondly, a new gradient-based image retexturing method is introduced, which consists of three steps: 1) converting HDR images into low dynamic range (LDR) images using our fast trilateral filtering-based tone mapping method; 2) recovering the gradient luminance maps for the region to be retextured; 3) reconstructing the final retextured image by solving the Poisson equation. The proposed approach is suitable for HDR image tone mapping and retexturing, and experimental results have demonstrated the satisfactory performance of our method.     

Performance engineering to achieve real-time high dynamic range imaging

Image-processing applications like high dynamic range imaging can be done efficiently in the gradient space. For it, the image has to be transformed to gradient space and back. While the forward transformation to gradient space is fast by using simple finite differences, the backward transformation requires the solution of a partial differential equation. Although one can use an efficient multigrid solver for the backward transformation, it shows that a straightforward implementation of the standard algorithm does not lead to satisfactory runtime results for real-time high dynamic range compression of larger 2D X-ray images even on GPUs. Therefore, we do a rigorous performance analysis and derive a performance model for our multigrid algorithm that guides us to an improved implementation, where we achieve an overall performance of more than 25 frames per second for 16.8 Megapixel images doing full high dynamic range compression including data transfers between CPU and GPU. Together with a simple OpenGL visualization it becomes possible to perform real-time parameter studies on medical data sets.     

基于灰度级映射函数建模的多曝光高动态图像重建

为解决传统多曝光图像融合的实时性和动态场景鬼影消除问题,提出了基于灰度级映射函数建模的多曝光高动态图像重建算法。对任意大小的低动态范围（Low dynamic range,LDR）图像序列,仅需拟合与灰阶数目相同个数而不是与相机分辨率个数相同的视觉适应的S形曲线,利用最佳成像值判别方法直接融合,提高了算法的融合效率,能够达到实时性图像融合要求。对动态场景的融合,设计灰度级映射关系恢复理想状态的多曝光图像,利用差分法检测运动目标区域,作鬼影消除处理,融合得到一幅能够反映真实场景信息且不受鬼影影响的高动态范围图像。     

Rendering of HDR images to improve brightness discrimination

The impression of quality of images can be enhanced on a high dynamic range (HDR) displays. Generally, a conventional 8‐bit image can be processed to an HDR image by inverse tone mapping operators. Among the operators, brightness discrimination mapping by applying brightness adaptation model attempted to mimic the human visual system. In this paper, we use a brightness adaptation model to derive a brightness discrimination mapping algorithm for HDR displays. The proposed algorithm maximizes a function, which represents the local and global brightness discrimination range by exploiting characteristics of the human visual system. Enhancement of details is verified by visualizing HDR images from dark to bright regions. Improvement of dynamic range is quantified by measuring increased discrimination ratio.     

一种基于细节层分离的单曝光HDR图像生成算法

针对利用单幅低动态范围（Low dynamic range,LDR）图像生成高动态范围（High dynamic range,HDR）图像细节信息不足的问题,本文提出了一种基于细节层分离的单曝光HDR图像生成算法.该算法基于人类视觉系统模型,首先分别提取出LDR图像的亮度分量和色度分量,对伽马校正后的亮度分量进行双边滤波,提取出亮度分量的基本层,再对基本层和亮度分量进行遍历运算,得到亮度分量的细节层;然后,构造反色调映射函数,分别对细节层和伽马校正后的亮度图像进行扩展,得到各自的反色调映图像;之后,将反色调映射后亮度分量与压缩后的细节层进行融合,得到新的亮度分量.最后,融合色度分量与新的亮度分量,并对融合后图像进行去噪,得到最终的HDR图像.实验表明该算法能挖掘出部分隐藏的图像细节信息,处理效果较好,运行效率高,具有较好的鲁棒性.     

A Psychophysical Evaluation of Inverse Tone Mapping Techniques

In recent years inverse tone mapping techniques have been proposed for enhancing low-dynamic range (LDR) content for a high-dynamic range (HDR) experience on HDR displays, and for image based lighting. In this paper, we present a psychophysical study to evaluate the performance of inverse (reverse) tone mapping algorithms. Some of these techniques are computationally expensive because they need to resolve quantization problems that can occur when expanding an LDR image. Even if they can be implemented efficiently on hardware, the computational cost can still be high. An alternative is to utilize less complex operators; although these may suffer in terms of accuracy. Our study investigates, firstly, if a high level of complexity is needed for inverse tone mapping and, secondly, if a correlation exists between image content and quality. Two main applications have been considered: visualization on an HDR monitor and image-based lighting.     

Reconstruction of high contrast images for dynamic scenes

High Dynamic Range (HDR) imaging requires one to composite multiple, differently exposed images of a scene in the irradiance domain and perform tone mapping of the generated HDR image for displaying on Low Dynamic Range (LDR) devices. In the case of dynamic scenes, standard techniques may introduce artifacts called ghosts if the scene changes are not accounted for. In this paper, we consider the blind HDR problem for dynamic scenes. We develop a novel bottom-up segmentation algorithm through superpixel grouping which enables us to detect scene changes. We then employ a piecewise patch-based compositing methodology in the gradient domain to directly generate the ghost-free LDR image of the dynamic scene. Being a blind method, the primary advantage of our approach is that we do not assume any knowledge of camera response function and exposure settings while preserving the contrast even in the non-stationary regions of the scene. We compare the results of our approach for both static and dynamic scenes with that of the state-of-the-art techniques.     

Application‐Specific Tone Mapping Via Genetic Programming

High dynamic range (HDR) imagery permits the manipulation of real‐world data distinct from the limitations of the traditional, low dynamic range (LDR), content. The process of retargeting HDR content to traditional LDR imagery via tone mapping operators (TMOs) is useful for visualizing HDR content on traditional displays, supporting backwards‐compatible HDR compression and, more recently, is being frequently used for input into a wide variety of computer vision applications. This work presents the automatic generation of TMOs for specific applications via the evolutionary computing method of genetic programming (GP). A straightforward, generic GP method that generates TMOs for a given fitness function and HDR content is presented. Its efficacy is demonstrated in the context of three applications: Visualization of HDR content on LDR displays, feature mapping and compression. For these applications, results show good performance for the generated TMOs when compared to traditional methods. Furthermore, they demonstrate that the method is generalizable and could be used across various applications that require TMOs but for which dedicated successful TMOs have not yet been discovered.     

Evaluation of tone mapping operators in night-time virtual worlds

The subjective quality of a virtual world depends on the quality of displayed images. In the present paper, we address a technical aspect of image quality in virtual environments. Due to the recent development of high dynamic range (HDR) imaging in computer graphics applications, tone mapping operators (TMO) are needed in the graphic pipeline, and their impact on the final image quality needs to be tested. Previous evaluations of such operators have emphasized the fact that the specific merit of a given operator may depend on both the scene and the application. The dynamic behavior of tone mapping operators was not tested before, and we have designed two psychophysical experiments in order to assess the relevance of various TMO for a specific class of virtual worlds, outdoor scenes at night and an interactive application, to explore an outdoor virtual world at night. In a first experiment, 5 HDR video clips were tone-mapped using 8 operators from the literature, resulting in 40 videos. These 40 videos were presented to 14 subjects, which were asked to rate their realism. However, the subject’s evaluation was not a direct comparison with the HDR videos. In a second experiment, 9 HDR photographs of urban scenes at night were tone-mapped with the same 8 operators. The resulting 72 photographs were presented to 13 subjects, at the location where the photographs were taken. The subjects were asked to rate the realism of each tone-mapped image, displayed on a laptop, with respect to the physical scene they experienced. The first experiment emphasized the importance of modeling the temporal visual adaptation for a night-time application.     

Physiological inverse tone mapping based on retina response

The mismatch between the Low Dynamic Range (LDR) content and the High Dynamic Range (HDR) display arouses the research on inverse tone mapping algorithms. In this paper, we present a physiological inverse tone mapping algorithm inspired by the property of the Human Visual System (HVS). It first imitates the retina response and deduce it to be local adaptive; then estimates local adaptation luminance at each point in the image; finally, the LDR image and local luminance are applied to the inversed local retina response to reconstruct the dynamic range of the original scene. The good performance and high-visual quality were validated by operating on 40 test images. Comparison results with several existing inverse tone mapping methods prove the conciseness and efficiency of the proposed algorithm.     

Robust artifact-free high dynamic range imaging of dynamic scenes

The irradiance range of the real-world scene is often beyond the capability of digital cameras. Therefore, High Dynamic Range (HDR) images can be generated by fusing images with different exposure of the same scene. However, moving objects pose the most severe problem in the HDR imaging, leading to the annoying ghost artifacts in the fused image. In this paper, we present a novel HDR technique to address the moving objects problem. Since the input low dynamic range (LDR) images captured by a camera act as static linear related backgrounds with moving objects during each individual exposures, we formulate the detection of foreground moving objects as a rank minimization problem. Meanwhile, in order to eliminate the image blurring caused by background slightly change of LDR images, we further rectify the background by employing the irradiances alignment. Experiments on image sequences show that the proposed algorithm performs significant gains in synthesized HDR image quality compare to state-of-the-art methods.