High dynamic range imaging with short- and long-exposures based on artificial remapping using multiscale exposure fusion

High dynamic range imaging (HDRI) is an excellent high-quality image acquisition technique, which can reflect real human visual characteristics from one (or several) captured low dynamic range (LDR) image. However, the input LDR image only provides partial information of the scene. Besides, in traditional HDRI methods that require multiple captured images as input, field of view errors can be induced, which will be difficult to apply it to the emerging image acquisition systems. Here, we propose a novel HDRI method that reconstructs an HDR image from only a pair of short- and long-exposure images based on artificial remapping using multi-scale exposure fusion. Firstly, we introduce a simulated exposure model called artificial remapping to synthesize a multi-exposure image sequence from the input LDR image pairs. Then, weighting maps of the sequence for fusion can be obtained according to the evaluation factors of contrast, saturation, as well as improved exposedness. Finally, we utilize the pyramid based multiscale exposure fusion framework to integrate them into an enhanced HDR image. Comparative experiments, fully implemented on some source images, have been demonstrated that better performance can be realized compared with some competing methods in qualitative and quantitative evaluation. Note that the operation of the proposed method is simple yet effective, which is easy to popularize. The method thus can be potentially applied to the emerging image acquisition systems where two images are captured simultaneously by two image sensors or by one image sensor with a pair of short- and long-exposure setting.     

Extending depth of field and dynamic range from differently focused and exposed images

Focus stacking and high dynamic range (HDR) imaging are two paradigms of computational photography. Focus stacking aims to produce an image with greater depth of field (DOF) from a set of images taken with different focus distances; HDR imaging aims to produce an image with higher dynamic range from a set of images taken with different exposure values. In this paper, we present an algorithm which combines focus stacking and HDR imaging in order to produce an image with both extended DOF and dynamic range from a set of differently focused and exposed images. The key step in our algorithm is focus stacking regardless of the differences in exposure values of input images. This step includes photometric and spatial registration of images, and image fusion to produce all-in-focus images. This is followed by HDR radiance estimation and tonemapping. We provide experimental results with real data to illustrate the algorithm.     

Minimal-bracketing sets for high-dynamic-range image capture

This paper considers the problem of high-dynamic-range (HDR) image capture using low-dynamic-range (LDR) cameras. We present three different minimal-bracketing algorithms for computing minimum-sized exposure sets bracketing of HDR scenes. Each algorithm is applicable to a different HDR-imaging scenario depending on the amount of target-scene-irradiance information and real-time image processing available at the time of image acquisition. We prove the optimality of each algorithm with respect to its ability to obtain a theoretically minimum-size bracketing set of exposures. We also provide closed-form expressions for computing minimal-bracketing exposure sets for two common types of HDR-imaging systems, those with geometrically varying and arithmetically varying exposure settings. We experimentally demonstrate the advantages of the proposed methods by capturing and processing multiple HDR scenes using minimal-bracketing and 1-stop bracketing methods. The results show that minimal-bracketing can be used to produce high-quality HDR images, while requiring only one third as many LDR images be acquired compared to 1-stop bracketing. We also perform a detailed SNR analysis that quantifies the tradeoff between signal-to-noise ratio and image-bracketing-set size.     

Noise reduction in high dynamic range images

Multiple images with different exposures are used to produce a high dynamic range (HDR) image. Sometimes high-sensitivity setting is needed for capturing images in low light condition as in an indoor room. However, current digital cameras do not produce a high-quality HDR image when noise occurs in low light condition or high-sensitivity setting. In this paper, we propose a noise reduction method in generating HDR images using a set of low dynamic range (LDR) images with different exposures, where ghost artifacts are effectively removed by image registration and local motion information. In high-sensitivity setting, motion information is used in generating a HDR image. We analyze the characteristics of the proposed method and compare the performance of the proposed and existing HDR image generation methods, in which Reinhard et al.’s global tone mapping method is used for displaying the final HDR images. Experiments with several sets of test LDR images with different exposures show that the proposed method gives better performance than existing methods in terms of visual quality and computation time.     

High dynamic range video reconstruction from a stereo camera setup

To overcome the dynamic range limitations in images taken with regular consumer cameras, several methods exist for creating high dynamic range (HDR) content. Current low-budget solutions apply a temporal exposure bracketing which is not applicable for dynamic scenes or HDR video. In this article, a framework is presented that utilizes two cameras to realize a spatial exposure bracketing, for which the different exposures are distributed among the cameras. Such a setup allows for HDR images of dynamic scenes and HDR video due to its frame by frame operating principle, but faces challenges in the stereo matching and HDR generation steps. Therefore, the modules in this framework are selected to alleviate these challenges and to properly handle under- and oversaturated regions. In comparison to existing work, the camera response calculation is shifted to an offline process and a masking with a saturation map before the actual HDR generation is proposed. The first aspect enables the use of more complex camera setups with different sensors and provides robust camera responses. The second one makes sure that only necessary pixel values are used from the additional camera view, and thus, reduces errors in the final HDR image. The resulting HDR images are compared with the quality metric HDR-VDP-2 and numerical results are given for the first time. For the Middlebury test images, an average gain of 52 points on a 0-100 mean opinion score is achieved in comparison to temporal exposure bracketing with camera motion. Finally, HDR video results are provided.     

基于主成分分析和引导滤波的色调映射算法

针对传统高动态范围(HDR)图像色调映射算法普遍 存在色偏和效率低的问题,提出了一种基于主成分 分析(PCA)进行色彩空间转换和具有边缘保持特性的引导滤波图像增强的色调映射算法。首 先利用图像的平均亮 度实现图像Gamma自适应校正;然后采用PCA方法将图像的3个颜色分量转换为相 互正交的亮度 分量和色差分量,模拟相机响应函数(CRF)曲线压缩亮度分量的动态范围,并将结果与加权 后的色差分量联合, 经PCA逆转换得到新的RGB分量;最后利用引导滤波方法对图像进行增强,提高 图像对比度,避 免光晕现象。实验结果表明,本文算法效率比经典的iCAM06算法提高了4倍以 上,并能有效避免偏色和 光晕现象,处理后图像的细节清晰层次感强、色彩自然丰富,不仅能用于传统的便携式显示 设备,而且可用于穿戴式智能视觉辅助设备和无屏幕显示器等新兴领域。     

基于双边滤波的多尺度分层色调映射算法

为增强高动态范围图像(High dynamic range,HDR)的显示效果,本文提出一种多尺度梯度域色调映射算法。首先提取高动态范围图像的亮度信息,利用双边滤波将高动态范围图像的亮度数据进行多尺度分解,得到基本层和细节层。由于视觉上的亮度变化体现为图像数据的梯度变化,因此可在梯度域对基本层图像进行自适应动态范围压缩,然后加上细节层图像信息,最后再通过色彩校正算法恢复图像色彩,实现HDR图像的动态范围压缩。通过对比算法的定量分析表明,本文算法的方差和信息熵的客观指标分别提高了30.8%和5.9%,因此,本文方法在压缩HDR图像的动态范围的同时,可以更好地保留边界、纹理等细节信息。     

基于虚拟曝光图像的立体高动态范围图像合成算法

本文提出基于多视点多曝光图像的立体高动态范围图像合成算法。首 先,考虑多视点多曝光 图像以及相机响应函数曲线的特性,提出一种虚拟曝光图像绘制算法,将不同曝光的图像绘 制到同一视点;然后, 为了使绘制曝光图像保留更多细节和结构,需要对绘制虚拟曝光图像进行空洞填补及边缘修 复,故引入了边缘差值 掩膜图,对图像边缘信息进行校正平滑处理;最后利用绘制的虚拟曝光图像合成立体高动态 范围图像。实验结果表 明,获得的绘制曝光图像与参考曝光视点图像之间的结构相似性高达0.99以上,且合成的 高动态范围图像质量高。     

Rate–distortion optimization of multi-exposure image coding for high dynamic range image coding

High dynamic range (HDR) images have many practical applications because they offer an extended dynamic range and a more realistic visual experience. A HDR image is usually stored in floating-point format, so pre-processing is required to make the HDR image compatible with coding standards. A transfer function is also used to achieve better coding efficiency. Typically, HDR images are generated using several low dynamic range (LDR) images with different exposures. Instead of compressing the HDR image when it is generated from images with multiple exposures, this study proposes a technique to compress the multi-exposure images. The HDR image generation, as well as the multi-exposure images fusion, can be realized in the decoder. The proposed framework encodes the multi-exposure images using MV-HEVC where the inter-view redundancy is well exploited when an accurate intensity-mapping function between the multi-exposure images has been established. Multi-exposure image coding is used to produce a high-quality HDR image so the rate–distortion optimization (RDO) is modified by considering both the reconstruction quality of the current block and its effect on the multi-exposure fused image. A Lagrange multiplier is modified to maintain a balance between the rate and the modified distortion during the RDO process. Compared to encoding the generated HDR image using HEVC range extension, the experimental results show that the proposed technique achieves significant bitrate savings for equivalent quality in terms of HDR-VDP-2.     

Ghost detection and removal for high dynamic range images: Recent advances

High dynamic range (HDR) image generation and display technologies are becoming increasingly popular in various applications. A standard and commonly used approach to obtain an HDR image is the multiple exposures' fusion technique which consists of combining multiple images of the same scene with varying exposure times. However, if the scene is not static during the sequence acquisition, moving objects manifest themselves as ghosting artefacts in the final HDR image. Detecting and removing ghosting artefacts is an important issue for automatically generating HDR images of dynamic scenes. The aim of this paper is to provide an up-to-date review of the recently proposed methods for ghost-free HDR image generation. Moreover, a classification and comparison of the reviewed methods is reported to serve as a useful guide for future research on this topic.     

一种基于LCD-LED双调制显示器的HDR图像显示方法

提出一种针对HDR图像的显示方法,该方法应用于LCD-LED双调制高动态显示器,旨在使显示器更好地显示HDR图像要求的亮度细节,提高视觉质量。首先,建立HDR图像亮度与LED亮度范围对应关系,使用光扩散函数作为背光LED亮度取值范围的权重。然后,统计LED的亮度辐射区域内不同LED发光强度等级的权重,取权重最大者对应的亮度作为该背光LED亮度。最后,利用依据LED亮度生成LED背光扩散图像与HDR图像计算出LCD显示图像。仿真结果表明,该算法可以使显示器亮度失真率系数低至0.006以下。在LCD-LED双调制高动态显示器上实验验证结果表明,该算法可以有效提高图像的层次感,有效抑制LCD-LED双调制高动态显示器的"光晕"以及亮度截断现象,提高显示质量。     

A new automatic exposure algorithm for video cameras using luminance histogram

Automatic exposure (AE) is one of the indispensable functions of modern video cameras. According to the attention mechanism of human visual systems, peak regions in luminance histogram correspond to the region of no interest in an image. Based on this assumption, a new AE algorithm using the luminance histogram of an image is proposed in this paper. The algorithm finds the first two largest peak regions in the histogram and calculates the mean weighted luminance (MWL) of the entire image by weighting the luminance of pixels inside the two peak regions. The MWL is then used to control the exposure of video cameras. The weight of pixel luminance is decided by a set of quadratic curves, and the parameters of the quadratic curves are affected by the brightness of the image background. Fuzzy logic is also applied to optimize the practical AE systems. Results show that the proposed algorithm gives efficient exposure control over various scene tests.     

Recovering high dynamic range by Multi-Exposure Retinex

The matter of generating high dynamic range (HDR) image from a number of differently exposed pictures arises to satisfy the needs of high-quality imaging and industrial applications. A number of HDR image generation algorithms have been proposed in the past. However, the HDR radiance map recovered by these classical methods cannot completely exclude the noisy pixels in the input images and thus are unable to produce the optimal result with highest possible SNR. In this paper we are going to introduce a new HDR generation algorithm based on the Multi-Exposure Retinex model deduced in this paper for HDR image composition. The luminance component L and the reflectance R are synthesized independently before being combined together. A novel R image composition method is introduced to help the composed result image reach the highest possible SNR. The method is tested on grey-level images in this paper, but it can be easily extended to the color-image version.     

Dual information hiding algorithm based on the regularity of 3D mesh model

An adaptive thresholds algorithm is proposed in this letter, which is used to determine the global optimal thresholds for multi-bit quanta image sensor (MB-QIS). Firstly, the senor model of MB-QIS is set up. Then global optimal thresholds theory is analyzed and a thresholds optimization algorithm based on the binary search is designed to determine the optimal global thresholds. Finally, the high dynamic range (HDR) images are reconstructed by the non-iterative maximum likelihood estimation (MLE) image reconstruction method. The results of simulation prove that HDR imaging of MB-QIS is realized by the proposed method effectively.     

Blind quality assessment of tone-mapped images using multi-exposure sequences

The tone mapping operator (TMO) enables high dynamic range (HDR) images to be presented on low dynamic range (LDR) consumer electronic devices. However, the results obtained by this method are not always ideal due to the reduced number of bits. In comparison, the multi-exposure image fusion (MEF) bypasses the intermediate HDR image composition and directly produces an image presented on standard devices. Inspired by this, this paper proposes a quality assessment method for tone-mapped image (TMI) based on generating multi-exposure sequences. Specifically, the method uses a generative adversarial network (GAN) to generate a set of sequences with different exposure levels based on the TMIs. Then a two-branch convolutional neural network (CNN) is used to extract features from the tone-mapped images and the multi-exposure reference sequences, respectively. Finally, the transformer is used to mine the intrinsic connections between TMIs and multi-exposure sequences and learn the mapping relationships from feature space to quality space. We conducted extensive experiments on the ESPL-LIVE HDR database. The applicability and effectiveness of the proposed method are verified by comparing and analyzing relevant features and model configurations with existing mainstream evaluation algorithms.     

Reversible data hiding scheme for high dynamic range images based on multiple prediction error expansion

For sensitive areas that even the slight distortion in images is not tolerated, we propose a prediction error expansion-based reversible data hiding algorithm which can embed reversible watermark in high dynamic range (HDR) image with low distortion. On the one hand, considering unique floating-point storage format and perceptual characteristics of HDR image, the multiple carriers are generated with low distortion and various characteristics to improve the embedding capacity. On the other hand, multi-model predictor and multi-level embedding strength are adopted adaptively to realize multi-level reversible HDR image information hiding based on prediction error expansion. The experimental results show that the proposed algorithm has good invisibility by making full use of HDR image characteristics. Meanwhile, the multiple strategies of multiple carriers, multi-model predictor and multi-level embedding strength, which further improve the embedding capacity and security of the algorithm, can meet the needs of sensitive areas such as medical processing, judicial authentication, and so on.     

High dynamic range image reconstruction for multi-bit quanta image sensor

An adaptive thresholds algorithm is proposed in this letter, which is used to determine the global optimal thresholds for multi-bit quanta image sensor(MB-QIS). Firstly, the senor model of MB-QIS is set up. Then global optimal thresholds theory is analyzed and a thresholds optimization algorithm based on the binary search is designed to determine the optimal global thresholds. Finally, the high dynamic range(HDR) images are reconstructed by the noniterative maximum likelihood estimation(MLE) ima...     

基于图像直方图统计的CCD相机自动调光算法

在一些特殊应用场合,CCD相机成像时具有场景不可重复、应用环境多样性等特点,使得自动调光成为一个研究难点。针对该问题提出了一种基于图像直方图统计的自动调光算法。该算法首先分析一帧图像的亮度分布,并计算出下一帧图像的最佳均值。然后根据该均值调整下一帧图像的曝光量,使其合理曝光。实验表明,利用此方法可以获得整体亮度合适的图像,同时此方法具备较好的场景适应能力,满足场景时刻变化时CCD相机自动调光的需求。     

POCS超分辨率图像重构的快速算法

超分辨率图像重构是将多帧低分辨率图像重构成一幅高分辨率图像的过程。由于其求解是一大型病态求逆问题，计算量随着放大倍数的增加而急剧上升，如何降低计算复杂度是超分辨率成像所面临的一个急需解决的课题。提出了一个基于PoCs的高分辨率图像重构的快速算法。其原理是利用各低分辨率图像之间位移的关系将所有的低分辨率图像进行重组，然后对每个组进行PoCs超分辨图象重构。实验结果表明。该快速算法较大地提高了超分辨图像重构的速度。     

基于局部信息熵最大的多曝光控制方法

基于融合多幅低动态图像来获取高动态图像的过程中,传统方法中低动态图像获取时对曝光时间选取的策略简单,使拍摄的多幅图像信息冗余较多,严重影响融合效率。提出了一种基于局部信息熵最大准则的多曝光控制方法。讨论了低动态场景图像信息熵与曝光时间的关系,得出了低动态范围场景的图像信息熵随曝光时间的增加呈现先增加后减小的规律,并在某个曝光时间处信息熵最大。对于高动态场景,首先,利用图像平均灰度响应与曝光时间的近似线性关系确定场景的曝光时间范围;然后,根据图像直方图将高动态场景分成若干个低动态范围场景区域;最后,以信息熵最大为优化目标,设计一维搜索算法,搜索各个低动态范围区域的最优曝光时间,直到所有区域都搜索到最优曝光时间。此方法将场景的局部信息熵与曝光时间联系起来,能针对不同的区域进行曝光时间优化,目的性强,有效地避免了传统曝光控制中的缺点,实验证明:用该方法获取的图像进行融合获得了良好的效果。