Registration Based Non‐uniform Motion Deblurring

This paper proposes an algorithm which uses image registration to estimate a non‐uniform motion blur point spread function (PSF) caused by camera shake. Our study is based on a motion blur model which models blur effects of camera shakes using a set of planar perspective projections (i.e., homographies). This representation can fully describe motions of camera shakes in 3D which cause non‐uniform motion blurs. We transform the non‐uniform PSF estimation problem into a set of image registration problems which estimate homographies of the motion blur model one‐by‐one through the Lucas‐Kanade algorithm. We demonstrate the performance of our algorithm using both synthetic and real world examples. We also discuss the effectiveness and limitations of our algorithm for non‐uniform deblurring.     

Space-variant blur deconvolution and denoising in the dual exposure problem

In this paper we propose a space-variant blur estimation and effective denoising/deconvolution method for combining a long exposure blurry image with a short exposure noisy one. The blur in the long exposure shot is mainly caused by camera shake or object motion, and the noise in the underexposed image is introduced by the gain factor applied to the sensor when the ISO is set to an high value. Due to the space variant degradation, the image pair is divided into overlapping patches for processing. The main idea in the deconvolution algorithm is to incorporate a combination of prior image models into a spatially-varying deblurring/denoising framework which is applied to each patch. The method employs a kernel and parameter estimation method to choose between denoising or deblurring each patch. Experiments on both synthetic and real images are provided to validate the proposed approach.     

基于非均匀模糊核的RL改进算法

针对传统均匀模糊模型下的Richardson Lucy（RL）算法存在振铃效应和细节丢失等问题,提出了一种基于非均匀模糊模型下的改进RL算法。基于Yu-Wing的射影运动模糊模型,通过在迭代过程中采用基于局部极值分解的方法提取含有图像结构但无振铃的边缘图像,将模糊图像和上述提取的边缘图像作为输入图像,用添加了局部先验项的RL法对输入图像进行去模糊。实验验证了算法抑制振铃效应的有效性,同时很好地保留了图像细节,表明非均匀模糊模型在相机抖动产生的模糊图像去模糊中更为有效。     

Non-uniform image deblurring using an optical computing system

Removing non-uniform blur caused by camera shaking is troublesome because of its high computational cost. We analyze the efficiency bottlenecks of a non-uniform deblurring algorithm and propose an efficient optical computation deblurring framework that implements the time-consuming and repeatedly required modules, i.e., non-uniform convolution and perspective warping, by light transportation. Specifically, the non-uniform convolution and perspective warping are optically computed by a hybrid system that is composed of an off-the-shelf projector and a camera mounted on a programmable motion platform. Benefitting from the high speed and parallelism of optical computation, our system has the potential to accelerate existing non-uniform motion deblurring algorithms significantly. To validate the effectiveness of the proposed approach, we also develop a prototype system that is incorporated into an iterative deblurring framework to effectively address the image blur of planar scenes that is caused by 3D camera rotation around the x-, y- and z-axes. The results show that the proposed approach has a high efficiency while obtaining a promising accuracy and has a high generalizability to more complex camera motions.     

图像去模糊研究综述

图像模糊是指在图像捕捉或传输过程中,由于镜头或相机运动、光照条件等因素导致图像失去清晰度和细节,从而影响图像的质量和可用性。为了消除这种影响,图像去模糊技术应运而生。其目的在于通过构建计算机数学模型来衡量图像的模糊信息,从而自动预测去模糊后的清晰图像。图像去模糊算法的研究发展不仅为计算机视觉领域的其他任务提供了便利,同时也为生活领域提供了便捷和保障,如安全监控等。1）回顾了整个图像去模糊领域的发展历程,对盲图像去模糊和非盲图像去模糊中具有影响力的算法进行论述和分析。2）讨论了图像模糊的常见原因以及去模糊图像的质量评价方法。3）全面阐述了传统方法和基于深度学习方法的基本思想,并针对图像非盲去模糊和图像盲去模糊两方面的一些文献进行了综述。其中,基于深度学习的方法包括基于卷积神经网络、基于循环神经网络、基于生成式对抗网络和基于Transformer的方法等。4）简要介绍了图像去模糊领域的常用数据集并比较分析了一些代表性图像去模糊算法的性能。5）探讨了图像去模糊领域所面临的挑战,并对未来的研究方法进行了展望。     

单图像盲去模糊方法概述

图像去模糊长期以来一直是计算机视觉和图像处理中的研究热点.由相机抖动、物体运动或失焦引起的运动模糊或焦点模糊图像会严重影响图像的使用和后续处理.传统的盲去模糊方法利用图像运动模糊产生的不同原因,可将运动模糊分为全局运动模糊和局部运动模糊.概述了近年来图像盲去模糊的方法和研究现状.在深度学习图像去模糊方法的基础上,总结了...     

基于广义规范化稀疏模型的图像盲去模糊算法

针对基于规范化稀疏先验的图像盲去模糊方法估计精度低、计算速度慢、参数选择敏感等问题,提出一种Tikhonov正则增强的广义规范化稀疏模型,且将其作为中间清晰图像和运动模糊核的共同先验约束。随后,利用算子分裂、交替方向乘子法以及快速傅立叶变换,最小化关于中间清晰图像与运动模糊核的目标函数,导出一种快速图像盲去模糊算法。在标准测试集以及实际彩色模糊图像上的实验结果验证了提出方法的有效性和鲁棒性。此外,在同等条件下与近期文献中的盲去模糊方法进行比较,显示了本文方法在估计精度和估计效率上的双重优势。     

Multi-scale recursive codec network with authority parameters (MRCN-AP) for RFID multi-label deblurring

The dynamic non-uniform blur caused by Radio Frequency Identification (RFID) multi-label motion seriously affects the identification and location of labels. It is an ill-posed inverse problem for that the blur kernel and sharp image are unknown. The traditional method of removing the blur is very time-consuming. In this work, we propose Multi-scale Recursive Codec Network based on the Authority Parameter (MRCN-AP) to deblur RFID multi-label images in a vision-based RFID multi-label 3D measurement system. This network is composed of a stack of three encoder-decoder subnets of different scales, which restores the blurry image in an end-to-end manner, and extracts the detail edge on each scale effectively from coarse to fine. The proposed authority parameters reduce the parameters memory of redundant networks and improve the speed of the deblurring network. Also, we propose new large-scale RFID multi-label blur-sharp image pairs captured by the dual CCD camera. The proposed model is implemented on an extended dataset. We prove that our method improves the speed by at least 0.55 s, and also increases Peak Signal to Noise Ratio (PSNR) by 2.43dB. Besides, better visual effects are obtained by MRCN-AP deblurring network for RFID multi-label image, which is more conducive to subsequent positioning and optimization.

     

基于运动检测的图像去模糊算法

针对相机曝光期间,由于相机和被拍摄物体之间存在相对运动而产生的图像模糊的问题,提出一种基于运动检测的图像去模糊算法。本算法根据相机成像的数学模型,推导出模糊核的参数（长度和方向）与相对运动之间的定量关系,通过维纳滤波对图像进行滤波去模糊。通过实验,可以观察到滤波过后的图像在细节上更加清晰,表明本方法能够一定程度上地去除模糊。     

Motion-based motion deblurring

Motion blur due to camera motion can significantly degrade the quality of an image. Since the path of the camera motion can be arbitrary, deblurring of motion blurred images is a hard problem. Previous methods to deal with this problem have included blind restoration of motion blurred images, optical correction using stabilized lenses, and special cmos sensors that limit the exposure time in the presence of motion. In this paper, we exploit the fundamental trade off between spatial resolution and temporal resolution to construct a hybrid camera that can measure its own motion during image integration. The acquired motion information is used to compute a point spread function (psf) that represents the path of the camera during integration. This psf is then used to deblur the image. To verify the feasibility of hybrid imaging for motion deblurring, we have implemented a prototype hybrid camera. This prototype system was evaluated in different indoor and outdoor scenes using long exposures and complex camera motion paths. The results show that, with minimal resources, hybrid imaging outperforms previous approaches to the motion blur problem. We conclude with a brief discussion on how our ideas can be extended beyond the case of global camera motion to the case where individual objects in the scene move with different velocities.     

Feature Detection Performance Based Benchmarking of Motion Deblurring Methods: Applications to Vision for Legged Robots

Dexterous legged robots can move on variable terrain at high speeds. The locomotion of these legged platforms on such terrain causes severe oscillations of the robot body depending on the surface and locomotion speed. Camera sensors mounted on such platforms experience the same disturbances, hence resulting in motion blur. This is a particular corruption of the image and results in information loss further resulting in degradation or loss of important image features. Although motion blur is a significant problem for legged mobile robots, it is of more general interest since it is present in many other handheld/mobile camera applications. Deblurring methods exist in the literature to compensate for blur, however most proposed performance metrics focus on the visual quality of compensated images. From the perspective of computer vision algorithms, feature detection performance is an essential factor that determines vision performance. In this study, we claim that existing image quality based metrics are not suitable to assess the performance of deblurring algorithms when the output is used for computer vision in general and legged robotics in particular. For comparatively evaluating deblurring algorithms, we define a novel performance metric based on the feature detection accuracy on sharp and deblurred images. We rank these algorithms according to the new metric as well as image quality based metrics from the literature and experimentally demonstrate that existing metrics may not be good indicators of algorithm performance, hence good selection criteria for computer vision application. Additionally, noting that a suitable data set to evaluate the effects of motion blur and its compensation for legged platforms is lacking in the literature, we develop a comprehensive multi-sensor data set for that purpose. The data set consists of monocular image sequences collected in synchronization with a low cost MEMS gyroscope, an accurate fiber optic gyroscope and an externally measured ground truth motion data. We make use of this data set for an extensive benchmarking of prominent motion deblurring methods from the literature in terms of existing and the proposed feature based metric.     

自适应色彩矫正图像增强算法仿真研究

研究图像增强优化问题,由于大量的图像由于拍照抖动产生噪声,造成图像模糊问题,而传统的去除图像运动模糊的算法,具有计算复杂度过高和特定的假设条件的局限,为了改善图像视觉效果,提出了一种改进的计算量小的自适应矫正图像增强算法,利用模糊图像作为参考,对欠曝光图像进行非线性自适应色调矫正。首先利用非线性函数对不同通道颜色进行调节,然后使用自适应方法对亮度进行矫正仿真,得到最终清晰图像。仿真结果表明改算法可以有效增强图像,改善了图像的质量,具有一定的实际应用价值。     

一种基于改进的暗通道先验的运动模糊核估计方法

传统的暗通道先验已成功地运用于单一图像去模糊问题,但是,当模糊图像具有显著噪声时,暗通道先验无法对模糊核估计起到作用.因此,得益于分数阶计算能够有效地抑制信号的噪声并对信号的低频部分进行增强,将分数阶计算理论与模糊图像的暗通道先验相结合,提出一种基于改进的暗通道先验的运动模糊核估计方法.首先,结合最大后验估计算法与分数阶暗通道先验,构建出运动模糊图像的核估计模型;其次,利用半二次方分裂法解决模型的非凸问题;最后,根据粗糙-精细的策略,利用多尺度迭代框架估计出准确图像的模糊核,进而利用非盲去模糊的方法求解清晰图像.实验结果表明:在有无显著噪声的模糊图像中,所提出的算法虽然所需计算时间较长,但是能够获得较为准确的模糊核,并且能够减少图像噪声以及振铃伪影,提高清晰图像估计的质量;此外,对于不同类型的模糊图像,所提出的算法也同样适用.     

密集连接卷积网络图像去模糊

目的 非均匀盲去运动模糊是图像处理和计算机视觉中的基础课题之一。传统去模糊算法有处理模糊种类单一、耗费时间两大缺点,且一直未能有效解决。随着神经网络在图像生成领域的出色表现,本文把去运动模糊视为图像生成的一种特殊问题,提出一种基于神经网络的快速去模糊方法。方法 首先,将图像分类方向表现优异的密集连接卷积网络（dense connected convolutional network, DenseNets）应用到去模糊领域,该网络能充分利用中间层的有用信息。在损失函数方面,采用更符合去模糊目的的感知损失（perceptual loss）,保证生成图像和清晰图像在内容上的一致性。采用生成对抗网络（generative adversarial network,GAN）,使生成的图像在感官上与清晰图像更加接近。结果 通过测试生成图像相对于清晰图像的峰值信噪比 （peak signal to noise ratio,PSNR）,结构相似性 （structural similarity,SSIM）和复原时间来评价算法性能的优劣。相比DeblurGAN（blind motion deblurring using conditional adversarial networks）,本文算法在GOPRO测试集上的平均PSNR提高了0.91,复原时间缩短了0.32 s,能成功恢复出因运动模糊而丢失的细节信息。在Kohler数据集上的性能也优于当前主流算法,能够处理不同的模糊核,鲁棒性强。结论 本文算法网络结构简单,复原效果好,生成图像的速度也明显快于其他方法。同时,该算法鲁棒性强,适合处理各种因运动模糊而导致的图像退化问题。     

Combining deblurring and denoising for handheld HDR imaging in low light conditions

This paper proposes a probability formulation that unifies both single-image deblurring and multi-image denoising using variational inference. The proposed formulation is based on a theoretical analysis that compares denoising and deblurring in the same probabilistic framework, and supported by a practical approach that deal with general motion that creates HDR images in the presence of spatially varying motion. Based on this formulation, a new algorithm for deblurring a noisy and blurry image pair is presented. Besides, we provide also an approach that combines existing optical flow and image denoising techniques for High Dynamic Range imaging.     

融合运动信息的图像运动模糊去除算法

现有运动去模糊算法难以有效复原含有大尺度旋转的复合运动模糊,针对此问题提出了一种基于U-net模型的神经网络框架。该框架通过融合运动信息至网络输入,给定每一像素点不同的运动约束。经过网络的编码器与解码器结构,得到每一像素点的预测值,实现端对端的方式直接获得复原图像。实验在通用数据集上与当前先进去模糊算法进行比较,该方法相比性能最好的算法PSNR(peak signal-to-noise ratio)值提高了0.14 dB,相比实时性最好的算法运行时间减少了0.1 s;同时在含有旋转运动的测试集上进行验证,证明了该算法可获得较好的复原质量。     

A Hybrid Motion Deblurring Strategy Using Patch Based Edge Restoration and Bilateral Filter

Motion blur is a common problem in digital photography. In the dim light, a long exposure time is needed to acquire a satisfactory photograph, and if the camera shakes during exposure, a motion blur is captured. Image deblurring has become a crucial image-processing challenge, because of the increased popularity of handheld cameras. Traditional motion deblurring methods assume that the blur degradation is shift-invariant; therefore, the deblurring problem can be reduced to a deconvolution problem. Edge-specific motion deblurring sharpened the strong edges of the image and then used them to estimate the blur kernel. However, this also enhanced noise and narrow edges, which cause ambiguity and ringing artifacts. We propose a hybrid-based single image motion deblurring algorithm to solve these problems. First, we separated the blurred image into strong edge parts and smooth parts. We applied the improved patch-based sharpening method to enhance the strong edge for kernel estimation, but for the smooth part, we used the bilateral filter to remove the narrow edge and the noise for avoiding the generation of ringing artifacts. Experimental results show that the proposed method is efficient at deblurring for a variety of images and can produce images of a quality comparable to other state-of-the-art techniques.     

基于稀疏正则化和变量分裂的图像去模糊

传统的图像去模糊方法易产生振铃和边缘模糊等“伪像”效应,针对这一问题,采用非光滑的正则项约束图像在稀疏字典下表示系数的稀疏性,并引入非负约束项,提出了图像的稀疏正则化去模糊模型。进一步,基于交替方向拉格朗日乘子算法,提出了求解该模型的多变量分裂迭代快速算法,将复杂问题求解转化为三个简单子问题的迭代求解,降低了模型求解的复杂性。实验结果表明,所提出的去模糊模型及其快速算法相对较好地保持了图像的结构特征和平滑性,并降低了计算复杂性。     

Shape from Sharp and Motion-Blurred Image Pair

Motion blur due to camera shake is a common occurrence. During image capture, the apparent motion of a scene point in the image plane varies according to both camera motion and scene structure. Our objective is to infer the camera motion and the depth map of static scenes using motion blur as a cue. To this end, we use an unblurred–blurred image pair. Initially, we develop a technique to estimate the transformation spread function (TSF) which symbolizes the camera shake. This technique uses blur kernels estimated at different points across the image. Based on the estimated TSF, we recover the complete depth map of the scene within a regularization framework.     

Deblurring Shaken and Partially Saturated Images

We address the problem of deblurring images degraded by camera shake blur and saturated (over-exposed) pixels. Saturated pixels violate the common assumption that the image-formation process is linear, and often cause ringing in deblurred outputs. We provide an analysis of ringing in general, and show that in order to prevent ringing, it is insufficient to simply discard saturated pixels. We show that even when saturated pixels are removed, ringing is caused by attempting to estimate the values of latent pixels that are brighter than the sensor’s maximum output. Estimating these latent pixels is likely to cause large errors, and these errors propagate across the rest of the image in the form of ringing. We propose a new deblurring algorithm that locates these error-prone bright pixels in the latent sharp image, and by decoupling them from the remainder of the latent image, greatly reduces ringing. In addition, we propose an approximate forward model for saturated images, which allows us to estimate these error-prone pixels separately without causing artefacts. Results are shown for non-blind deblurring of real photographs containing saturated regions, demonstrating improved deblurred image quality compared to previous work.