PCA-Based Spatially Adaptive Denoising of CFA Images for Single-Sensor Digital Cameras

Single-sensor digital color cameras use a process called color demosaicking to produce full color images from the data captured by a color filter array (CFA). The quality of demosaicked images is degraded due to the sensor noise introduced during the image acquisition process. The conventional solution to combating CFA sensor noise is demosaicking first, followed by a separate denoising processing. This strategy will generate many noise-caused color artifacts in the demosaicking process, which are hard to remove in the denoising process. Few denoising schemes that work directly on the CFA images have been presented because of the difficulties arisen from the red, green and blue interlaced mosaic pattern, yet a well designed “denoising first and demosaicking later” scheme can have advantages such as less noise-caused color artifacts and cost-effective implementation. This paper presents a principle component analysis (PCA) based spatially-adaptive denoising algorithm, which works directly on the CFA data using a supporting window to analyze the local image statistics. By exploiting the spatial and spectral correlations existed in the CFA image, the proposed method can effectively suppress noise while preserving color edges and details. Experiments using both simulated and real CFA images indicate that the proposed scheme outperforms many existing approaches, including those sophisticated demosaicking and denoising schemes, in terms of both objective measurement and visual evaluation.      

Novel color demosaicking for noisy color filter array data

Single sensor digital color still/video cameras use color demosaicking to reproduce full color images from color filter array (CFA) data. The quality of interpolated image will be degraded due to the sensor noise introduced during the image capture process. Many conventional demosaicking-denoising solutions adopt the channel-dependent noise model, which may fit the CMOS/CCD image sensor less than signal-dependent noise model. In this paper, the wavelet sub-band decomposition and synthesis are applied to interpolate the CFA data with signal-dependent noise model. The major contributions of this work include: (1) The combination of LMMSE and statistical calculation in wavelet domain are utilized to suppress the signal-dependent noise, which is separated into additive noise and multiplicative noise. (2) In CFA data, it has been verified that the quantitative relationship between the current pixel and the adjacent pixel, which locate in the same edge. Both simulated and real CFA images are employed to compare the proposed algorithm with the state-of-the-art techniques reported in the literature. The experimental results confirm that our method outperforms them both on demosaicking performance and on computational cost, when they process the noisy color filter array data.     

Color filter array demosaicking: new method and performance measures

Single-sensor digital cameras capture imagery by covering the sensor surface with a color filter array (CFA) such that each sensor pixel only samples one of three primary color values. To render a full-color image, an interpolation process, commonly referred to as CFA demosaicking, is required to estimate the other two missing color values at each pixel. In this paper, we present two contributions to the CFA demosaicking: a new and improved CFA demosaicking method for producing high quality color images and new image measures for quantifying the performance of demosaicking methods. The proposed demosaicking method consists of two successive steps: an interpolation step that estimates missing color values by exploiting spatial and spectral correlations among neighboring pixels, and a post-processing step that suppresses noticeable demosaicking artifacts by adaptive median filtering. Moreover, in recognition of the limitations of current image measures, we propose two types of image measures to quantify the performance of different demosaicking methods; the first type evaluates the fidelity of demosaicked images by computing the peak signal-to-noise ratio and CIELAB /spl utri/E/sup *//sub ab/ for edge and smooth regions separately, and the second type accounts for one major demosaicking artifact-zipper effect. We gauge the proposed demosaicking method and image measures using several existing methods as benchmarks, and demonstrate their efficacy using a variety of test images.     

Color demosaicking via directional linear minimum mean square-error estimation.

Digital cameras sample scenes using a color filter array of mosaic pattern (e.g., the Bayer pattern). The demosaicking of the color samples is critical to the image quality. This paper presents a new color demosaicking technique of optimal directional filtering of the green-red and green-blue difference signals. Under the assumption that the primary difference signals (PDS) between the green and red/blue channels are low pass, the missing green samples are adaptively estimated in both horizontal and vertical directions by the linear minimum mean square-error estimation (LMMSE) technique. These directional estimates are then optimally fused to further improve the green estimates. Finally, guided by the demosaicked full-resolution green channel, the other two color channels are reconstructed from the LMMSE filtered and fused PDS. The experimental results show that the presented color demosaicking technique outperforms the existing methods both in PSNR measure and visual perception.     

Improvement of color video demosaicking in temporal domain.

Color demosaicking is critical to the image quality of digital still and video cameras that use a single-sensor array. Limited by the mosaic sampling pattern of the color filter array (CFA), color artifacts may occur in a demosaicked image in areas of high-frequency and/or sharp color transition structures. However, a color digital video camera captures a sequence of mosaic images and the temporal dimension of the color signals provides a rich source of information about the scene via camera and object motions. This paper proposes an inter-frame demosaicking approach to take advantage of all three forms of pixel correlations: spatial, spectral, and temporal. By motion estimation and statistical data fusion between adjacent mosaic frames, the new approach can remove much of the color artifacts that survive intra-frame demosaicking and also improve tone reproduction accuracy. Empirical results show that the proposed inter-frame demosaicking approach consistently outperforms its intra-frame counterparts both in peak signal-to-noise measure and subjective visual quality.     

引入双边滤波器优化的彩色滤波阵列插值

基于彩色滤波阵列(CFA)的图像传感器在每个像素 位置获得三原色红(R)、绿(G)和蓝(B)中的一种分量 ,其缺失 分量需要根据周围像素插值得到。目前提出的许多种插值算法,绝大部分采用Bayer排列模 式。本文在色 差恒定假设基础上,提出一种基于双边滤波器的自适应Bayer模型插值算法,对G通道的估计采 用自适应滤波器进行插值,对R和B通道的插值采用双边滤波器。算法利用待插值像素 与不同距离像 素相关性不同的思想,根据图像边缘自适应设定滤波模板,能较准确估计G、B和R、G通 道之间的色差值。 实验结果表明,对比多尺度色差梯度算法和边缘强度滤波等算法,插值后的图像不仅 主观视觉, 且客观评价指标(彩色峰值信噪比,CPSNR)均优于这些算法。     

Primary-consistent soft-decision color demosaicking for digital cameras (patent pending)

Color mosaic sampling schemes are widely used in digital cameras. Given the resolution of CCD sensor arrays, the image quality of digital cameras using mosaic sampling largely depends on the performance of the color demosaicking process. A common problem with existing color demosaicking algorithms is an inconsistency of sample interpolations in different primary color channels, which is the cause of the most objectionable color artifacts. To cure the problem, we propose a new primary-consistent soft-decision framework (PCSD) of color demosaicking. In the PCSD framework, we make multiple estimates of a missing color sample under different hypotheses on edge or texture directions. The estimates are made via a primary consistent interpolation, meaning that all three primary components of a color are interpolated in the same direction. The final estimate of a color sample is obtained by testing different interpolation hypotheses in the reconstructed full-resolution color image and selecting the best via an optimal statistical decision or inference process. A concrete color demosaicking method of the PCSD framework is presented. This new method eliminates certain types of color artifacts of existing color demosaicking methods. Extensive experimental results demonstrate that the PCSD approach can significantly improve the image quality of digital cameras in both subjective and objective measures. In some instances, our gain over the competing methods can be as much as 7 dB.     

Adaptive filtering for color filter array demosaicking.

Most digital still cameras acquire imagery with a color filter array (CFA), sampling only one color value for each pixel and interpolating the other two color values afterwards. The interpolation process is commonly known as demosaicking. In general, a good demosaicking method should preserve the high-frequency information of imagery as much as possible, since such information is essential for image visual quality. We discuss in this paper two key observations for preserving high-frequency information in CFA demosaicking: (1) the high frequencies are similar across three color components, and (2) the high frequencies along the horizontal and vertical axes are essential for image quality. Our frequency analysis of CFA samples indicates that filtering a CFA image can better preserve high frequencies than filtering each color component separately. This motivates us to design an efficient filter for estimating the luminance at green pixels of the CFA image and devise an adaptive filtering approach to estimating the luminance at red and blue pixels. Experimental results on simulated CFA images, as well as raw CFA data, verify that the proposed method outperforms the existing state-of-the-art methods both visually and in terms of peak signal-to-noise ratio, at a notably lower computational cost.     

基于非局部均值的彩色图像去噪

传统的彩色图像去噪算法通常是分层处理的,而忽略了彩色图像RGB通道之间的相关性,因此基于RGB通道联合相似度估计提出了一种新的彩色图像非局部均值去噪方法。在用非局部均值滤波对彩色图像进行去噪时,首先以目标像素为中心确定其支撑区域,然后根据多通道联合相似度估计确定权重,最后采用逐块滤波的方法对每一层进行滤波。并且针对彩色图像中含有的高斯噪声提出了一种新的噪声参数估计方法。由实验结果可以看出该算法比传统的去噪算法在PSNR和FSIM方面都有提高。因此可以看出在图像去噪过程中考虑三通道之间的相关性是必要的,同时也证明了算法的有效性。     

时空自适应的分焦平面偏振视频PCA去噪

分焦平面式（DoFP）偏振成像探测器通过集成式微偏振阵列实现偏振信息的实时获取。然而由于成像过程中存在噪声,对后续的偏振图像去马赛克超分辨、场景偏振信息解算产生了严重影响。基于主成分分析（PCA）提出一种时空自适应DoFP视频数据去噪算法,对于每个待去噪的DoFP图像块,在其局部时空邻域内选取相似的图像块,然后利用主成分分析对其去噪。该算法充分利用DoFP视频数据的时空信息构建训练样本,且块匹配过程无需采用运动估计,可直接用于DoFP视频数据去噪。进一步提出基于双边滤波的残余噪声去除算法,从而得到更好的去噪效果。通过模拟与真实数据对所提算法进行实验验证,结果证明:所提算法可有效抑制噪声,在相同测试条件下,所提算法优于现有算法。     

Demosaicing Method for Digital Cameras with White‐RGB Color Filter Array

Demosaicing, or color filter array (CFA) interpolation, estimates missing color channels of raw mosaiced images from a CFA to reproduce full‐color images. It is an essential process for single‐sensor digital cameras with CFAs. In this paper, a new demosaicing method for digital cameras with Bayer‐like W‐RGB CFAs is proposed. To preserve the edge structure when reproducing full‐color images, we propose an edge direction–adaptive method using color difference estimation between different channels, which can be applied to practical digital camera use. To evaluate the performance of the proposed method in terms of CPSNR, FSIM, and S‐CIELAB color distance measures, we perform simulations on sets of mosaiced images captured by an actual prototype digital camera with a Bayer‐like W‐RGB CFA. The simulation results show that the proposed method demosaics better than a conventional one by approximately +22.4% CPSNR, +0.9% FSIM, and +36.7% S‐CIELAB distance.     

A low-complexity joint color demosaicking and zooming algorithm for digital camera.

This paper presents a low complexity joint color demosaicking and digital zooming algorithm for single-sensor digital cameras. The proposed algorithm directly extracts edge information from raw sensor data for interpolation in both demosaicking and zooming to preserve edge features in its output. This allows the extracted information to be exploited consistently in both stages and also efficiently, as no separate extraction process is required in different stages. The proposed algorithm can produce a zoomed full-color image as well as a zoomed Bayer color filter array image with outstanding performance as compared with conventional approaches which generally combine separate color demosaicking and digital zooming schemes.     

Context-aware joint dictionary learning for color image demosaicking

Most digital cameras are overlaid with color filter arrays (CFA) on their electronic sensors, and thus only one particular color value would be captured at every pixel location. When producing the output image, one needs to recover the full color image from such incomplete color samples, and this process is known as demosaicking. In this paper, we propose a novel context-constrained demosaicking algorithm via sparse-representation based joint dictionary learning. Given a single mosaicked image with incomplete color samples, we perform color and texture constrained image segmentation and learn a dictionary with different context categories. A joint sparse representation is employed on different image components for predicting the missing color information in the resulting high-resolution image. During the dictionary learning and sparse coding processes, we advocate a locality constraint in our algorithm, which allows us to locate most relevant image data and thus achieve improved demosaicking performance. Experimental results show that the proposed method outperforms several existing or state-of-the-art techniques in terms of both subjective and objective evaluations.     

Reversing demosaicking and compression in color filter array image processing: performance analysis and modeling.

In the conventional processing chain of single-sensor digital still cameras (DSCs), the images are captured with color filter arrays (CFAs) and the CFA samples are demosaicked into a full color image before compression. To avoid additional data redundancy created by the demosaicking process, an alternative processing chain has been proposed to move the compression process before the demosaicking. Recent empirical studies have shown that the alternative chain can outperform the conventional one in terms of image quality at low compression ratios. To provide a theoretically sound basis for such conclusion, we propose analytical models for the reconstruction errors of the two processing chains. The models developed confirm the results of existing empirical studies and provide better understanding of DSC processing chains. The modeling also allows performance predictions for more advanced compression and demosaicking methods, thus providing important cues for future development in this area.     

Joint demosaicing and denoising.

The output image of a digital camera is subject to a severe degradation due to noise in the image sensor. This paper proposes a novel technique to combine demosaicing and denoising procedures systematically into a single operation by exploiting their obvious similarities. We first design a filter as if we are optimally estimating a pixel value from a noisy single-color (sensor) image. With additional constraints, we show that the same filter coefficients are appropriate for color filter array interpolation (demosaicing) given noisy sensor data. The proposed technique can combine many existing denoising algorithms with the demosaicing operation. In this paper, a total least squares denoising method is used to demonstrate the concept. The algorithm is tested on color images with pseudorandom noise and on raw sensor data from a real CMOS digital camera that we calibrated. The experimental results confirm that the proposed method suppresses noise (CMOS/CCD image sensor noise model) while effectively interpolating the missing pixel components, demonstrating a significant improvement in image quality when compared to treating demosaicing and denoising problems independently.     

基于压缩感知的正六边形CFA模式彩色图像去马赛克方法

针对基于四边形排列的去马赛克(Demosaicking)的 传统方法存在拉链现象和虚假色等问题,本文尝试将更加符合人眼视觉特性的六边形采 样方式应用于彩色图像成像 系统,并从图像稀疏特性角度出发,提出基于压缩感知(Compressive sensing,CS)框架的 彩色图像去马赛克方法。本文方法 充分挖掘了彩色分量间和分量内的稀疏特性,可使复原图像的纹理细节与色彩更加逼真,有 效地避免了拉链现象和虚假色现象。实验结果验证了本文方法的有效性。     

基于色差空间的Bayer 图像的迭代插值算法

单CCD/CMOS 传感器相机捕捉图像信息靠在传感器表面覆盖一层颜色滤波阵列(CFA),经过CFA 后每个像素点只能获得物理三基色(红,绿,蓝)其中一种分量。另外缺少的两种颜色分量,需要通过周围像素的值来估算。首先利用55 模板内的像素来估计插值的方向并用最优的权重系数来插值G 分量。其次利用了基于有理函数的二维插值算子在色差空间插值R(B)处缺少的B(R)分量。再次利用色差插值G 处缺少的R 和B 分量。最后,使用方差约束条件,迭代插值过程被重复多次直到达到了最优的插值结果。通过在24 幅柯达图片以及笔者相机拍摄的图片上的Matlab 仿真实验,结果显示,被提出的算法无论是在视觉方面还是在量化的数据方面都表现出了优势。     

基于残差编解码器的通道自适应超声图像去噪方法

超声图像去噪对提高超声图像的视觉质量和完成其他相关的计算机视觉任务都至关重要。超声图像中的特征信息与斑点噪声信号较为相似,用已有的去噪方法对超声图像去噪,容易造成超声图像纹理特征丢失,这会对临床诊断的准确性产生严重的干扰。因此,在去除斑点噪声的过程中,需尽量保留图像的边缘纹理信息才能更好地完成超声图像去噪任务。该文提出一种基于残差编解码器的通道自适应去噪模型(RED-SENet),能有效去除超声图像中的斑点噪声。在去噪模型的解码器部分引入注意力反卷积残差块,使本模型可以学习并利用全局信息,从而选择性地强调关键通道的内容特征,抑制无用特征,能提高模型去噪的性能。在2个私有数据集和2个公开数据集上对该模型进行定性评估和定量分析,与一些先进的方法相比,该模型的去噪性能有显著提升,并在噪声抑制以及结构保持方面具有良好的效果。     

Robust Color Demosaicking With Adaptation to Varying Spectral Correlations

Almost all existing color demosaicking algorithms for digital cameras are designed on the assumption of high correlation between red, green, blue (or some other primary color) bands. They exploit spectral correlations between the primary color bands to interpolate the missing color samples, but in areas of no or weak spectral correlations, these algorithms are prone to large interpolation errors. Such demosaicking errors are visually objectionable because they tend to correlate with object boundaries and edges. This paper proposes a remedy to the above problem that has long been overlooked in the literature. The main contribution of this work is a hybrid demosaicking approach that supplements an existing color demosaicking algorithm by combining its results with those of adaptive intraband interpolation. This is formulated as an optimal data fusion problem, and two solutions are proposed: one is based on linear minimum mean-square estimation and the other based on support vector regression. Experimental results demonstrate that the new hybrid approach is more robust and eliminates the worst type of color artifacts of existing color demosaicking methods.      

Binary tree-based generic demosaicking algorithm for multispectral filter arrays.

In this paper, we extend the idea of using mosaicked color filter array (CFA) in color imaging, which has been widely adopted in the digital color camera industry, to the use of multispectral filter array (MSFA) in multispectral imaging. The filter array technique can help reduce the cost, achieve exact registration, and improve the robustness of the imaging system. However, the extension from CFA to MSFA is not straightforward. First, most CFAs only deal with a few bands (3 or 4) within the narrow visual spectral region, while the design of MSFA needs to handle the arrangement of multiple bands (more than 3) across a much wider spectral range. Second, most existing CFA demosaicking algorithms assume the fixed Bayer CFA and are confined to properties only existed in the color domain. Therefore, they cannot be directly applied to multispectral demosaicking. The main challenges faced in multispectral demosaicking is how to design a generic algorithm that can handle the more diversified MSFA patterns, and how to improve performance with a coarser spatial resolution and a less degree of spectral correlation. In this paper, we present a binary tree based generic demosaicking method. Two metrics are used to evaluate the generic algorithm, including the root mean-square error (RMSE) for reconstruction performance and the classification accuracy for target discrimination performance. Experimental results show that the demosaicked images present low RMSE (less than 7) and comparable classification performance as original images. These results support that MSFA technique can be applied to multispectral imaging with unique advantages.