Color reproduction from noisy CFA data of single sensor digital cameras.

Single sensor digital color still/video cameras capture images using a color filter array (CFA) and require color interpolation (demosaicking) to reconstruct full color images. The color reproduction has to combat sensor noises which are channel dependent. If untreated in demosaicking, sensor noises can cause color artifacts that are hard to remove later by a separate denoising process, because the demosaicking process complicates the noise characteristics by blending noises of different color channels. This paper presents a joint demosaicking-denoising approach to overcome this difficulty. The color image is restored from noisy mosaic data in two steps. First, the difference signals of color channels are estimated by linear minimum mean square-error estimation. This process exploits both spectral and spatial correlations to simultaneously suppress sensor noise and interpolation error. With the estimated difference signals, the full resolution green channel is recovered. The second step involves in a wavelet-based denoising process to remove the CFA channel-dependent noises from the reconstructed green channel. The red and blue channels are subsequently recovered. Simulated and real CFA mosaic data are used to evaluate the performance of the proposed joint demosaicking-denoising scheme and compare it with many recently developed sophisticated demosaicking and denoising schemes.     

Adaptive filtering for color filter array demosaicking.

Most digital still cameras acquire imagery with a color filter array (CFA), sampling only one color value for each pixel and interpolating the other two color values afterwards. The interpolation process is commonly known as demosaicking. In general, a good demosaicking method should preserve the high-frequency information of imagery as much as possible, since such information is essential for image visual quality. We discuss in this paper two key observations for preserving high-frequency information in CFA demosaicking: (1) the high frequencies are similar across three color components, and (2) the high frequencies along the horizontal and vertical axes are essential for image quality. Our frequency analysis of CFA samples indicates that filtering a CFA image can better preserve high frequencies than filtering each color component separately. This motivates us to design an efficient filter for estimating the luminance at green pixels of the CFA image and devise an adaptive filtering approach to estimating the luminance at red and blue pixels. Experimental results on simulated CFA images, as well as raw CFA data, verify that the proposed method outperforms the existing state-of-the-art methods both visually and in terms of peak signal-to-noise ratio, at a notably lower computational cost.     

基于色差空间的Bayer 图像的迭代插值算法

单CCD/CMOS 传感器相机捕捉图像信息靠在传感器表面覆盖一层颜色滤波阵列(CFA),经过CFA 后每个像素点只能获得物理三基色(红,绿,蓝)其中一种分量。另外缺少的两种颜色分量,需要通过周围像素的值来估算。首先利用55 模板内的像素来估计插值的方向并用最优的权重系数来插值G 分量。其次利用了基于有理函数的二维插值算子在色差空间插值R(B)处缺少的B(R)分量。再次利用色差插值G 处缺少的R 和B 分量。最后,使用方差约束条件,迭代插值过程被重复多次直到达到了最优的插值结果。通过在24 幅柯达图片以及笔者相机拍摄的图片上的Matlab 仿真实验,结果显示,被提出的算法无论是在视觉方面还是在量化的数据方面都表现出了优势。     

Novel color demosaicking for noisy color filter array data

Single sensor digital color still/video cameras use color demosaicking to reproduce full color images from color filter array (CFA) data. The quality of interpolated image will be degraded due to the sensor noise introduced during the image capture process. Many conventional demosaicking-denoising solutions adopt the channel-dependent noise model, which may fit the CMOS/CCD image sensor less than signal-dependent noise model. In this paper, the wavelet sub-band decomposition and synthesis are applied to interpolate the CFA data with signal-dependent noise model. The major contributions of this work include: (1) The combination of LMMSE and statistical calculation in wavelet domain are utilized to suppress the signal-dependent noise, which is separated into additive noise and multiplicative noise. (2) In CFA data, it has been verified that the quantitative relationship between the current pixel and the adjacent pixel, which locate in the same edge. Both simulated and real CFA images are employed to compare the proposed algorithm with the state-of-the-art techniques reported in the literature. The experimental results confirm that our method outperforms them both on demosaicking performance and on computational cost, when they process the noisy color filter array data.     

PCA-Based Spatially Adaptive Denoising of CFA Images for Single-Sensor Digital Cameras

Single-sensor digital color cameras use a process called color demosaicking to produce full color images from the data captured by a color filter array (CFA). The quality of demosaicked images is degraded due to the sensor noise introduced during the image acquisition process. The conventional solution to combating CFA sensor noise is demosaicking first, followed by a separate denoising processing. This strategy will generate many noise-caused color artifacts in the demosaicking process, which are hard to remove in the denoising process. Few denoising schemes that work directly on the CFA images have been presented because of the difficulties arisen from the red, green and blue interlaced mosaic pattern, yet a well designed “denoising first and demosaicking later” scheme can have advantages such as less noise-caused color artifacts and cost-effective implementation. This paper presents a principle component analysis (PCA) based spatially-adaptive denoising algorithm, which works directly on the CFA data using a supporting window to analyze the local image statistics. By exploiting the spatial and spectral correlations existed in the CFA image, the proposed method can effectively suppress noise while preserving color edges and details. Experiments using both simulated and real CFA images indicate that the proposed scheme outperforms many existing approaches, including those sophisticated demosaicking and denoising schemes, in terms of both objective measurement and visual evaluation.      

Primary-consistent soft-decision color demosaicking for digital cameras (patent pending)

Color mosaic sampling schemes are widely used in digital cameras. Given the resolution of CCD sensor arrays, the image quality of digital cameras using mosaic sampling largely depends on the performance of the color demosaicking process. A common problem with existing color demosaicking algorithms is an inconsistency of sample interpolations in different primary color channels, which is the cause of the most objectionable color artifacts. To cure the problem, we propose a new primary-consistent soft-decision framework (PCSD) of color demosaicking. In the PCSD framework, we make multiple estimates of a missing color sample under different hypotheses on edge or texture directions. The estimates are made via a primary consistent interpolation, meaning that all three primary components of a color are interpolated in the same direction. The final estimate of a color sample is obtained by testing different interpolation hypotheses in the reconstructed full-resolution color image and selecting the best via an optimal statistical decision or inference process. A concrete color demosaicking method of the PCSD framework is presented. This new method eliminates certain types of color artifacts of existing color demosaicking methods. Extensive experimental results demonstrate that the PCSD approach can significantly improve the image quality of digital cameras in both subjective and objective measures. In some instances, our gain over the competing methods can be as much as 7 dB.     

引入双边滤波器优化的彩色滤波阵列插值

基于彩色滤波阵列(CFA)的图像传感器在每个像素 位置获得三原色红(R)、绿(G)和蓝(B)中的一种分量 ,其缺失 分量需要根据周围像素插值得到。目前提出的许多种插值算法,绝大部分采用Bayer排列模 式。本文在色 差恒定假设基础上,提出一种基于双边滤波器的自适应Bayer模型插值算法,对G通道的估计采 用自适应滤波器进行插值,对R和B通道的插值采用双边滤波器。算法利用待插值像素 与不同距离像 素相关性不同的思想,根据图像边缘自适应设定滤波模板,能较准确估计G、B和R、G通 道之间的色差值。 实验结果表明,对比多尺度色差梯度算法和边缘强度滤波等算法,插值后的图像不仅 主观视觉, 且客观评价指标(彩色峰值信噪比,CPSNR)均优于这些算法。     

Context-aware joint dictionary learning for color image demosaicking

Most digital cameras are overlaid with color filter arrays (CFA) on their electronic sensors, and thus only one particular color value would be captured at every pixel location. When producing the output image, one needs to recover the full color image from such incomplete color samples, and this process is known as demosaicking. In this paper, we propose a novel context-constrained demosaicking algorithm via sparse-representation based joint dictionary learning. Given a single mosaicked image with incomplete color samples, we perform color and texture constrained image segmentation and learn a dictionary with different context categories. A joint sparse representation is employed on different image components for predicting the missing color information in the resulting high-resolution image. During the dictionary learning and sparse coding processes, we advocate a locality constraint in our algorithm, which allows us to locate most relevant image data and thus achieve improved demosaicking performance. Experimental results show that the proposed method outperforms several existing or state-of-the-art techniques in terms of both subjective and objective evaluations.     

Improvement of color video demosaicking in temporal domain.

Color demosaicking is critical to the image quality of digital still and video cameras that use a single-sensor array. Limited by the mosaic sampling pattern of the color filter array (CFA), color artifacts may occur in a demosaicked image in areas of high-frequency and/or sharp color transition structures. However, a color digital video camera captures a sequence of mosaic images and the temporal dimension of the color signals provides a rich source of information about the scene via camera and object motions. This paper proposes an inter-frame demosaicking approach to take advantage of all three forms of pixel correlations: spatial, spectral, and temporal. By motion estimation and statistical data fusion between adjacent mosaic frames, the new approach can remove much of the color artifacts that survive intra-frame demosaicking and also improve tone reproduction accuracy. Empirical results show that the proposed inter-frame demosaicking approach consistently outperforms its intra-frame counterparts both in peak signal-to-noise measure and subjective visual quality.     

一种适用于无线传感网图像节点单元的彩色图像压缩方法

针对无线传感器网络节点单元的特点,提出了一种适合这种节点单元的高效、低复杂度彩色有损压缩方法,对从图像传感器采集的原始BAYER彩色阵列图像数据进行低通滤波与降采样后,再把数据从RGB空间转换到YCbCr空间,最后利用JPEG算法对该数据进行压缩.试验结果表明该压缩方法与常规先插值后压缩方法以及类似先压缩后插值的压缩方法相比,在低码率条件下有更高的压缩性能和更低的实现复杂度.     

Robust Color Demosaicking With Adaptation to Varying Spectral Correlations

Almost all existing color demosaicking algorithms for digital cameras are designed on the assumption of high correlation between red, green, blue (or some other primary color) bands. They exploit spectral correlations between the primary color bands to interpolate the missing color samples, but in areas of no or weak spectral correlations, these algorithms are prone to large interpolation errors. Such demosaicking errors are visually objectionable because they tend to correlate with object boundaries and edges. This paper proposes a remedy to the above problem that has long been overlooked in the literature. The main contribution of this work is a hybrid demosaicking approach that supplements an existing color demosaicking algorithm by combining its results with those of adaptive intraband interpolation. This is formulated as an optimal data fusion problem, and two solutions are proposed: one is based on linear minimum mean-square estimation and the other based on support vector regression. Experimental results demonstrate that the new hybrid approach is more robust and eliminates the worst type of color artifacts of existing color demosaicking methods.      

Demosaicing algorithms for area- and line-scan cameras in print inspection

Most color image sensors use color filter arrays (CFA). With this sensor design the captured information at each sensor pixel position is restricted to a specific spectral portion (typically red, green and blue bands). To obtain the missing color responses at each pixel position, so-called CFA demosaicing algorithms are commonly used. We propose two new CFA demosaicing algorithms, which are well suited for industrial print inspection with respect to the requirements in accuracy and speed. As a main contribution, we introduce novel demosaicing algorithms for specific high-speed color digital time delay and integration (DTDI) CFA line-scan cameras. We compare the suggested CFA demosaicing algorithms to state-of-the art algorithms for area and line-scan camera operation modes. We show that the two new algorithms perform superior to conventional algorithms as indicated by reconstruction error.     

Demosaicing Method for Digital Cameras with White‐RGB Color Filter Array

Demosaicing, or color filter array (CFA) interpolation, estimates missing color channels of raw mosaiced images from a CFA to reproduce full‐color images. It is an essential process for single‐sensor digital cameras with CFAs. In this paper, a new demosaicing method for digital cameras with Bayer‐like W‐RGB CFAs is proposed. To preserve the edge structure when reproducing full‐color images, we propose an edge direction–adaptive method using color difference estimation between different channels, which can be applied to practical digital camera use. To evaluate the performance of the proposed method in terms of CPSNR, FSIM, and S‐CIELAB color distance measures, we perform simulations on sets of mosaiced images captured by an actual prototype digital camera with a Bayer‐like W‐RGB CFA. The simulation results show that the proposed method demosaics better than a conventional one by approximately +22.4% CPSNR, +0.9% FSIM, and +36.7% S‐CIELAB distance.     

Joint demosaicing and denoising.

The output image of a digital camera is subject to a severe degradation due to noise in the image sensor. This paper proposes a novel technique to combine demosaicing and denoising procedures systematically into a single operation by exploiting their obvious similarities. We first design a filter as if we are optimally estimating a pixel value from a noisy single-color (sensor) image. With additional constraints, we show that the same filter coefficients are appropriate for color filter array interpolation (demosaicing) given noisy sensor data. The proposed technique can combine many existing denoising algorithms with the demosaicing operation. In this paper, a total least squares denoising method is used to demonstrate the concept. The algorithm is tested on color images with pseudorandom noise and on raw sensor data from a real CMOS digital camera that we calibrated. The experimental results confirm that the proposed method suppresses noise (CMOS/CCD image sensor noise model) while effectively interpolating the missing pixel components, demonstrating a significant improvement in image quality when compared to treating demosaicing and denoising problems independently.     

An adaptive color plane interpolation method based on edge detection

In this letter, an adaptive interpolation algorithm based on edge detection is proposed. With this algorithm, all the missing green values can be reconstructed in Bayer pattern image by using edge detection interpolation method. Reconstructed images composed of green pixels are classified according to the high frequency components in image, and the threshold T needed for all kinds of green images in the edge detection is determined through experiments. The edge detection is carried out based on the one Dimensional （1D） gradient operator. If the gradient value is greater than T, this pixel is located on the edge; otherwise the pixel is in the smooth area of the image. Finally, the simple bilinear interpolation is used for the smooth area while the Laplacian interpolation with the second-order correction term is adopted to reconstruct the other red/blue values on the edge. This algorithm resolves effectively the conflicts between reconstructing high quality color image and reducing computational complexity, and thus largely enhances the processing speed for the reconstructed color image.     

Adaptive homogeneity-directed demosaicing algorithm.

A cost-effective digital camera uses a single-image sensor, applying alternating patterns of red, green, and blue color filters to each pixel location. A way to reconstruct a full three-color representation of color images by estimating the missing pixel components in each color plane is called a demosaicing algorithm. This paper presents three inherent problems often associated with demosaicing algorithms that incorporate two-dimensional (2-D) directional interpolation: misguidance color artifacts, interpolation color artifacts, and aliasing. The level of misguidance color artifacts present in two images can be compared using metric neighborhood modeling. The proposed demosaicing algorithm estimates missing pixels by interpolating in the direction with fewer color artifacts. The aliasing problem is addressed by applying filterbank techniques to 2-D directional interpolation. The interpolation artifacts are reduced using a nonlinear iterative procedure. Experimental results using digital images confirm the effectiveness of this approach.     

基于压缩感知的正六边形CFA模式彩色图像去马赛克方法

针对基于四边形排列的去马赛克(Demosaicking)的 传统方法存在拉链现象和虚假色等问题,本文尝试将更加符合人眼视觉特性的六边形采 样方式应用于彩色图像成像 系统,并从图像稀疏特性角度出发,提出基于压缩感知(Compressive sensing,CS)框架的 彩色图像去马赛克方法。本文方法 充分挖掘了彩色分量间和分量内的稀疏特性,可使复原图像的纹理细节与色彩更加逼真,有 效地避免了拉链现象和虚假色现象。实验结果验证了本文方法的有效性。     

Reversing demosaicking and compression in color filter array image processing: performance analysis and modeling.

In the conventional processing chain of single-sensor digital still cameras (DSCs), the images are captured with color filter arrays (CFAs) and the CFA samples are demosaicked into a full color image before compression. To avoid additional data redundancy created by the demosaicking process, an alternative processing chain has been proposed to move the compression process before the demosaicking. Recent empirical studies have shown that the alternative chain can outperform the conventional one in terms of image quality at low compression ratios. To provide a theoretically sound basis for such conclusion, we propose analytical models for the reconstruction errors of the two processing chains. The models developed confirm the results of existing empirical studies and provide better understanding of DSC processing chains. The modeling also allows performance predictions for more advanced compression and demosaicking methods, thus providing important cues for future development in this area.     

A low-complexity joint color demosaicking and zooming algorithm for digital camera.

This paper presents a low complexity joint color demosaicking and digital zooming algorithm for single-sensor digital cameras. The proposed algorithm directly extracts edge information from raw sensor data for interpolation in both demosaicking and zooming to preserve edge features in its output. This allows the extracted information to be exploited consistently in both stages and also efficiently, as no separate extraction process is required in different stages. The proposed algorithm can produce a zoomed full-color image as well as a zoomed Bayer color filter array image with outstanding performance as compared with conventional approaches which generally combine separate color demosaicking and digital zooming schemes.     

非局部相似性去马赛克算法

单传感器数码相机得到的色彩图像在每一个像素点处只有一种色彩值,为了得到一幅全彩色图像,需要在每一个像素位置上估计出另外两个缺失的色彩值。现有主要算法都是利用像素的相关性进行估计和插值,在那些边缘色彩跳变处和色彩高饱和度处容易估计失误,出现所谓的马赛克失真。为了克服这类马赛克现象,本文提出了一种利用图像的非局部相似性,即利用处于图像中不同位置处的像素点往往表现出很强的相关性这一特点,结合图像内容的局部平坦度自适应去马赛克的插值算法。该算法,首先根据相似度函数搜索与被插像素最相似的像素,然后利用区域水平和垂直方向的梯度组算子来计算区域的平坦度,从而根据相似程度和平坦度自适应地选择图像块进行插值。实验结果表明,相对于传统插值算法,该算法提高了图像的峰值信噪比,锐化了图像的纹理和边缘,减少了虚假色和锯齿现象,改善了图像的视觉效果。