Laplacian colormaps: a framework for structure‐preserving color transformations

Mappings between color spaces are ubiquitous in image processing problems such as gamut mapping, decolorization, and image optimization for color‐blind people. Simple color transformations often result in information loss and ambiguities, and one wishes to find an image‐specific transformation that would preserve as much as possible the structure of the original image in the target color space. In this paper, we propose Laplacian colormaps, a generic framework for structure‐preserving color transformations between images. We use the image Laplacian to capture the structural information, and show that if the color transformation between two images preserves the structure, the respective Laplacians have similar eigenvectors, or in other words, are approximately jointly diagonalizable. Employing the relation between joint diagonalizability and commutativity of matrices, we use Laplacians commutativity as a criterion of color mapping quality and minimize it w.r.t. the parameters of a color transformation to achieve optimal structure preservation. We show numerous applications of our approach, including color‐to‐gray conversion, gamut mapping, multispectral image fusion, and image optimization for color deficient viewers.     

Gamut Constrained Illuminant Estimation

This paper presents a novel solution to the illuminant estimation problem: the problem of how, given an image of a scene taken under an unknown illuminant, we can recover an estimate of that light. The work is founded on previous gamut mapping solutions to the problem which solve for a scene illuminant by determining the set of diagonal mappings which take image data captured under an unknown light to a gamut of reference colours taken under a known light. Unfortunately, a diagonal model is not always a valid model of illumination change and so previous approaches sometimes return a null solution. In addition, previous methods are difficult to implement. We address these problems by recasting the problem as one of illuminant classification: we define a priori a set of plausible lights thus ensuring that a scene illuminant estimate will always be found. A plausible light is represented by the gamut of colours observable under it and the illuminant in an image is classified by determining the plausible light whose gamut is most consistent with the image data. We show that this step (the main computational burden of the algorithm) can be performed simply and efficiently by means of a non-negative least-squares optimisation. We report results on a large set of real images which show that it provides excellent illuminant estimation, outperforming previous algorithms. First online version published in February, 2006     

Example‐based Multiple Local Color Transfer by Strokes

This paper investigates a new approach for color transfer. Rather than transferring color from one image to another globally, we propose a system with a stroke‐based user interface to provide a direct indication mechanism. We further present a multiple local color transfer method. Through our system the user can easily enhance a defect (source) photo by referring to some other good quality (target) images by simply drawing some strokes. Then, the system will perform the multiple local color transfer automatically. The system consists of two major steps. First, the user draws some strokes on the source and target images to indicate corresponding regions and also the regions he or she wants to preserve. The regions to be preserved which will be masked out based on an improved graph cuts algorithm. Second, a multiple local color transfer method is presented to transfer the color from the target image(s) to the source image through gradient‐guided pixel‐wise color transfer functions. Finally, the defect (source) image can be enhanced seamlessly by multiple local color transfer based on some good quality (target) examples through an interactive and intuitive stroke‐based user interface.     

White‐emitting OLED devices in an RGBW format with microelement white subpixels

Abstract— OLED devices with an RGBW pixel format using an unpatterned white emitter have the potential to provide very good efficiency and color gamut while enabling lower‐cost and large‐format manufacturing. However, the white subpixel often has unacceptably large color shifts with viewing angle. Furthermore, for some architectures such as top‐emitting microcavity devices, it can even be difficult to produce a white subpixel with good on‐axis color. In this paper, we describe the use of a white subpixel made up of a combination of differently tuned microelements and demonstrate how such an approach can overcome these problems. By carefully tuning the color and areas of each of the microelements in the white subpixel, we can trade off between better on‐axis color, less color change with angle, and higher efficiency. Furthermore, it was demonstrated that an RGBW top‐emitter microcavity device with a microelement white subpixel can achieve an increase in both power efficiency and color gamut relative to a conventional RGBW bottom‐emitter non‐microcavity device.     

Effect of DLP projector white channel on perceptual gamut

Abstract— The effect of white‐channel enhancement as implemented in the Texas Instrument DLP? digital projector technology is evaluated theoretically using both the CIELAB and the CIECAM02 color appearance models and experimentally through psychophysical testing using real images. Both theory and test results confirm a compression of perceptual gamut in both chroma and colorfulness as a result of the added white channel. Hence, while this technology is ideal for viewing graphics and text under ambient conference‐room conditions where lightness contrast is important, it is necessarily less than ideal for viewing images or in home‐theater environments where color is important.     

深度残差学习下的光源颜色估计

目的 颜色恒常性通常指人类在任意光源条件下正确感知物体颜色的自适应能力,是实现识别、分割、3维视觉等高层任务的重要前提。对图像进行光源颜色估计是实现颜色恒常性计算的主要途径之一,现有光源颜色估计方法往往因局部场景的歧义颜色导致估计误差较大。为此,提出一种基于深度残差学习的光源颜色估计方法。方法 将输入图像均匀分块,根据局部图像块的光源颜色估计整幅图像的全局光源颜色。算法包括光源颜色估计和图像块选择两个残差网络：光源颜色估计网络通过较深的网络层次和残差结构提高光源颜色估计的准确性;图像块选择网络按照光源颜色估计误差对图像块进行分类,根据分类结果去除图像中误差较大的图像块,进一步提高全局光源颜色估计精度。此外,对输入图像进行对数色度预处理,可以降低图像亮度对光源颜色估计的影响,提高计算效率。结果 在NUS-8和重处理的ColorChecker数据集上的实验结果表明,本文方法的估计精度和稳健性较好;此外,在相同条件下,对数色度图像比原始图像的估计误差低10% 15%,图像块选择网络能够进一步使光源颜色估计网络的误差降低约5%。结论 在两组单光源数据集上的实验表明,本文方法的总体设计合理有效,算法精度和稳健性好,可应用于需要进行色彩校正的图像处理和计算机视觉等领域。     

Thermal management of edge‐lit wide color gamut backlight for LCD using red laser diodes and cyan LEDs

The color gamut is one of the critical parameters that dictate the image quality of displays. The liquid crystal displays using white color light‐emitting diodes (LEDs) as the backlight, though having been widely employed recently, are not very satisfactory in terms of their color gamut because of the broad spectrum inherent to white LEDs. This prompted the authors to develop improved liquid crystal displays using an edge‐lit wide color gamut backlight that used red laser diodes and cyan LEDs. Generating laser beams with high color purity, the laser diodes are light sources with a significant effect on expanding the color gamut. However, laser diodes, red ones in particular, have unfavorable thermal characteristics. To cope with this shortcoming, the authors clearly defined the restrictive criteria for laying out two kinds of light source on the edge‐lit backlight and made a prototype 55‐type laser backlight for performance evaluation.     

Color and illuminant voting

A geometric-vision approach to color constancy and illuminant estimation is presented in this paper. We show a general framework, based on ideas from the generalized probabilistic Hough transform, to estimate the illuminant and reflectance of natural images. Each image pixel “votes” for possible illuminants and the estimation is based on cumulative votes. The framework is natural for the introduction of physical constraints in the color constancy problem. We show the relationship of this work to previous algorithms for color constancy and present examples     

勾画式局部颜色迁移

图像之间的颜色传输有效地利用了图像的基本统计信息.提出一种基于图像的简单统计信息和梯度域信息的局部颜色迁移算法,其中提供了简单易用的操作界面,在迁移过程中能够保持源图像的颜色细节;将其推广到灰度图上色问题,可有效地提高上色速度.实验结果表明,该箅法较好地实现了源图像和目标图像的局部区域颜色对应,生成具有较好视觉效果的真实自然的新图像.     

Sample‐Based Manifold Filtering for Interactive Global Illumination and Depth of Field

We present a fast reconstruction filtering method for images generated with Monte Carlo–based rendering techniques. Our approach specializes in reducing global illumination noise in the presence of depth‐of‐field effects at very low sampling rates and interactive frame rates. We employ edge‐aware filtering in the sample space to locally improve outgoing radiance of each sample. The improved samples are then distributed in the image plane using a fast, linear manifold‐based approach supporting very large circles of confusion. We evaluate our filter by applying it to several images containing noise caused by Monte Carlo–simulated global illumination, area light sources and depth of field. We show that our filter can efficiently denoise such images at interactive frame rates on current GPUs and with as few as 4–16 samples per pixel. Our method operates only on the colour and geometric sample information output of the initial rendering process. It does not make any assumptions on the underlying rendering technique and sampling strategy and can therefore be implemented completely as a post‐process filter.     

Multi‐primary image projector using programmable spectral light source

This paper presents a practical prototype of a multi‐primary image projector system in which light source spectra can be programmable for suiting any purpose. Our multi‐primary projection system is mainly configured with a light source component and an image projection component. The programmable light source can reproduce any spectral curve. Spatial images are then generated using a digital mirror device chip that quickly controls the intensity of the light source spectra in 2D image plane. The multi‐primary images in our projection system are reproduced by multiplexing the time‐sequential images with different primary colors. Our multi‐primary image projector realizes not only wide gamut projection but also spectral projection. To achieve this, we also show how light source spectra of four or six primary colors are designed.     

Generalized Gamut Mapping using Image Derivative Structures for Color Constancy

The gamut mapping algorithm is one of the most promising methods to achieve computational color constancy. However, so far, gamut mapping algorithms are restricted to the use of pixel values to estimate the illuminant. Therefore, in this paper, gamut mapping is extended to incorporate the statistical nature of images. It is analytically shown that the proposed gamut mapping framework is able to include any linear filter output. The main focus is on the local n-jet describing the derivative structure of an image. It is shown that derivatives have the advantage over pixel values to be invariant to disturbing effects (i.e. deviations of the diagonal model) such as saturated colors and diffuse light. Further, as the n-jet based gamut mapping has the ability to use more information than pixel values alone, the combination of these algorithms are more stable than the regular gamut mapping algorithm. Different methods of combining are proposed. Based on theoretical and experimental results conducted on large scale data sets of hyperspectral, laboratory and real-world scenes, it can be derived that (1) in case of deviations of the diagonal model, the derivative-based approach outperforms the pixel-based gamut mapping, (2) state-of-the-art algorithms are outperformed by the n-jet based gamut mapping, (3) the combination of the different n-jet based gamut mappings provide more stable solutions, and (4) the fusion strategy based on the intersection of feasible sets provides better color constancy results than the union of the feasible sets.     

Heuristic approach to selecting a color appearance model for hue‐preserved wide‐color gamut mapping

With the advent of wide‐gamut system colorimetry for ultra‐high definition television, the development of a gamut mapping algorithm for wide to standard gamuts is greatly needed. Most gamut mapping techniques have been developed to retain the perceived hues of the source as predicted by a color appearance model. However, some color appearance models erroneously predict the saturated colors, generating a serious hue discontinuity in wide‐color gamut mapping onto the Rec. 709 gamut boundary. This paper presents a heuristic approach to determining whether a color appearance model may create such artifacts when used in hue‐preserved gamut mapping algorithms.     

Contrast‐Enhanced Black and White Images

This paper investigates contrast enhancement as an approach to tone reduction, aiming to convert a photograph to black and white. Using a filter‐based approach to strengthen contrast, we avoid making a hard decision about how to assign tones to segmented regions. Our method is inspired by sticks filtering, used to enhance medical images but not previously used in non‐photorealistic rendering. We amplify contrast of pixels along the direction of greatest local difference from the mean, strengthening even weak features if they are most prominent. A final thresholding step converts the contrast‐enhanced image to black and white. Local smoothing and contrast enhancement balances abstraction and structure preservation; the main advantage of our method is its faithful depiction of image detail. Our method can create a set of effects: line drawing, hatching, and black and white, all having superior details to previous black and white methods.     

Preferred balance between luminance and color gamut in mobile displays

Abstract— To improve the image quality of a mobile display, the balance between color‐gamut size and luminance was studied in two subjective experiments. The first experiment was performed during the Asian Society for Information Display (ASID) conference in Nanjing, February 2004. Nearly 600 participants ranked the quality of images displayed for fixed combinations of color‐gamut size and display luminance on small color supertwisted nematic (CSTN) and thin‐film transistor (TFT) twistednematic (TN) displays. In the second experiment, a broader range of color‐gamut sizes and luminance levels were simulated on a cathode‐ray tube (CRT) display, and 20 participants were asked to score perceived image quality. The results of these experiments were used to model image quality as a function of color‐gamut size and display luminance for images differing in the level of chromaticity of their content. This model can be used to estimate the increase in luminance required to compensate for a reduction in color‐gamut size.     

Analysis of color volume of multi‐chromatic displays using gamut rings

There are claims that multi‐chromatic displays can achieve a wider color gamut by the use of additional highly saturated secondary color channels. However, there are other claims that these displays lose lightness and/or color saturation at brighter levels. These apparently divergent views have led to some controversy in the display industry and at standard setting organizations. This study examines the color gamut volume for a variety of simulated and measured multi‐chromatic (sometimes incorrectly referred to as “multi‐primary”) displays using combinations of white and/or secondary color channels, such as cyan, magenta, and yellow. Furthermore, a two‐dimensional gamut representation, referred to as “gamut rings,” is introduced to illustrate that the addition of nonprimary optical color channels to a trichromatic (RGB) display can result in a significant decrease in the chroma at higher lightness levels. The additional saturated color channels can increase the gamut volume only around their hues at darker levels. The results also confirm the validity of comparing the color light output and white light output for revealing the design trade‐offs between the high‐peak white and the color‐image brightness for multi‐chromatic displays.     

Color in perspective

Simple constraints on the sets of possible surface reflectance and illuminants are exploited in a new color constancy algorithm that builds upon Forsyth's (1990) theory of color constancy. Forsyth's method invokes the constraint that the surface colors under a canonical illuminant all fall within an established maximal convex gamut of possible colors. However, the method works only when restrictive conditions are imposed on the world: the illumination must be uniform, the surfaces must be planar, and there can be no specularities. To overcome these restrictions, we modify Forsyth's algorithm so that it works with the colors under a perspective projection (in a chromaticity space). The new algorithm working in perspective is simpler than Forsyth's method and more importantly the restrictions on the illuminant, surface shape and specularities can be relaxed. The algorithm is then extended to include a maximal gamut constraint on a set of illuminants that is analogous to the gamut constraint on surface colors. Tests on real images show that the algorithm provides good color constancy     

Generating Pointillism Paintings Based on Seurat's Color Composition

This paper presents a novel example‐based stippling technique that employs a simple and intuitive concept to convert a color image into a pointillism painting. Our method relies on analyzing and imitating the color distributions of Seurat's paintings to obtain a statistical color model. Then, this model can be easily combined with the modified multi‐class blue noise sampling to stylize an input image with characteristics of color composition in Seurat's paintings. The blue noise property of the output image also ensures that the color points are randomly located but remain spatially uniform. In our experiments, the multivariate goodness‐of‐fit tests were adopted to quantitatively analyze the results of the proposed and previous methods, further confirming that the color composition of our results are more similar to Seurat's painting style than that of previous approaches. Additionally, we also conducted a user study participated by artists to qualitatively evaluate the synthesized images of the proposed method.     

Consistent color and detail transfer from multiple source images for video and images

In this paper, we propose a method to jointly transfer the color and detail of multiple source images to a target video or image. Our method is based on a probabilistic segmentation scheme using Gaussian mixture model (GMM) to divide each source image as well as the target video frames or image into soft regions and determine the relevant source regions for each target region. For detail transfer, we first decompose each image as well as the target video frames or image into base and detail components. Then histogram matching is performed for detail components to transfer the detail of matching regions from source images to the target. We propose a unified framework to perform both color and detail transforms in an integrated manner. We also propose a method to maintain consistency for video targets, by enforcing consistent region segmentations for consecutive video frames using GMM-based parameter propagation and adaptive scene change detection. Experimental results demonstrate that our method automatically produces consistent color and detail transferred videos and images from a set of source images.     

Group‐Theme Recoloring for Multi‐Image Color Consistency

Modifying the colors of an image is a fundamental editing task with a wide range of methods available. Manipulating multiple images to share similar colors is more challenging, with limited tools available. Methods such as color transfer are effective in making an image share similar colors with a target image; however, color transfer is not suitable for modifying multiple images. Approaches for color consistency for photo collections give good results when the photo collection contains similar scene content, but are not applicable for general input images. To address these gaps, we propose an application framework for achieving color consistency for multi‐image input. Our framework derives a group color theme from the input images′ individual color palettes and uses this group color theme to recolor the image collection. This group‐theme recoloring provides an effective way to ensure color consistency among multiple images and naturally lends itself to the inclusion of an additional external color theme. We detail our group‐theme recoloring approach and demonstrate its effectiveness on a number of examples.