Reproducing Spectral Reflectances From Tristimulus Colours

Physically based rendering systems often support spectral rendering to simulate light transport in the real world. Material representations in such simulations need to be defined as spectral distributions. Since commonly available material data are in tristimulus colours, we ideally would like to obtain spectral distributions from tristimulus colours as an input to spectral rendering systems. Reproduction of spectral distributions given tristimulus colours, however, has been considered an ill‐posed problem since single tristimulus colour corresponds to a set of different spectra due to metamerism. We show how to resolve this problem using a data‐driven approach based on measured spectra and propose a practical algorithm that can faithfully reproduce a corresponding spectrum only from the given tristimulus colour. The key observation in colour science is that a natural measured spectrum is usually well approximated by a weighted sum of a few basis functions. We show how to reformulate conversion of tristimulus colours to spectra via principal component analysis. To improve accuracy of conversion, we propose a greedy clustering algorithm which minimizes reconstruction error. Using pre‐computation, the runtime computation is just a single matrix multiplication with an input tristimulus colour. Numerical experiments show that our method well reproduces the reference measured spectra using only the tristimulus colours as input.     

Fugue for MMX [parallel programming]

When I was an undergraduate in a hardware architecture course, one of my instructors compared programming for parallel processors with writing a symphony. At the time parallel processors were largely theoretical, and there were none around for me to play with. Now MMX adds more instruments to the orchestra. This article documents some of my early experiences with programming a simple compositing routine for MMX and the lessons I've learned from it. The program deals with pixels comprised of red, green, blue, and alpha (coverage) components, and with the assumption that the RGB components have already been multiplied by their own alpha component. I implement the most common image compositing operation, the Porter-Duff over operator     

A technique for generating natural colour images from false colour composite images

Colour is widely used in remote sensing work. In many instances, the use of colour conveys additional information both visually and scientifically. Remote sensing satellites view the earth in different spectral bands, viz. near infrared (NIR), red, green, and blue bands, in a conventional multispectral imaging system. In the absence of a blue channel, colour images can be generated using near infrared, red, and green bands in what is known as a false colour composite (FCC) and does not look natural, like the image we see with the naked eye. For a trained interpreter, this does not pose any problems. However, when the intended use is a fly‐through of a draped terrain, visual interpretation, or a display, meant for the non‐remote sensing professional, this becomes a handicap. To overcome this, there is a requirement to generate natural colour composites (NCC) from the given false colour composite, which demands the simulation of a blue band to be combined with green and red bands. This paper describes a unique method of generating a blue band to form natural colour images from a given false colour image set. We use a spectral transformation method to establish a relationship between the false colour and true colour image pairs provided by a sensor with all the four bands, which has a broader spectral coverage. A transformation function is fitted by selecting radiometric control points along the line of geometric registration to find a set of coefficients to be used for simulating a blue band. This blue band, along with the green and red bands, provides a near true colour or ‘natural colour’ on the display. In this paper, we present a set of adjustable radiometric transformation coefficients to accommodate variation in spatial and dynamic range offered by sensors to generate natural colour. These coefficients seem to work on a large number of images of different seasons, provided similar spectral bands and terrain are used. The proposed ‘natural colour generator’ can be used in changing false colour images to natural colour images with the aim of ‘what you get is what you would have seen’.     

Denoising Deep Monte Carlo Renderings

We present a novel algorithm to denoise deep Monte Carlo renderings, in which pixels contain multiple colour values, each for a different range of depths. Deep images are a more expressive representation of the scene than conventional flat images. However, since each depth bin receives only a fraction of the flat pixel's samples, denoising the bins is harder due to the less accurate mean and variance estimates. Furthermore, deep images lack a regular structure in depth—the number of depth bins and their depth ranges vary across pixels. This prevents a straightforward application of patch‐based distance metrics frequently used to improve the robustness of existing denoising filters. We address these constraints by combining a flat image‐space non‐local means filter operating on pixel colours with a deep cross‐bilateral filter operating on auxiliary features (albedo, normal, etc.). Our approach significantly reduces noise in deep images while preserving their structure. To our best knowledge, our algorithm is the first to enable efficient deep‐compositing workflows with denoised Monte Carlo renderings. We demonstrate the performance of our filter on a range of scenes highlighting the challenges and advantages of denoising deep images.     

Adaptive colour constancy algorithm using discrete wavelet transform

The colours of chromatically homogeneous object surfaces measured by a sensor vary with the illuminant colour used to illuminate the objects. In contrast, colour constancy enables humans to identify the true colours of the surfaces under varying illumination. This paper proposes an adaptive colour constancy algorithm which estimates the illuminant colour from wavelet coefficients at each scale of the decomposition by discrete wavelet transform of the input image. The angular error between the estimated illuminant colours in consecutive scales are used to determine the optimum scale for the best estimate of the true illuminant colour. The estimated illuminant colour is then used to modify the approximation subbands of the image so as to generate the illuminant-colour corrected image via inverse discrete wavelet transform. The experiments show that the colour constancy results generated by the proposed algorithm are comparable or better than those of the state-of-the-art colour constancy algorithms that use low-level image features.     

Technical Note: Pseudo natural colour aerial imagery for urban and suburban mapping

Due to their near‐infrared data channel, digital airborne four‐channel imagers provide a potentially good discrimination between vegetation and human‐made materials, which is very useful in automated mapping. Due to their red, green and blue data channels, they also provide natural colour images, which are very useful in traditional (manual) mapping. In this paper, an algorithm is described which provides an approximation to the spectral capabilities of the four‐channel imagers by using a colour‐infrared aerial photo as input. The algorithm is tailored to urban/suburban surroundings, where the quality of the generated (pseudo) natural colour images are fully acceptable for manual mapping. This brings the combined availability of near‐infrared and (pseudo) natural colours within reach for mapping projects based on traditional photogrammetry, which is valuable since traditional analytical cameras still by far outnumber the relatively new family of digital airborne four‐channel imagers.     

Comparison of multitemporal compositing methods for burnt area detection in Southeast Asian conditions

This study investigated the usability of different multitemporal compositing methods for burnt area detection purposes in the humid tropical conditions of insular Southeast Asia, characterised by persisting cloud cover, varying fire‐induced spectral changes and large amount of small burn scars. Six monthly composites of the Moderate Resolution Imaging Spectroradiometer (MODIS) surface reflectance images (MOD09) were built using six different algorithms. The performance and usability of the compositing algorithms were evaluated using three criteria: separability between burnt and unburnt areas, homogeneity and average sensor zenith angle. Maximum surface temperature method (band 31, 11.0 µm) produced the most homogeneous composites with preference to close‐to‐nadir observations. However, these composites showed unexpectedly low separability between burnt and unburnt areas, mainly due to low spatial resolution of band 31 (1000 m). Overall, taking into account the performance of the compositing methods and the large amount of small burnt areas in insular Southeast Asia, a minimum NIR (band 2, 0.86 µm) method, combined with pre‐compositing cloud and cloud shadow removal, was seen as the most suitable compositing method for burnt area detection in this region. Its strengths were 250 m spatial resolution in pixel selection and high burnt/unburnt separability combined with reasonably good performance on homogeneity and average sensor zenith angle.     

Apparent Greyscale: A Simple and Fast Conversion to Perceptually Accurate Images and Video

This paper presents a quick and simple method for converting complex images and video to perceptually accurate greyscale versions. We use a two‐step approach first to globally assign grey values and determine colour ordering, then second, to locally enhance the greyscale to reproduce the original contrast. Our global mapping is image independent and incorporates the Helmholtz‐Kohlrausch colour appearance effect for predicting differences between isoluminant colours. Our multiscale local contrast enhancement reintroduces lost discontinuities only in regions that insufficiently represent original chromatic contrast. All operations are restricted so that they preserve the overall image appearance, lightness range and differences, colour ordering, and spatial details, resulting in perceptually accurate achromatic reproductions of the colour original.     

A New QEM for Parametrization of Raster Images

We present an image processing method that converts a raster image to a simplical two‐complex which has only a small number of vertices (base mesh) plus a parametrization that maps each pixel in the original image to a combination of the barycentric coordinates of the triangle it is finally mapped into. Such a conversion of a raster image into a base mesh plus parametrization can be useful for many applications such as segmentation, image retargeting, multi‐resolution editing with arbitrary topologies, edge preserving smoothing, compression, etc. The goal of the algorithm is to produce a base mesh such that it has a small colour distortion as well as high shape fairness, and a parametrization that is globally continuous visually and numerically. Inspired by multi‐resolution adaptive parametrization of surfaces and quadric error metric, the algorithm converts pixels in the image to a dense triangle mesh and performs error‐bounded simplification jointly considering geometry and colour. The eliminated vertices are projected to an existing face. The implementation is iterative and stops when it reaches a prescribed error threshold. The algorithm is feature‐sensitive, i.e. salient feature edges in the images are preserved where possible and it takes colour into account thereby producing a better quality triangulation.     

A new method of data transformation for satellite images: I. Methodology and transformation equations for TM images

These two papers deal with a new method of data transformation. By analysing grey level curves (broken lines) of various ground features in image bands of different satellites, we have found that, inherent in 3‐ or 4‐band satellite images (SPOT, IKONOS, Quick Bird, OrbView, FORMOSAT and MSS) there are three basic remote‐sensing characteristics as follows: (1) the general radiance level L; (2) the visible–infrared radiation balance B; and (3) the band radiance variation vector (direction and speed) V. However, inherent in 5‐ or 7‐band satellite images (NOAA, TM), besides the above three, there is an extra basic remote‐sensing characteristic, i.e. the thermal radiation intensity I. This is denoted by thermal bands, i.e. the TM band 6 or NOAA (AVHRR) bands 4 and 5, which are relatively independent and can be used directly, and hence are unnecessary for information extraction or data transformation. Therefore, the data transformation only lies in extracting the L, B and V from original satellite images. Furthermore, we have also found that there are three basic ground‐cover elements on the Earth's surface: i.e. the bare land (in a broad sense), the vegetation and the water body, which, in different proportions, constitute all ground cover. Moreover, there are three basic (primitive) colours on colour image (including colour composite of satellite images): i.e. red, green and blue, which generate all colours on the colour image. Further research has revealed that the three basic remote‐sensing characteristics, the three basic ground‐cover elements and the three basic colours on the composite can conceptually constitute a three‐to‐three corresponding regular triangle scheme. Perhaps a good method of data transformation should make the scheme realistic, i.e. make the three ‘threes’ all correspond to each other. The research presented here has completed this task by regression calculations and selection of specific variables. First, the methodology and transformation equations for TM images are discussed. The transformed L, B and V images have relatively independent and equally distributed information as well as clear and definite physical, mathematical and geographical significance. They can be used effectively for generating high‐quality colour composites, on which the red, green, blue, yellow, pink, cyan and other colours of various kinds are all generated and all pure, saturated, equilibrated, meaning‐definite and close to the colours of ground features in nature. As a result, interpretations and discriminations of ground features can be easier and conducted not only by experience, but also by logic. The L, B and V images can also be used effectively for classification and digital analysis of ground features. With regard to the transformation equations for SPOT, NOAA, IKONOS, Quick Bird, OrbView, FORMOSAT and MSS images and the method application will be dealt with in the second paper.     

The metric of colour space

The space of colours is a fascinating space. It is a real vector space, but no matter what inner product you put on the space the resulting Euclidean distance does not correspond to human perception of difference between colours.In 1942 MacAdam performed the first experiments on colour matching and found the MacAdam ellipses which are often interpreted as defining the metric tensor at their centres. An important question is whether it is possible to define colour coordinates such that the Euclidean distance in these coordinates correspond to human perception.Using cubic splines to represent the colour coordinates and an optimisation approach we find new colour coordinates that make the MacAdam ellipses closer to uniform circles than the existing standards.     

A Physically Based Colour Model

We propose an intuitively simple way of representing colour which has the additional virtue that, it permits mixing and overlaying of transparent and opaque paints to an arbitrary degree. Our approach is related to the earlier alpha channel model used for compositing. It includes this as a special case but has applications in many other areas, especially animation, paint programs and graphics libraries.     

Evaluating the performance of multitemporal image compositing algorithms for burned area analysis

The main objective of this study was to compare the adequacy of various multitemporal image compositing algorithms to produce composite images suitable for burned area analysis. Satellite imagery from the NOAA Advanced Very High Resolution Radiometer (AVHRR) from three different regions (Portugal, central Africa, and South America) were used to compare six algorithms, two of which involve the sequential application of two criteria. Performance of the algorithms was assessed with the Jeffries-Matusita distance, to quantify spectral separability of the burned and unburned classes in the composite images. The ability of the algorithms to avoid the retention of cloud shadows was assessed visually with red-green-blue colour composites, and the level of radiometric speckle in the composite images was quantified with the Moran's I spatial autocorrelation statistic. The commonly used NDVI maximum value compositing procedure was found to be the least appropriate to produce composites to be used for burned area mapping, from all standpoints. The best spectral separability is provided by the minimum channel 2 (m2) compositing approach which has, however, the drawback of retaining cloud shadows. A two-criterion approach which complements m2 with maximization of brightness temperature in a subset of the data (m2M4) is considered the better method.     

Optimization‐Based Gradient Mesh Colour Transfer

In vector graphics, gradient meshes represent an image object by one or more regularly connected grids. Every grid point has attributes as the position, colour and gradients of these quantities specified. Editing the attributes of an existing gradient mesh (such as the colour gradients) is not only non‐intuitive but also time‐consuming. To facilitate user‐friendly colour editing, we develop an optimization‐based colour transfer method for gradient meshes. The key idea is built on the fact that we can approximate a colour transfer operation on gradient meshes with a linear transfer function. In this paper, we formulate the approximation as an optimization problem, which aims to minimize the colour distribution of the example image and the transferred gradient mesh. By adding proper constraints, i.e. image gradients, to the optimization problem, the details of the gradient meshes can be better preserved. With the linear transfer function, we are able to edit the colours and colour gradients of the mesh points automatically, while preserving the structure of the gradient mesh. The experimental results show that our method can generate pleasing recoloured gradient meshes.     

Vector Ordering and Morphological Operations for Colour Image Processing: Fundamentals and Applications

: The extension of concepts of greyscale morphology to colour image processing requires the use of a proper ordering of vectors (colours) and the definitions of infimum and supremum operators in an appropriate colour space. In this paper, a new approach to colour image morphology is proposed. It is based on a new ordering of vectors in the HSV colour space that is partial ordering. The proposed approach is hue preserving, and it is not a component-wise technique. Its basic characteristic is that it is compatible to the standard greyscale morphology: its fundamental and secondary operations possess the same basic properties as their greyscale counterparts, and furthermore, it is identical to greyscale morphology when it is applied to greyscale images. Examples that illustrate the application of the defined operations to colour images are provided. Moreover, the usefulness of the new method in various colour image processing applications, such as colour image edge detection, object recognition, vector top-hat filtering and skeleton extraction, is demonstrated. Received: 14 July 2000, Received in revised form: 24 April 2001, Accepted: 19 June 2001     

Physically Meaningful Rendering using Tristimulus Colours

In photorealistic image synthesis the radiative transfer equation is often not solved by simulating every wavelength of light, but instead by computing tristimulus transport, for instance using sRGB primaries as a basis. This choice is convenient, because input texture data is usually stored in RGB colour spaces. However, there are problems with this approach which are often overlooked or ignored. By comparing to spectral reference renderings, we show how rendering in tristimulus colour spaces introduces colour shifts in indirect light, violation of energy conservation, and unexpected behaviour in participating media. Furthermore, we introduce a fast method to compute spectra from almost any given XYZ input colour. It creates spectra that match the input colour precisely. Additionally, like in natural reflectance spectra, their energy is smoothly distributed over wide wavelength bands. This method is both useful to upsample RGB input data when spectral transport is used and as an intermediate step for corrected tristimulus‐based transport. Finally, we show how energy conservation can be enforced in RGB by mapping colours to valid reflectances.     

Fun with premultiplied alpha

The computer graphics universe consists of pixels. Pixels, in turn, consist of components: red, green, blue, and the coverage or opacity value alpha. For various reasons it is convenient to store and process a given rgbα quadruple with the rgb values already multiplied by α. This was first pointed out in the original Porter-Duff compositing paper (1984) and I presented some further justifications in an earlier column (“Image Compositing-Theory”, ibid., p.83-7, Sept. 1994). This premultiplication has some other interesting implications, and that's what I discuss this time     

Pipelining image compositing in heterogeneous networking environments

Because of intensive inter‐node communications, image compositing has always been a bottleneck in parallel visualization systems. In a heterogeneous networking environment, the variation of link bandwidth and latency adds more uncertainty to the system performance. In this paper, we present a pipelining image compositing algorithm in heterogeneous networking environments, which is able to rearrange the direction of data flow of a compositing pipeline under strict ordering constraint. We introduce a novel directional image compositing operator that specifies not only the color and α channels of the output but also the direction of data flow when performing compositing. Based on this new operator, we thoroughly study the properties of image compositing pipelines in heterogeneous environments. We develop an optimization algorithm that could find the optimal pipeline from an exponentially large searching space in polynomial time. We conducted a comprehensive evaluation on the ns‐3 network simulator. Experimental results demonstrate the efficiency of our method. Copyright © 2016 John Wiley & Sons, Ltd.     

Undersampled Light Field Rendering by a Plane Sweep

Images synthesized by light field rendering exhibit aliasing artifacts when the light field is undersampled; adding new light field samples improves the image quality and reduces aliasing but new samples are expensive to acquire. Light field rays are traditionally gathered directly from the source images, but new rays can also be inferred through geometry estimation. This paper describes a light field rendering approach based on this principle that estimates geometry from the set of source images using multi‐baseline stereo reconstruction to supplement the existing light field rays to meet the minimum sampling requirement. The rendering and reconstruction steps are computed over a set of planes in the scene volume, and output images are synthesized by compositing results from these planes together. The planes are each processed independently and the number of planes can be adjusted to scale the amount of computation to achieve the desired frame rate. The reconstruction fidelity (and by extension image quality) is improved by a library of matching templates to support matches along discontinuities in the image or geometry (e.g. object profiles and concavities). Given a set of silhouette images, the visual hull can be constructed and applied to further improve reconstruction by removing outlier matches. The algorithm is efficiently implemented by a set of image filter operations on commodity graphics hardware and achieves image synthesis at interactive rates.