Predicting frequency of metamerism in natural scenes by entropy of colors

Estimating the frequency of metameric surfaces in natural scenes usually requires many comparisons of surface colors to determine which are visually indistinguishable under one light but distinguishable-by a certain criterion degree-under another. The aim here was to test the predictive power of a simpler approach to estimation based on the entropy of colors. In simulations with 50 hyperspectral images of natural scenes, the logarithm of the observed relative frequency of metamerism in each scene under two successive daylights was regressed on combinations of the estimated Shannon differential entropies of the colors of the scene under the same two daylights. The regression was strong, and it remained so when restricted to the estimated differential entropy under just the first daylight, providing that the criterion degree of metamerism was limited. When the criterion degree was made more extreme, however, the restricted regression failed. A possible explanation of the predictive power of differential entropy is briefly considered.     

Chromatic structure of natural scenes

We applied independent component analysis (ICA) to hyperspectral images in order to learn an efficient representation of color in natural scenes. In the spectra of single pixels, the algorithm found basis functions that had broadband spectra and basis functions that were similar to natural reflectance spectra. When applied to small image patches, the algorithm found some basis functions that were achromatic and others with overall chromatic variation along lines in color space, indicating color opponency. The directions of opponency were not strictly orthogonal. Comparison with principal-component analysis on the basis of statistical measures such as average mutual information, kurtosis, and entropy, shows that the ICA transformation results in much sparser coefficients and gives higher coding efficiency. Our findings suggest that nonorthogonal opponent encoding of photoreceptor signals leads to higher coding efficiency and that ICA may be used to reveal the underlying statistical properties of color information in natural scenes.     

Statistics of spatial cone-excitation ratios in natural scenes

For some sets of surfaces, the spatial ratios of cone-photoreceptor excitations produced by light reflected from pairs of surfaces are almost invariant under illuminant changes. These sets include large populations of spectral reflectances, some of which represent individual natural surfaces but not their relative abundances in nature. The aim of this study was to determine whether spatial cone-excitation ratios are preserved under illuminant changes within the natural visual environment. A fast hyperspectral imaging system was used to obtain populations of 640,000 reflectance spectra from each of 30 natural scenes. The statistics of spatial cone-excitation ratios for randomly selected pairs of points in these scenes were determined for two extreme daylights. Almost-invariant ratios were common, suggesting that they represent a reliable property of the natural visual environment and a suitable foundation for visual color constancy.     

Characteristics of color memory for natural scenes

To study the characteristics of color memory for natural images, a memory-identification task was performed with differing color contrasts; three of the contrasts were defined by chromatic and luminance components of the image, and the others were defined with respect to the categorical colors. After observing a series of pictures successively, subjects identified the pictures using a confidence rating. Detection of increased contrasts tended to be harder than detection of decreased contrasts, suggesting that the chromaticness of pictures is enhanced in memory. Detecting changes within each color category was more difficult than across the categories. A multiple mechanism that processes color differences and categorical colors is briefly considered.     

Structural sparseness and spatial phase alignment in natural scenes

The Fourier phase spectrum plays a central role regarding where in an image contours occur, thereby defining the spatial relationship between those structures in the overall scene. Only a handful of studies have demonstrated psychophysically the relevance of the Fourier phase spectrum with respect to human visual processing, and none have demonstrated the relative amount of local cross-scale spatial phase alignment needed to perceptually extract meaningful structure from an image. We investigated the relative amount of spatial phase alignment needed for humans to perceptually match natural scene image structures at three different spatial frequencies [3, 6, and 12 cycles per degree (cpd)] as a function of the number of structures within the image (i.e., "structural sparseness"). The results showed that (1) the amount of spatial phase alignment needed to match structures depends on structural sparseness, with a bias for matching structures at 6 cpd and (2) the ability to match partially phase-randomized images at a given spatial frequency is independent of structural sparseness at other spatial frequencies. The findings of the current study are discussed in terms of a network of feature integrators in the human visual system.     

Visual search in natural scenes explained by local color properties

Success in visually searching for a small object or target in a natural scene depends on many factors, including the spatial structure of the scene and the pattern of observers' eye movements. The aim of this study was to determine to what extent local color properties of natural scenes can account for target-detection performance. A computer-controlled high-resolution color monitor was used to present images of natural scenes containing a small, randomly located, shaded gray sphere, which served as the target. Observers' gaze position was simultaneously monitored with an infrared video eye-tracker. About 60% of the adjusted variance in observers' detection performance was accounted for by local color properties, namely, lightness and the red-green and blue-yellow components of chroma. A similar level of variance was accounted for by observers' fixations. These results suggest that local color can be as influential as gaze position in determining observers' search performance in natural scenes.     

Color signals in natural scenes: characteristics of reflectance spectra and effects of natural illuminants

Multispectral images of natural scenes were collected from both forests and coral reefs to represent typical, complex scenes that might be viewed by modern animals. Both reflectance spectra and modeled visual color signals in these scenes were decorrelated spectrally by principal-component analysis. Nearly 98% of the variance of reflectance spectra and color signals can be described by the first three principal components for both forest and coral reef scenes, which implies that three well-designed visual channels can recover almost all of the spectral information of natural scenes. A variety of natural illuminants affects color signals of forest scenes only slightly, but the variation in ambient irradiance spectra that is due to the absorption of light by water has dramatic influences on the spectral characteristics of coral reef scenes.     

Comparison of sensitivity to color changes in natural and phase-scrambled scenes

Traditionally, thresholds for detecting photometric changes have been measured by using stimuli such as disks or gratings and accounted for in terms of relatively low-level mechanisms in the visual pathway. Therefore one might not expect the higher-order structures that characterize natural scenes to influence thresholds for detecting uniform photometric changes. We compared thresholds for detecting uniform photometric changes for natural and phase-scrambled versions of images of natural scenes. The chromaticity and luminance of every pixel was represented as a vector in a modified version of the MacLeod-Boynton color space and was translated, rotated, or compressed within that color space. Thresholds for all types of transformation were significantly lower in the raw compared with phase-scrambled scenes, and we attribute this to the influence of higher-order structure.     

Stability of the color-opponent signals under changes of illuminant in natural scenes

Illumination varies greatly both across parts of a natural scene and as a function of time, whereas the spectral reflectance function of surfaces remains more stable and is of much greater relevance when searching for specific targets. This study investigates the functional properties of postreceptoral opponent-channel responses, in particular regarding their stability against spatial and temporal variation in illumination. We studied images of natural scenes obtained in UK and Uganda with digital cameras calibrated to produce estimated L-, M-, and S-cone responses of trichromatic primates (human) and birds (starling). For both primates and birds we calculated luminance and red-green opponent (RG) responses. We also calculated a primate blue-yellow-opponent (BY) response. The BY response varies with changes in illumination, both across time and across the image, rendering this factor less invariant. The RG response is much more stable than the BY response across such changes in illumination for primates, less so for birds. These differences between species are due to the greater separation of bird L and M cones in wavelength and the narrower bandwidth of the cone action spectra. This greater separation also produces a larger chromatic signal for a given change in spectral reflectance. Thus bird vision seems to suffer a greater degree of spatiotemporal "clutter" than primate vision, but also enhances differences between targets and background. Therefore, there may be a trade-off between the degree of chromatic clutter in a visual system versus the degree of chromatic difference between a target and its background. Primate and bird visual systems have found different solutions to this trade-off.     

Statistical correlations between two-dimensional images and three-dimensional structures in natural scenes

In spite of the recent surge in the popularity of statistical approaches to vision, the joint statistics of coregistered range and light-intensity images have gone relatively unexplored. We investigate statistical correlations between images and the surface shapes that produced them. We determine which linear properties of range images can be best predicted from simple computations on intensity information, and we determine those properties of intensity images that best predict range information. We find that significant (up to p = 0.45) and potentially exploitable correlations exist between linear properties of range and intensity images, and we explore the structure of these correlations.     

Surface segmentation based on the luminance and color statistics of natural scenes

The luminance and color of surfaces in natural scenes are relatively independent under certain linear transformations, with the luminance of a surface providing little information about the color of that surface, and vice versa. However, differences in luminance between two locations in a natural scene remain strongly associated with differences in color. We used the statistics of the spatiochromatic structure of natural scenes as the priors for a Bayesian model that decides whether or not two points within an image fall on the same surface. This model provides a biologically plausible algorithm for surface segmentation that models observer segmentations well.     

Estimates of the information content and dimensionality of natural scenes from proximity distributions

Natural scenes, like most all natural data sets, show considerable redundancy. Although many forms of redundancy have been investigated (e.g., pixel distributions, power spectra, contour relationships, etc.), estimates of the true entropy of natural scenes have been largely considered intractable. We describe a technique for estimating the entropy and relative dimensionality of image patches based on a function we call the proximity distribution (a nearest-neighbor technique). The advantage of this function over simple statistics such as the power spectrum is that the proximity distribution is dependent on all forms of redundancy. We demonstrate that this function can be used to estimate the entropy (redundancy) of 3x3 patches of known entropy as well as 8x8 patches of Gaussian white noise, natural scenes, and noise with the same power spectrum as natural scenes. The techniques are based on assumptions regarding the intrinsic dimensionality of the data, and although the estimates depend on an extrapolation model for images larger than 3x3, we argue that this approach provides the best current estimates of the entropy and compressibility of natural-scene patches and that it provides insights into the efficiency of any coding strategy that aims to reduce redundancy. We show that the sample of 8x8 patches of natural scenes used in this study has less than half the entropy of 8x8 white noise and less than 60% of the entropy of noise with the same power spectrum. In addition, given a finite number of samples (<2(20)) drawn randomly from the space of 8x8 patches, the subspace of 8x8 natural-scene patches shows a dimensionality that depends on the sampling density and that for low densities is significantly lower dimensional than the space of 8x8 patches of white noise and noise with the same power spectrum.     

Image statistics of American Sign Language: comparison with faces and natural scenes

Several lines of evidence suggest that the image statistics of the environment shape visual abilities. To date, the image statistics of natural scenes and faces have been well characterized using Fourier analysis. We employed Fourier analysis to characterize images of signs in American Sign Language (ASL). These images are highly relevant to signers who rely on ASL for communication, and thus the image statistics of ASL might influence signers' visual abilities. Fourier analysis was conducted on 105 static images of signs, and these images were compared with analyses of 100 natural scene images and 100 face images. We obtained two metrics from our Fourier analysis: mean amplitude and entropy of the amplitude across the image set (which is a measure from information theory) as a function of spatial frequency and orientation. The results of our analyses revealed interesting differences in image statistics across the three different image sets, setting up the possibility that ASL experience may alter visual perception in predictable ways. In addition, for all image sets, the mean amplitude results were markedly different from the entropy results, which raises the interesting question of which aspect of an image set (mean amplitude or entropy of the amplitude) is better able to account for known visual abilities.     

Unsupervised illuminant estimation from natural scenes: an RGB digital camera suffices

A linear pseudo-inverse method for unsupervised illuminant recovery from natural scenes is presented. The algorithm, which uses a digital RGB camera, selects the naturally occurring bright areas (not necessarily the white ones) in natural images and converts the RGB digital counts directly into the spectral power distribution of the illuminants using a learning-based spectral procedure. Computations show a good spectral and colorimetric performance when only three sensors (a three-band RGB camera) are used. These results go against previous findings concerning the recovery of spectral reflectances and radiances, which claimed that the greater the number of sensors, the better the spectral performance. Combining the device with the appropriate computations can yield spectral information about objects and illuminants simultaneously, avoiding the need for spectroradiometric measurements. The method works well and needs neither a white reference located in the natural scene nor direct measurements of the spectral power distribution of the light.     

Color changes in objects in natural scenes as a function of observation distance and weather conditions

We have analyzed the changes in the color of objects in natural scenes due to atmospheric scattering according to changes in the distance of observation. Hook-shaped curves were found in the chromaticity diagram when the object moved from zero distance to long distances, where the object chromaticity coordinates approached the color coordinates of the horizon. This trend is the result of the combined effect of attenuation in the direct light arriving to the observer from the object and the airlight added during its trajectory. Atmospheric scattering leads to a fall in the object's visibility, which is measurable as a difference in color between the object and the background (taken here to be the horizon). Focusing on color difference instead of luminance difference could produce different visibility values depending on the color tolerance used. We assessed the cone-excitation ratio constancy for several objects at different distances. Affine relationships were obtained when an object's cone excitations were represented both at zero distance and increasing distances. These results could help to explain color constancy in natural scenes for objects at different distances, a phenomenon that has been pointed out by different authors.     

Psychophysical estimates of the number of spectral-reflectance basis functions needed to reproduce natural scenes

Theoretical analyses of spectral reflectances of natural surfaces suggest that their perceived colors can be well reproduced by approximations comprising combinations of three or four spectral basis functions. The aim of the present work was to assess psychophysically the number of basis functions necessary to reproduce entire natural outdoor scenes. Hyperspectral images of 20 such scenes were each subjected to a principal component analysis and then reproduced with a variable number of basis functions. The quality of the color approximation under daylight illumination was quantified theoretically in CIELAB space and psychophysically by spatial and temporal two-alternative forced-choice measurements in which the original and the approximated images were compared on a calibrated color monitor. Although five basis functions produced on average unit error in CIELAB space, original images were visually indistinguishable from their approximations only if there were at least eight basis functions. The combination of the spectral diversity of the natural world and the observed levels of color discrimination suggest that estimates of the minimum number of basis functions necessary to reproduce natural scenes may need to be revised upward.     

Spatiochromatic statistics of natural scenes: first- and second-order information and their correlational structure

Spatial filters that mimic receptive fields of visual cortex neurons provide an efficient representation of achromatic image structure, but the extension of this idea to chromatic information is at an early stage. Relatively few studies have looked at the statistical relationships between the modeled responses to natural scenes of the luminance (LUM), red-green (RG), and blue-yellow (BY) postreceptoral channels of the primate visual system. Here we consider the correlations among these channel responses in terms of pixel, first-order, and second-order information. First-order linear filtering was implemented by convolving the cosine-windowed images with oriented Gabor functions, whose gains were scaled to give equal amplitude response across spatial frequency to random fractal images. Second-order filtering was implemented via a filter-rectify-filter cascade, with Gabor functions for both first- and second-stage filters. Both signed and unsigned filter responses were obtained across a range of filter parameters (spatial frequency, 2-64 cycles/image; orientation, 0-135 degrees). The filter responses to the LUM channel images were larger than those for either RG or BY channel images. Cross correlations between the first-order channel responses and between the first- and second-order channel responses were measured. Results showed that the unsigned correlations between first-order channel responses were higher than expected on the basis of previous studies and that first-order channel responses were highly correlated with LUM, but not with RG or BY, second-order responses. These findings imply that course-scale color information correlates well with course-scale changes of fine-scale texture.     

Correcting cross-media instrument metamerism for reflectance estimation in multispectral imaging

In multispectral imaging, the color accuracy of spectral reflectance estimation degrades significantly if the medium of test samples is different from that of calibration samples. This occurs mainly for two reasons, i.e., the different characteristics of spectral reflectances and the different measurement principles between an imaging system and a spectrophotometer. In this paper, this problem is referred to as cross-media instrument metamerism. We propose to correct it by using calibration samples from a standard color chart and a limited number of tuning samples with a target medium as a priori knowledge. The reflectance transform is computed by using both calibration and tuning samples, and the metamerism transform is calculated by modeling the correlation of camera responses between neighboring imaging channels. Experimental results show that the proposed method produces satisfactory spectral and colorimetric accuracy in reflectance estimation. The method could be deployed in practical applications when the available samples of certain media are inadequate for accurate reflectance estimation in a multispectral imaging system.