The Generic Bilinear Calibration-Estimation Problem

We identify a very general, recurring pattern in a number of well known problems in biological and machine vision. Many problems are of a peculiar double-sided nature: One attempts to estimate certain properties of the environment using a certain type of equipment and simultaneously one attempts to calibrate the same equipment on the structure of the environment. At first sight this appears the kind of the chicken and the egg problem that might well prove to be insoluble. However, due to basic constraints that universally apply (e.g., the world is only three-dimensional), a solution—up to a certain class of ambiguity transformations—often exists. The more complicated the problem is, the less important the remaining ambiguity will be, at least in a relative sense. Many well known problems are special in that they can be cast in bilinear form, sometimes after transformation or the introduction of dummy variables. Instances include photometric stereo, photometric estimations (e.g., of lightness), local (differential) image operators, a variety of photogrammetric problems, etc. It turns out that many of these problems—and together these make up a large fraction of the generic problems in machine vision today—can be cast in a simple universal framework. This framework enables one to handle arbitrarily large (that is, not minimal, consistent configurations), noisy (thus inconsistent) date sets automatically. The level at which prior information (either of a deterministic or a statistical nature) is used (assumptions such as constant albedo, rigidity, uniform distributions, etc.) is clearly separated as an additional, typically nonlinear, stage.     

The Structure of Visual Spaces

The “visual space” of an optical observer situated at a single, fixed viewpoint is necessarily very ambiguous. Although the structure of the “visual field” (the lateral dimensions, i.e., the “image”) is well defined, the “depth” dimension has to be inferred from the image on the basis of “monocular depth cues” such as occlusion, shading, etc. Such cues are in no way “given”, but are guesses on the basis of prior knowledge about the generic structure of the world and the laws of optics. Thus such a guess is like a hallucination that is used to tentatively interpret image structures as depth cues. The guesses are successful if they lead to a coherent interpretation. Such “controlled hallucination” (in psychological terminology) is similar to the “analysis by synthesis” of computer vision. Although highly ambiguous, visual spaces do have geometrical structure. The group of ambiguities left open by the cues (e.g., the well known bas-relief ambiguity in the case of shape from shading) may be interpreted as the group of congruences (proper motions) of the space. The general structure of visual spaces for different visual fields is explored in the paper. Applications include improved viewing systems for optical man-machine interfaces.     

The Gaussian scale-space paradigm and the multiscale local jet

A representation of local image structure is proposed which takes into account both the image's spatial structure at a given location, as well as its deep structure, that is, its local behaviour as a function of scale or resolution (scale-space). This is of interest for several low-level image tasks. The proposed basis of scale-space, for example, enables a precise local study of interactions of neighbouring image intensities in the course of the blurring process. It also provides an extrapolation scheme for local image data, obtained at a given spatial location and resolution, to a finite scale-space neighbourhood. This is especially useful for the determination of sampling rates and for interpolation algorithms in a multilocal context. Another, particularly straightforward application is image enhancement or deblurring, which is an instance of data extrapolation in the high-resolution direction.A potentially interesting feature of the proposed local image parametrisation is that it captures a trade-off between spatial and scale extrapolations from a given interior point that do not exceed a given tolerance. This (rade-off suggests the possibility of a fairly coarse scale sampling at the expense of a dense spatial sampling large relative spatial overlap of scale-space kernels).The central concept developed in this paper is an equivalence class called the multiscale local jet, which is a hierarchical, local characterisation of the image in a full scale-space neighbourhood. For this local jet, a basis of fundamental polynomials is constructed that captures the scale-space paradigm at the local level up to any given order.     

The “shading twist,” a dynamical shape cue

When a light source moves, the isophotes of the illuminance field (observed as “shading” in the case of Lambertian surfaces) move over the surfaces of illuminated objects. The sense of rotation of the isophotes relative to the sense of rotation of the “surface illuminance flow,” that is the tangential component of the illumination direction over the surface, depends on the Gaussian curvature of the surface. The temporal change of orientation, the “shading twist,” reveals this purely surface related quality. Since the shading twist depends only on the local curvature of the surface it doesn’t matter at all how the light source moves or where the light sources are, the “shading twist” is a pure surface property, that is to say, it behaves as being painted upon the surface. The formal relations pertaining to the shading twist are analyzed and a numerical simulation is presented that fully corroborates the conclusions from the formal study. The algorithm is also tested on a real scene.     

Illuminance Flow Estimation by Regression

We investigate the estimation of illuminance flow using Histograms of Oriented Gradient features (HOGs). In a regression setting, we found for both ridge regression and support vector machines, that the optimal solution shows close resemblance to the gradient based structure tensor (also known as the second moment matrix).     

Detection of coherent movement in peripherally viewed random-dot patterns

We studied the detection of coherent motion in stroboscopically moving random-dot patterns for foveal vision and at eccentricities of 6, 12, 24, and 48 deg in the temporal visual field. Threshold signal-to-noise ratios (SNR's) were determined as a function of velocity for a range of stimulus sizes. It was found that the motion-detection performance is roughly invariant throughout the temporal visual field, provided that the stimuli are scaled according to the cortical magnification factor to obtain equivalent cortical sizes and velocities at all eccentricities. The maximum field velocity compatible with the percept of coherent motion increased about linearly with the width of the square stimuli. At this high-velocity threshold any pixel crossed the field in five to nine equal steps with a constant total crossing time of 50-90 msec, regardless of stimulus size or eccentricity. The lowest SNR values were reached at the optimal or tuning velocity V0. They approached the amazingly low values of 0.04-0.05 for large stimuli and at all eccentricities. Regardless of stimulus size, the parameter V0 increased about linearly with eccentricity from roughly 1 deg sec-1 at the fovea to some 8 deg sec-1 at 48 deg in the temporal visual field.     

Gestalt and phenomenal transparency

Phenomenal transparency is commonly studied by using a stimulus configuration introduced by Metelli: a bipartite patch, divided into equal left and right halves is overlaid with a smaller, concentric bipartite patch, divided along the same line. Observers are instructed to report either a transparent patch over an opaque bipartite field or a mosaic of four opaque patches. We show theoretically and empirically that these are only two of five generic perceptual categories, namely, transparent patch, transparent annulus (hole), mosaic, partial transparency, and multiple transparency (ambiguous) cases. Thus Gestalt factors complicate the interpretation "phenomenal transparency." We propose a framework that avoids this complication. There is excellent agreement between predictions and results.     

Directed Emission from Self‐Assembled Microhelices

Bottom‐up assembly can organize simple building blocks into complex architectures for light manipulation. The optical properties of self‐assembled polycrystalline barium carbonate/silica double helices are studied using fluorescent Fourier and Mueller matrix microscopy. Helices doped with fluorescein direct light emission along the long axis of the structure. Furthermore, light transmission measured normal and parallel to the long axis exhibits twist sense‐specific circular retardance and waveguiding, respectively, although the measurements suffer from depolarization. The helices thus integrate highly directional emission with enantiomorph‐specific polarization. This optical response emerges from the arrangement of nanoscopic mineral crystallites in the microscopic helix, and demonstrates how bottom‐up assembly can achieve ordering across multiple length scales to form complex functional materials.