A multicamera setup for generating stereo panoramic video

Traditional visual communication systems convey only two-dimensional (2-D) fixed field-of-view (FOV) video information. The viewer is presented with a series of flat, nonstereoscopic images, which fail to provide a realistic sense of depth. Furthermore, traditional video is restricted to only a small part of the scene, based on the director's discretion and the user is not allowed to "look around" in an environment. The objective of this work is to address both of these issues and develop new techniques for creating stereo panoramic video sequences. A stereo panoramic video sequence should be able to provide the viewer with stereo vision at any direction (complete 360-degree FOV) at video rates. In this paper, we propose a new technique for creating stereo panoramic video using a multicamera approach, thus creating a high-resolution output. We present a setup that is an extension of a previously known approach, developed for the generation of still stereo panoramas, and demonstrate that it is capable of creating high-resolution stereo panoramic video sequences. We further explore the limitations involved in a practical implementation of the setup, namely the limited number of cameras and the nonzero physical size of real cameras. The relevant tradeoffs are identified and studied.     

On the extraction of 3D models from airborne video sensors for geolocation

Geolocation of a feature in a video sequence collected from a moving platform is an activity that must be undertaken in video exploitation, especially surveillance and reconnaissance applications. Examples of sensor systems that are the focus of this work include manned and unmanned aerial vehicles. The approach described here uses positional information from three sources to compute refined coordinates in three dimensions for any feature in the video sequence. These three sources are: first, sensor-platform metadata describing the likely sensor footprint based on sensor-platform positional and attitudinal information; second, 3D information of a scene inherent in a video sequence collected from a moving platform; and third, reference imagery of the region of interest that is geolocated and georectified such as aerial photography. We describe the overall steps involved in this process and the progress made to date.     

Procedures for correcting high resolution airborne video imagery

Airborne videography is a useful way of producing very high resolution remotely-sensed data. Before video data can be used for digital analysis, individual frames must be correcled for geometric distortion and brightness variation. The most obvious geometric distortion occurs because the odd and even fields in a video frame are collected sequentially and can be displaced relative to each other by aircraft roll and forward motion. We describe a procedure for correcting this distortion based on the cross-correla lion between individual fields at different spatial lags. Brightness variation across frames occurs because of dilTerences in viewing geometry, bi-directional reflectance variation and atmospheric scattering. We describe a method for removing much of this variation by calculating scattering angle across the image with an optional adjustment for sensor plane tilt. Relations between scattering angle and scene brightness statistics may then be calculated from sequences of images collected along a given transect. These relations can be used 10 normalise brightness. Tests of normalisation procedures based on mean, standard deviation, and median brightness show that the median gives the best results. This approach also produces better results than commonly-used band ratioing procedures.     

Assessment of grassland phytomass with airborne video imagery

Airborne video imagery was evaluated for assessing phytomass production within grass plots fertilized with five rates of nitrogen. Video imagery was taken with two black-and-white video cameras—one visible light and the other visible/near-infrared light sensitive. Red (0.644–0.656-μm) and infrared (0.815–0.827-μm) narrowband filters were used with the visible and visible/near-infrared light sensitive cameras, respectively. Hand-held red and infrared radiometric reflectance and phytomass measurements were made on the day that imagery was obtained. Red and infrared digital video data were obtained from the plots using an image processor. The plots were studied on two dates: 15 April and 22 May 1985. On 15 April, three phytomass levels could be distinguished within the infrared video image. Moreover, a infrared/red ratio video composite produced on an image processor enhanced differences among nitrogen treatments to the extent that generally four levels of phytomass could be identified. Coefficients of determination (r²), obtained by regressing red, infrared, and infrared/red reflectance, and red, infrared, and infrared/red digital video data on phytomass measurements were significant statistically. Imagery acquired on 22 May, however, showed few differences among treatments. This may have been caused by plant phenological and canopy architectural differences among the grass species. Nevertheless, video imagery should be a useful tool to assess phytomass production on rangelands when grasses are actively growing.     

A simplified method for retrieval of ground level reflectance of targets from airborne video imagery

This paper describes an efficient method for retrieval of ground reflectance characteristics of targets from calibrated multispectral airborne video data for routine operational airborne missions. The method uses a simplified atmospheric scattering model in combination with a dark-object subtraction procedure to estimate the effect of the atmosphere in the path between the target and the sensor, as well as the adjacent environmental effect, on the radiation signal received by an airborne sensor. The simplicity of the atmospheric scattering model is maintained by the assumption that the air density within the targetsensor path in the lower atmosphere is sufficiently uniform for operations of the Charles Sturt University's (CSU) Multispectral Airborne Video System (MAVS). The MAVS acquires imagery in blue, green, red and near-infrared (NIR) narrow spectral bands. The MAVS is radiometrically calibrated and has a consistent radiometric response in-flight. An important feature of the new method is the coupling of the image based brightness data (DN) of a dark-object and the system radiometric calibration coefficients to determine the path reflectance and the environmental reflectance of the target. The sum of the path reflectance and the environment reflectance is known as haze reflectance. The haze reflectance indicates the amount of atmospheric haze in the airborne imagery. The simplified atmospheric model is then employed to determine the actual ground reflectance of the targets using the haze subtracted apparent (total) reflectance of the target at the altitude of the airborne sensor. The apparent reflectance of the target at the sensor altitude is obtained directly from the image based DN data and the system radiometric calibration coefficients. An interesting aspect of this simplified method is that an estimate of the environmental reflectance can be obtained as a by-product of the atmospheric haze calculation using a dark-object subtraction technique. The retrieved ground reflectance characteristics from calibrated MAVS imagery are now being used routinely for remote quantitative monitoring of agricultural and environmental targets.     

Asymmetric self-recovery oriented stereo image watermarking method for three dimensional video system

To authenticate integrity of the stereo image for three dimensional video systems, a new asymmetric self-recovery oriented stereo image watermarking method is proposed by considering inter-correlations between the left and right views of the stereo image. An asymmetric self-recovery mechanism is presented to conduct allocation of watermarking capacity for asymmetrically embedding recovery references of each view. The presented mechanism is also used to recover tampered left and right views asymmetrically to obtain high quality of the stereo image with the help of disparity when the tampered regions of the stereo image are authenticated. For security, a chaotic function is employed to generate authentication bits to detect tamper. The high-frequency energy of discrete wavelet transform is utilized to divide blocks of each view into smooth and complex types, so that the recovery reference with alterable bits is computed. Moreover, to obtain a trade-off between transparency and watermarking capacity, human visual characteristics are taken into account and then a just-noticeable difference model is exploited to classify the blocks into sensitive and insensitive types, which defines two or three least significant bits of pixels are allocated for embedding watermark. Experimental results demonstrate that the proposed method can reconstruct tamper efficiently and outperform other stereo image watermarking methods, especially for extensive tamper.     

Detection of buried ancient walls using airborne thermal video radiometry

This study examined the feasibility of using a very sensitive thermal video radiometer to derive information about subsoil objects from the air. In this study we mounted a thermal sensor onboard a helicopter and acquired digital data from an altitude of 1333 m over an archaeological site on the Golan Heights, Israel. The site, namely, Leviah Enclosure, is an Early Bronze Age settlement that is covered by a thin layer of soil. The buried structures, made from basalt, could not be observed from the ground or in aerial photos. However, in the thermal images, the buried basalt structures were significantly enhanced because they have different thermal characteristics than the ground's surface. Based on the thermal images, it was possible to generate a map to use for future excavation activity. Referring to the thermal maps, a selected area was excavated, and verification on the ground, using traditional archaeological methods revealed a positive agreement between the thermal-based map and the actual location of the buried structures. The research highlights the fact that this technology can contribute additional and useful information to the field of archaeology. Based on these results, further study is planned in order to examine the capability of the sensor under different conditions and to further excavate the entire Leviah Enclosure.     

Racking focus and tracking focus on live video streams: a stereo solution

The ability to produce dynamic Depth of Field effects in live video streams was until recently a quality unique to movie cameras. In this paper, we present a computational camera solution coupled with real-time GPU processing to produce runtime dynamic Depth of Field effects. We first construct a hybrid-resolution stereo camera with a high-res/low-res camera pair. We recover a low-res disparity map of the scene using GPU-based Belief Propagation, and subsequently upsample it via fast Cross/Joint Bilateral Upsampling. With the recovered high-resolution disparity map, we warp the high-resolution video stream to nearby viewpoints to synthesize a light field toward the scene. We exploit parallel processing and atomic operations on the GPU to resolve visibility when multiple pixels warp to the same image location. Finally, we generate racking focus and tracking focus effects from the synthesized light field rendering. All processing stages are mapped onto NVIDIA’s CUDA architecture. Our system can produce racking and tracking focus effects for the resolution of 640×480 at 15 fps.     

Photogrammetric processing of high-resolution airborne and satellite linear array stereo images for mapping applications

This article introduces a mathematical model for photogrammetric processing of linear array stereo images acquired by high-resolution satellite imaging systems such as IKONOS. The experimental result of the generation of simulated IKONOS stereo images based on photogrammetric principles, IKONOS imaging geometry and a set of georeferenced aerial images is presented. An accuracy analysis of ground points derived from the simulated IKONOS stereo images is performed. The impact of the number of GCPs (ground control points), distribution of GCPs, and image measurement errors on the ground point accuracy is investigated. It is concluded that an accuracy of ground coordinates from 2 m to 3 m is attainable with GCPs, and 5 m to 12 m without GCPs. Two data sets of HRSC (high resolution stereo camera) and MOMS (modular opto-electronic multispectral stereo-scanner)-2P are also utilized to test the model and system. The presented data processing method is a key to the generation of mapping products such as digital terrain models (DEM) and digitial shorelines from high-resolution satellite images.     

Surfaces with occlusions from layered stereo

We propose a new binocular stereo algorithm that estimates scene structure as a collection of smooth surface patches. The disparities within each patch are modeled by a continuous-valued spline, while the extent of each patch is represented via a pixelwise partitioning of the images. Disparities and extents are alternately estimated in an iterative, energy minimization framework. Experimental results demonstrate that, for scenes consisting of smooth surfaces, the proposed algorithm significantly improves upon the state of the art.     

Three-dimensional shape from color photometric stereo

Computer vision systems can be used to determine the shapes of real three-dimensional objects for purposes of object recognition and pose estimation or for CAD applications. One method that has been developed is photometric stereo. This method uses several images taken from the same viewpoint, but with different lightings, to determine the three-dimensional shape of an object. Most previous work in photometric stereo has been with gray-tone images; color images have only been used for dielectric materials. In this paper we describe a procedure for color photometric stereo, which recovers the shape of a colored object from two or more color images of the object under white illumination. This method can handle different types of materials, such as composites and metals, and can employ various reflection models such as the Lambertian, dichromatic, and Torrance-Sparrow models. For composite materials, colored metals, and dielectrics, there are two advantages of utilizing color information: at each pixel, there are more constraints on the orientation, and the result is less sensitive to noise. Consequently, the shape can be found more accurately. The method has been tested on both artificial and real images of objects of various materials, and on real images of a multi-colored object.     

Validation of a large area land cover product using purpose-acquired airborne video

Large area land cover products generated from remotely sensed data are difficult to validate in a timely and cost effective manner. As a result, pre-existing data are often used for validation. Temporal, spatial, and attribute differences between the land cover product and pre-existing validation data can result in inconclusive depictions of map accuracy. This approach may therefore misrepresent the true accuracy of the land cover product, as well as the accuracy of the validation data, which is not assumed to be without error. Hence, purpose-acquired validation data is preferred; however, logistical constraints often preclude its use — especially for large area land cover products. Airborne digital video provides a cost-effective tool for collecting purpose-acquired validation data over large areas. An operational trial was conducted, involving the collection of airborne video for the validation of a 31,000 km² sub-sample of the Canadian large area Earth Observation for Sustainable Development of Forests (EOSD) land cover map (Vancouver Island, British Columbia, Canada). In this trial, one form of agreement between the EOSD product and the airborne video data was defined as a match between the mode land cover class of a 3 by 3 pixel neighbourhood surrounding the sample pixel and the primary or secondary choice of land cover for the interpreted video. This scenario produced the highest level of overall accuracy at 77% for level 4 of classification hierarchy (13 classes). The coniferous treed class, which represented 71% of Vancouver Island, had an estimated user's accuracy of 86%. Purpose acquired video was found to be a useful and cost-effective data source for validation of the EOSD land cover product. The impact of using multiple interpreters was also tested and documented. Improvements to the sampling and response designs that emerged from this trial will benefit a full-scale accuracy assessment of the EOSD product and also provides insights for other regional and global land cover mapping programs.     

Surface descriptions from stereo and shading

Surface reconstruction from stereo and shape-from-shading have both been discussed extensively in the literature. In this paper methods that attempt to supplement stereo with analysis of shading, are reviewed, with comments on their robustness. Theoretical results on uniqueness in shape from shading are presented, and on the difficulty of estimating surface shape with local intensity operators. Preliminary experiments based on the method of Koenderinck and van Doorn, for analysing variation of shading with viewpoint, are reported. It is concluded that there is a variety of techniques for deriving qualitative shape information, which might be sufficiently robust for use in computer vision systems.     

Rendering traditional mosaics

This paper discusses the principles of traditional mosaics and describes a technique for implementing a digital mosaicing system. The goal of this work is to transform digital images into traditional mosaic-like renderings. We achieve this effect by recovering free-form feature curves from the image and laying rows of tiles along these curves. Composition rules are applied to merge these tiles into an intricate jigsaw that conforms to classical mosaic styles. Mosaic rendering offers the user flexibility over every aspect of this craft, including tile arrangement, shapes, and colors. The result is a system that makes this wonderful craft more flexible and widely accessible than previously possible. Published online: 28 January 2003     

Stereoscopic video synthesis from a monocular video

This paper presents an automatic and robust approach to synthesize stereoscopic videos from ordinary monocular videos acquired by commodity video cameras. Instead of recovering the depth map, the proposed method synthesizes the binocular parallax in stereoscopic video directly from the motion parallax in monocular video, The synthesis is formulated as an optimization problem via introducing a cost function of the stereoscopic effects, the similarity, and the smoothness constraints. The optimization selects the most suitable frames in the input video for generating the stereoscopic video frames. With the optimized selection, convincing and smooth stereoscopic video can be synthesized even by simple constant-depth warping. No user interaction is required. We demonstrate the visually plausible results obtained given the input clips acquired by ordinary handheld video camera.     

Quantum knots and mosaics

In this paper, we give a precise and workable definition of a quantum knot system, the states of which are called quantum knots. This definition can be viewed as a blueprint for the construction of an actual physical quantum system. Moreover, this definition of a quantum knot system is intended to represent the “quantum embodiment” of a closed knotted physical piece of rope. A quantum knot, as a state of this system, represents the state of such a knotted closed piece of rope, i.e., the particular spatial configuration of the knot tied in the rope. Associated with a quantum knot system is a group of unitary transformations, called the ambient group, which represents all possible ways of moving the rope around (without cutting the rope, and without letting the rope pass through itself.) Of course, unlike a classical closed piece of rope, a quantum knot can exhibit non-classical behavior, such as quantum superposition and quantum entanglement. This raises some interesting and puzzling questions about the relation between topological and quantum entanglement. The knot type of a quantum knot is simply the orbit of the quantum knot under the action of the ambient group. We investigate quantum observables which are invariants of quantum knot type. We also study the Hamiltonians associated with the generators of the ambient group, and briefly look at the quantum tunneling of overcrossings into undercrossings. A basic building block in this paper is a mosaic system which is a formal (rewriting) system of symbol strings. We conjecture that this formal system fully captures in an axiomatic way all of the properties of tame knot theory.     

Active shape from stereo for highway inspection

Abstract. This paper describes an unsupervised algorithm for estimating the 3D profile of potholes in the highway surface, using structured illumination. Structured light is used to accelerate computation and to simplify the estimation of range. A low-resolution edge map is generated so that further processing may be focused on relevant regions of interest. Edge points in each region of interest are used to initialise open, active contour models, which are propagated and refined, via a pyramid, to a higher resolution. At each resolution, internal and external constraints are applied to a snake; the internal constraint is a smoothness function and the external one is a maximum-likelihood estimate of the grey-level response at the edge of each light stripe. Results of a provisional evaluation study indicate that this automated procedure provides estimates of pothole dimension suitable for use in a first, screening, assessment of highway condition. Received: 9 October 1998 / Accepted: 22 February 2000     

3-D surface description from binocular stereo

A stereo vision system that attempts to achieve robustness with respect to scene characteristics, from textured outdoor scenes to environments composed of highly regular man-made objects is presented. It integrates area-based and feature-based primitives. The area-based processing provides a dense disparity map, and the feature-based processing provides an accurate location of discontinuities. An area-based cross correlation, an ordering constraint, and a weak surface smoothness assumption are used to produce an initial disparity map. This disparity map is only a blurred version of the true one because of the smoothing introduced by the cross correlation. The problem can be reduced by introducing edge information. The disparity map is smoothed and the unsupported points removed. This method gives an active role to edgels parallel to the epipolar lines, whereas they are discarded in most feature-based systems. Very good results have been obtained on complex scenes in different domains