立体视频传输与再现研究

在分析人的立体视觉原理基础上,设计了一种立体视频传输方案:用两个摄像头代替人的双眼采集远端场景,采用RTP传输两路视频,为了确保两路视频同步,提出了时间戳匹配算法.最后介绍了使用头盔显示器立体显示方法.     

改进的多视点立体视频FGS可分级方案

针对现有多视点立体视频FGS（Fine-Granular-Scalability）可分级方案编码效率低的问题进行改进,充分利用相邻视点之间的相似性,适应解码端质量优先或视点优先的不同需求,提出一种折衷方案,其中分I、P、B帧三种情况,以中间视点作为基本层进行编码,其他视点均预测自相邻视点部分恢复的FGS增强层,以此为基础产生当前视点的FGS增强层。实验证明,该方案具有更为灵活、广泛的可分级性能,更能适应用户对视点图像质量和立体观感的需求。     

A multicamera setup for generating stereo panoramic video

Traditional visual communication systems convey only two-dimensional (2-D) fixed field-of-view (FOV) video information. The viewer is presented with a series of flat, nonstereoscopic images, which fail to provide a realistic sense of depth. Furthermore, traditional video is restricted to only a small part of the scene, based on the director's discretion and the user is not allowed to "look around" in an environment. The objective of this work is to address both of these issues and develop new techniques for creating stereo panoramic video sequences. A stereo panoramic video sequence should be able to provide the viewer with stereo vision at any direction (complete 360-degree FOV) at video rates. In this paper, we propose a new technique for creating stereo panoramic video using a multicamera approach, thus creating a high-resolution output. We present a setup that is an extension of a previously known approach, developed for the generation of still stereo panoramas, and demonstrate that it is capable of creating high-resolution stereo panoramic video sequences. We further explore the limitations involved in a practical implementation of the setup, namely the limited number of cameras and the nonzero physical size of real cameras. The relevant tradeoffs are identified and studied.     

一种基于中间视点的多视点立体视频FGS可分级方案

基于MPEG-4标准FGS可分级技术,提出一种将FGS可分级、视点可分级相结合的多视点立体视频可分级算法。该算法中,中间视点作为基本层进行编码,各视点均预测自中间视点,并产生各相应视点的FGS增强层。编码器结构中,分为I、P、B帧三种情况进行讨论,并针对这三种情况进行改进。所提方案具有灵活、广泛的可分级性能,能适应多种不同的传输需求。实验结果验证了该方案的多视点可分级性。     

On the extraction of 3D models from airborne video sensors for geolocation

Geolocation of a feature in a video sequence collected from a moving platform is an activity that must be undertaken in video exploitation, especially surveillance and reconnaissance applications. Examples of sensor systems that are the focus of this work include manned and unmanned aerial vehicles. The approach described here uses positional information from three sources to compute refined coordinates in three dimensions for any feature in the video sequence. These three sources are: first, sensor-platform metadata describing the likely sensor footprint based on sensor-platform positional and attitudinal information; second, 3D information of a scene inherent in a video sequence collected from a moving platform; and third, reference imagery of the region of interest that is geolocated and georectified such as aerial photography. We describe the overall steps involved in this process and the progress made to date.     

Procedures for correcting high resolution airborne video imagery

Airborne videography is a useful way of producing very high resolution remotely-sensed data. Before video data can be used for digital analysis, individual frames must be correcled for geometric distortion and brightness variation. The most obvious geometric distortion occurs because the odd and even fields in a video frame are collected sequentially and can be displaced relative to each other by aircraft roll and forward motion. We describe a procedure for correcting this distortion based on the cross-correla lion between individual fields at different spatial lags. Brightness variation across frames occurs because of dilTerences in viewing geometry, bi-directional reflectance variation and atmospheric scattering. We describe a method for removing much of this variation by calculating scattering angle across the image with an optional adjustment for sensor plane tilt. Relations between scattering angle and scene brightness statistics may then be calculated from sequences of images collected along a given transect. These relations can be used 10 normalise brightness. Tests of normalisation procedures based on mean, standard deviation, and median brightness show that the median gives the best results. This approach also produces better results than commonly-used band ratioing procedures.     

Assessment of grassland phytomass with airborne video imagery

Airborne video imagery was evaluated for assessing phytomass production within grass plots fertilized with five rates of nitrogen. Video imagery was taken with two black-and-white video cameras—one visible light and the other visible/near-infrared light sensitive. Red (0.644–0.656-μm) and infrared (0.815–0.827-μm) narrowband filters were used with the visible and visible/near-infrared light sensitive cameras, respectively. Hand-held red and infrared radiometric reflectance and phytomass measurements were made on the day that imagery was obtained. Red and infrared digital video data were obtained from the plots using an image processor. The plots were studied on two dates: 15 April and 22 May 1985. On 15 April, three phytomass levels could be distinguished within the infrared video image. Moreover, a infrared/red ratio video composite produced on an image processor enhanced differences among nitrogen treatments to the extent that generally four levels of phytomass could be identified. Coefficients of determination (r²), obtained by regressing red, infrared, and infrared/red reflectance, and red, infrared, and infrared/red digital video data on phytomass measurements were significant statistically. Imagery acquired on 22 May, however, showed few differences among treatments. This may have been caused by plant phenological and canopy architectural differences among the grass species. Nevertheless, video imagery should be a useful tool to assess phytomass production on rangelands when grasses are actively growing.     

立体视频对象分割及其三维重建算法研究*

为更加有效分析立体视频对象,本文提出了一种基于离散冗余小波变换的立体视频对象分割算法,首先采用离散冗余小波变换提取特征点结合DT网格技术的视差估计方法,获得了可靠的视差场,再利用视差信息对立体视频中静止对象进行分割。对于立体视频序列中的运动对象,采用离散冗余小波提取运动区域的方法进行分割。实验结果表明,本算法对有重叠的多视频对象具有较好的分割效果,可同时分割静止物体和运动物体,具有较好的精确性和鲁棒性。对于分割出的立体视频对象,结合深度信息对其进行三维重建,得到较好的三维效果。     

A simplified method for retrieval of ground level reflectance of targets from airborne video imagery

This paper describes an efficient method for retrieval of ground reflectance characteristics of targets from calibrated multispectral airborne video data for routine operational airborne missions. The method uses a simplified atmospheric scattering model in combination with a dark-object subtraction procedure to estimate the effect of the atmosphere in the path between the target and the sensor, as well as the adjacent environmental effect, on the radiation signal received by an airborne sensor. The simplicity of the atmospheric scattering model is maintained by the assumption that the air density within the targetsensor path in the lower atmosphere is sufficiently uniform for operations of the Charles Sturt University's (CSU) Multispectral Airborne Video System (MAVS). The MAVS acquires imagery in blue, green, red and near-infrared (NIR) narrow spectral bands. The MAVS is radiometrically calibrated and has a consistent radiometric response in-flight. An important feature of the new method is the coupling of the image based brightness data (DN) of a dark-object and the system radiometric calibration coefficients to determine the path reflectance and the environmental reflectance of the target. The sum of the path reflectance and the environment reflectance is known as haze reflectance. The haze reflectance indicates the amount of atmospheric haze in the airborne imagery. The simplified atmospheric model is then employed to determine the actual ground reflectance of the targets using the haze subtracted apparent (total) reflectance of the target at the altitude of the airborne sensor. The apparent reflectance of the target at the sensor altitude is obtained directly from the image based DN data and the system radiometric calibration coefficients. An interesting aspect of this simplified method is that an estimate of the environmental reflectance can be obtained as a by-product of the atmospheric haze calculation using a dark-object subtraction technique. The retrieved ground reflectance characteristics from calibrated MAVS imagery are now being used routinely for remote quantitative monitoring of agricultural and environmental targets.     

Asymmetric self-recovery oriented stereo image watermarking method for three dimensional video system

To authenticate integrity of the stereo image for three dimensional video systems, a new asymmetric self-recovery oriented stereo image watermarking method is proposed by considering inter-correlations between the left and right views of the stereo image. An asymmetric self-recovery mechanism is presented to conduct allocation of watermarking capacity for asymmetrically embedding recovery references of each view. The presented mechanism is also used to recover tampered left and right views asymmetrically to obtain high quality of the stereo image with the help of disparity when the tampered regions of the stereo image are authenticated. For security, a chaotic function is employed to generate authentication bits to detect tamper. The high-frequency energy of discrete wavelet transform is utilized to divide blocks of each view into smooth and complex types, so that the recovery reference with alterable bits is computed. Moreover, to obtain a trade-off between transparency and watermarking capacity, human visual characteristics are taken into account and then a just-noticeable difference model is exploited to classify the blocks into sensitive and insensitive types, which defines two or three least significant bits of pixels are allocated for embedding watermark. Experimental results demonstrate that the proposed method can reconstruct tamper efficiently and outperform other stereo image watermarking methods, especially for extensive tamper.     

Detection of buried ancient walls using airborne thermal video radiometry

This study examined the feasibility of using a very sensitive thermal video radiometer to derive information about subsoil objects from the air. In this study we mounted a thermal sensor onboard a helicopter and acquired digital data from an altitude of 1333 m over an archaeological site on the Golan Heights, Israel. The site, namely, Leviah Enclosure, is an Early Bronze Age settlement that is covered by a thin layer of soil. The buried structures, made from basalt, could not be observed from the ground or in aerial photos. However, in the thermal images, the buried basalt structures were significantly enhanced because they have different thermal characteristics than the ground's surface. Based on the thermal images, it was possible to generate a map to use for future excavation activity. Referring to the thermal maps, a selected area was excavated, and verification on the ground, using traditional archaeological methods revealed a positive agreement between the thermal-based map and the actual location of the buried structures. The research highlights the fact that this technology can contribute additional and useful information to the field of archaeology. Based on these results, further study is planned in order to examine the capability of the sensor under different conditions and to further excavate the entire Leviah Enclosure.     

基于DM8168的小型机载视频记录仪设计

针对飞行器搭载相机视频存储和传输的需求,设计了一种基于异构多核处理器DM8168的小型机载视频记录仪。首先,基于DM8168和FPGA处理器设计小型化低功耗的硬件系统;接着,基于McFW软件架构设计视频采集编码的数据流,充分发挥DM8168多核体系结构的优势,实现两路1080P60视频的采集,并且对每路视频分别进行主次码流的H.264编码;最后,采用GPMC总线,将视频码流传输到FPGA处理器,以实现本地视频存储和远程视频传输。本记录仪体积小功耗低、记录时间长、视频传输时延小,能够很好地满足飞行器多路视频记录的需求。     

Photogrammetric processing of high-resolution airborne and satellite linear array stereo images for mapping applications

This article introduces a mathematical model for photogrammetric processing of linear array stereo images acquired by high-resolution satellite imaging systems such as IKONOS. The experimental result of the generation of simulated IKONOS stereo images based on photogrammetric principles, IKONOS imaging geometry and a set of georeferenced aerial images is presented. An accuracy analysis of ground points derived from the simulated IKONOS stereo images is performed. The impact of the number of GCPs (ground control points), distribution of GCPs, and image measurement errors on the ground point accuracy is investigated. It is concluded that an accuracy of ground coordinates from 2 m to 3 m is attainable with GCPs, and 5 m to 12 m without GCPs. Two data sets of HRSC (high resolution stereo camera) and MOMS (modular opto-electronic multispectral stereo-scanner)-2P are also utilized to test the model and system. The presented data processing method is a key to the generation of mapping products such as digital terrain models (DEM) and digitial shorelines from high-resolution satellite images.     

Racking focus and tracking focus on live video streams: a stereo solution

The ability to produce dynamic Depth of Field effects in live video streams was until recently a quality unique to movie cameras. In this paper, we present a computational camera solution coupled with real-time GPU processing to produce runtime dynamic Depth of Field effects. We first construct a hybrid-resolution stereo camera with a high-res/low-res camera pair. We recover a low-res disparity map of the scene using GPU-based Belief Propagation, and subsequently upsample it via fast Cross/Joint Bilateral Upsampling. With the recovered high-resolution disparity map, we warp the high-resolution video stream to nearby viewpoints to synthesize a light field toward the scene. We exploit parallel processing and atomic operations on the GPU to resolve visibility when multiple pixels warp to the same image location. Finally, we generate racking focus and tracking focus effects from the synthesized light field rendering. All processing stages are mapped onto NVIDIA’s CUDA architecture. Our system can produce racking and tracking focus effects for the resolution of 640×480 at 15 fps.     

基于H.264的多视点立体视频解码器优化算法研究*

与平面(单视点)视频相比,多视点立体视频的数据量成倍地增加,对解码速度以及播放的流畅性影响很大,成为限制其广泛应用的重要因素之一。为了提高多视点立体视频的解码速度,基于H.264/MVC标准,根据码字前缀的特点,将原有码表划分为若干区域,精确了查表范围,优化了熵解码中CAVLC的查表过程,并将优化后的解码器移植到播放器中。实验结果表明,提出的优化算法使查表部分的速度提高70%左右,整体解码时间提高了5.9%。最终达到了一定的解码优化效果,并实现了播放器对264格式8视点立体视频文件的解码及播放功能。     

Validation of a large area land cover product using purpose-acquired airborne video

Large area land cover products generated from remotely sensed data are difficult to validate in a timely and cost effective manner. As a result, pre-existing data are often used for validation. Temporal, spatial, and attribute differences between the land cover product and pre-existing validation data can result in inconclusive depictions of map accuracy. This approach may therefore misrepresent the true accuracy of the land cover product, as well as the accuracy of the validation data, which is not assumed to be without error. Hence, purpose-acquired validation data is preferred; however, logistical constraints often preclude its use — especially for large area land cover products. Airborne digital video provides a cost-effective tool for collecting purpose-acquired validation data over large areas. An operational trial was conducted, involving the collection of airborne video for the validation of a 31,000 km² sub-sample of the Canadian large area Earth Observation for Sustainable Development of Forests (EOSD) land cover map (Vancouver Island, British Columbia, Canada). In this trial, one form of agreement between the EOSD product and the airborne video data was defined as a match between the mode land cover class of a 3 by 3 pixel neighbourhood surrounding the sample pixel and the primary or secondary choice of land cover for the interpreted video. This scenario produced the highest level of overall accuracy at 77% for level 4 of classification hierarchy (13 classes). The coniferous treed class, which represented 71% of Vancouver Island, had an estimated user's accuracy of 86%. Purpose acquired video was found to be a useful and cost-effective data source for validation of the EOSD land cover product. The impact of using multiple interpreters was also tested and documented. Improvements to the sampling and response designs that emerged from this trial will benefit a full-scale accuracy assessment of the EOSD product and also provides insights for other regional and global land cover mapping programs.     

Rendering traditional mosaics

This paper discusses the principles of traditional mosaics and describes a technique for implementing a digital mosaicing system. The goal of this work is to transform digital images into traditional mosaic-like renderings. We achieve this effect by recovering free-form feature curves from the image and laying rows of tiles along these curves. Composition rules are applied to merge these tiles into an intricate jigsaw that conforms to classical mosaic styles. Mosaic rendering offers the user flexibility over every aspect of this craft, including tile arrangement, shapes, and colors. The result is a system that makes this wonderful craft more flexible and widely accessible than previously possible. Published online: 28 January 2003     

Surface descriptions from stereo and shading

Surface reconstruction from stereo and shape-from-shading have both been discussed extensively in the literature. In this paper methods that attempt to supplement stereo with analysis of shading, are reviewed, with comments on their robustness. Theoretical results on uniqueness in shape from shading are presented, and on the difficulty of estimating surface shape with local intensity operators. Preliminary experiments based on the method of Koenderinck and van Doorn, for analysing variation of shading with viewpoint, are reported. It is concluded that there is a variety of techniques for deriving qualitative shape information, which might be sufficiently robust for use in computer vision systems.     

Three-dimensional shape from color photometric stereo

Computer vision systems can be used to determine the shapes of real three-dimensional objects for purposes of object recognition and pose estimation or for CAD applications. One method that has been developed is photometric stereo. This method uses several images taken from the same viewpoint, but with different lightings, to determine the three-dimensional shape of an object. Most previous work in photometric stereo has been with gray-tone images; color images have only been used for dielectric materials. In this paper we describe a procedure for color photometric stereo, which recovers the shape of a colored object from two or more color images of the object under white illumination. This method can handle different types of materials, such as composites and metals, and can employ various reflection models such as the Lambertian, dichromatic, and Torrance-Sparrow models. For composite materials, colored metals, and dielectrics, there are two advantages of utilizing color information: at each pixel, there are more constraints on the orientation, and the result is less sensitive to noise. Consequently, the shape can be found more accurately. The method has been tested on both artificial and real images of objects of various materials, and on real images of a multi-colored object.