Towards 3-D model-based video coding

In the domain of telecommunication applications, videophony, teleconferency, the representation and modelization of human face, and its expressions, knows an important development. In this paper, we present the basic principles of image sequences coding with main approaches and methods to lead to 3D model-based coding. Then, we introduce our 3D wire-frame model with which we have developed some compression and triangulated surface representation methods. An original approach to simulate and reproduce facial expressions with radial basis functions is also presented.     

Statistical bias in 3-D reconstruction from a monocular video.

The present state-of-the-art in computing the error statistics in three-dimensional (3-D) reconstruction from video concentrates on estimating the error covariance. A different source of error which has not received much attention is the fact that the reconstruction estimates are often significantly statistically biased. In this paper, we derive a precise expression for the bias in the depth estimate, based on the continuous (differentiable) version of structure from motion (SfM). Many SfM algorithms, or certain portions of them, can be posed in a linear least-squares (LS) framework Ax = b. Examples include initialization procedures for bundle adjustment or algorithms that alternately estimate depth and camera motion. It is a well-known fact that the LS estimate is biased if the system matrix A is noisy. In SfM, the matrix A contains point correspondences, which are always difficult to obtain precisely; thus, it is expected that the structure and motion estimates in such a formulation of the problem would be biased. Existing results on the minimum achievable variance of the SfM estimator are extended by deriving a generalized Cramer-Rao lower bound. A detailed analysis of the effect of various camera motion parameters on the bias is presented. We conclude by presenting the effect of bias compensation on reconstructing 3-D face models from rendered images.     

Surface triangulation for 3-D scene model construction from stereo

The construction of an accurate 3-D scene model is a fundamental aspect of any model-based image coding scheme. This article describes the generation of a triangular facet surface representation from the data acquired by a calibrated binocular (stereo) camera system     

Multiresolution 3-D reconstruction from side-scan sonar images.

In this paper, a new method for the estimation of seabed elevation maps from side-scan sonar images is presented. The side-scan image formation process is represented by a Lambertian diffuse model, which is then inverted by a multiresolution optimization procedure inspired by expectation-maximization to account for the characteristics of the imaged seafloor region. On convergence of the model, approximations for seabed reflectivity, side-scan beam pattern, and seabed altitude are obtained. The performance of the system is evaluated against a real structure of known dimensions. Reconstruction results for images acquired by different sonar sensors are presented. Applications to augmented reality for the simulation of targets in sonar imagery are also discussed.     

基于连续超薄切片的线粒体三维重构

研究各种细胞器的三维形态,进而结合各种生物标记技术研究功能蛋白在细胞或细胞器中的空间分布,对于了解功能蛋白的功能和机理具有重要的价值。早在1954年,Gay和Anderson就进行了连续细胞水平的超薄切片的三维重构的工作。1974年,sjophistmnd将连续切片的三维重构分辨率提高到了     

Microwave image reconstruction from 3-D fields coupled to 2-D parameter estimation

An efficient Gauss-Newton iterative imaging technique utilizing a three-dimensional (3-D) field solution coupled to a two-dimensional (2-D) parameter estimation scheme (3-D/2-D) is presented for microwave tomographic imaging in medical applications. While electromagnetic wave propagation is described fully by a 3-D vector field, a 3-D scalar model has been applied to improve the efficiency of the iterative reconstruction process with apparently limited reduction in accuracy. In addition, the image recovery has been restricted to 2-D but is generalizable to three dimensions. Image artifacts related primarily to 3-D effects are reduced when compared with results from an entirely two-dimensional inversion (2-D/2-D). Important advances in terms of improving algorithmic efficiency include use of a block solver for computing the field solutions and application of the dual mesh scheme and adjoint approach for Jacobian construction. Methods which enhance the image quality such as the log-magnitude/unwrapped phase minimization were also applied. Results obtained from synthetic measurement data show that the new 3-D/2-D algorithm consistently outperforms its 2-D/2-D counterpart in terms of reducing the effective imaging slice thickness in both permittivity and conductivity images over a range of inclusion sizes and background medium contrasts.     

Motion-compensated 3-D subband coding of video

This paper describes a video coding system based on motion-compensated three-dimensional (3-D) subband/wavelet coding (MC-3DSBC), which can overcome the limits of both 3-D SBC and MC prediction-based coding. In this new system, spatio-temporal subbands are generated by MC temporal analysis and a spatial wavelet transform, and then encoded by 3-D subband-finite state scalar quantization (3DSB-FSSQ). The rate allocation from the GOP level to each class of subbands is optimized by utilizing the structural property of MC-3DSBC that additive superposition approximately holds for both rate and distortion. The proposed video coding system is applied to several test video clips. Its performance exceeds that of both a known MPEG-1 implementation and a similar subband MC predictive coder while maintaining modest computational complexity and memory size.     

Multirate 3-D subband coding of video

We propose a full color video compression strategy, based on 3-D subband coding with camera pan compensation, to generate a single embedded bit stream supporting multiple decoder display formats and a wide, finely gradated range of bit rates. An experimental implementation of our algorithm produces a single bit stream, from which suitable subsets are extracted to be compatible with many decoder frame sizes and frame rates and to satisfy transmission bandwidth constraints ranging from several tens of kilobits per second to several megabits per second. Reconstructed video quality from any of these bit stream subsets is often found to exceed that obtained from an MPEG-1 implementation, operated with equivalent bit rate constraints, in both perceptual quality and mean squared error. In addition, when restricted to 2-D, the algorithm produces some of the best results available in still image compression.     

单幅高分辨率SAR图像建筑物三维模型重构

提出了一种利用高分辨率SAR图像进行建筑物提取和三维重构的方法.首先,分析了高分辨率SAR图像建筑物产生的电磁散射的类型,给出了不同类型散射区域的后向散射计算方法,并在此基础上给出了一种利用建筑物三维CAD模型进行SAR建筑物特征区域图像仿真的方法;其次,给出了利用建筑物的二次散射结构确定建筑物底部轮廓位置和方向的方法,并提出了一种基于分布密度函数差异的仿真图像迭代匹配方法,进行建筑物高度的反演.仿真SAR图像后向散射系数用来划分建筑物不同的散射区域,通过计算特征区域之间的分布密度函数差异,以取得最大匹配度值的仿真图像对应的检验高度作为建筑物的反演高度;最后,选用了两幅不同屋顶类型的实际机载高分辨率SAR图像进行建筑物提取和三维重构实验,试验结果较为理想,验证了所提方法的可行性和有效性.     

Automated 3-D reconstruction of vascular structures from highdefinition casts

Three-dimensional reconstruction and computer modeling is becoming recognized as a powerful tool for studying vascular structures. The computational approach, as well as the computer hardware selected for the task, however, depend upon the information desired. For the modeling of surface geometry, as in the case of the aortic valve, a surface formation technique is favorable over the more computationally demanding volume rendering approach. Automated surface formation, however, requires good quality, high contrast images. We therefore present a technique for producing high contrast images from high definition casts. We also describe the methodology used for automatic contour tracing, generating a mesh of variable density, and the schemes used to reconstruct bifurcating objects. With this approach, 98 mbytes of imaging data could be reduced to 180 kbytes of polygon vertices, and manipulated at near real-time speed on a medium performance graphics workstation. Such a system is therefore well suited for detailed, quantitative analyses of the reconstructed structures. Overall, this paper outlines the procedures used to create a high definition, three-dimensional computer model of any vascular structure.     

On-line broadcast archives for interactive video

Delivery of user specified interactive multimedia digital services is drawing worldwide attention. With the availability of a greater number of channels as a result of video compression and the receding geographical barriers, content providers are hard pressed to meet the programme requirements. Broadcast archives are valuable multimedia resource centres, stocking audio and video material of interest to a vast population across the globe. There is a great potential for using these archives for on-line interactive access of these material. The paper describes the system architecture and implementation issues of an on-line broadcast archives management system (OLBAMS) which entails various functions of storage and dissemination, taking advantage of emerging digital technology, viz. video servers, OODBMS, CORBA, MPI video, use of Java applets for set-tops, webTV, or computers     

基于3G网络的视频直播传输系统方案设计

实现了标清视频通过3G网络进行直播传输,针对3G网络现状上行速率不足的问题,采用多信道绑定和信道均衡预处理技术,提出系统的解决方案,并给出了系统发送端设备的硬件和软件的实现方案,最后给出样机的测试和试用情况,结果显示该方案的有效性.     

FPGA在广播视频处理中的应用

本论文论述了提供一个完整的基于个人电脑的广播系统的可能性,以及这一系统为业界带来的全新的性价比和灵活性,同时介绍这一由Avvida公司研制的采用Altera公司的全新StratixFPGA系列组成的TsunamiPCI板。     

Improving calibration of 3-D video oculography systems

Eye movement recordings with video-based techniques have become very popular, as long as they are restricted to the horizontal and vertical movements of the eye. Reliable measurement of the torsional component of eye movements, which is especially important in the diagnosis and investigation of pathologies, has remained a coveted goal. One of the main reasons is unresolved technical difficulties in the analysis of video-based images of the eye. Based on simulations, we present solutions to two of the primary problems: a robust and reliable calibration of horizontal and vertical eye movement recordings, and the extraction of suitable iris patterns for the determination of the torsional eye position component.     

Spatio-temporal adaptive 3-D Kalman filter for video

This paper presents three-dimensional (spatio-temporal) Kalman filters for video as the extension of the two-dimensional (2-D) reduced update Kalman filter (RUKF) approach for images. We start out with three-dimensional (3-D) RUKF, a shift-invariant recursive estimator with efficiency advantages over the 3-D Wiener filter. Then, we turn to the motion-compensated extension MC-RUKF, which gives improved performance when coupled with a motion estimator. Since motion compensation sometimes fails, causing severe fluctuations in temporal correlation, we then present multimodel MC-RUKF, to adapt to variation in temporal and spatial correlation, by detecting the local image model out of a class, and using it in MC-RUKF. Finally, we introduce a novel multiscale model detection algorithm for use in high noise environments.     

3-D discrete shearlet transform and video processing

In this paper, we introduce a digital implementation of the 3-D shearlet transform and illustrate its application to problems of video denoising and enhancement. The shearlet representation is a multiscale pyramid of well-localized waveforms defined at various locations and orientations, which was introduced to overcome the limitations of traditional multiscale systems in dealing with multidimensional data. While the shearlet approach shares the general philosophy of curvelets and surfacelets, it is based on a very different mathematical framework, which is derived from the theory of affine systems and uses shearing matrices rather than rotations. This allows a natural transition from the continuous setting to the digital setting and a more flexible mathematical structure. The 3-D digital shearlet transform algorithm presented in this paper consists in a cascade of a multiscale decomposition and a directional filtering stage. The filters employed in this decomposition are implemented as finite-length filters, and this ensures that the transform is local and numerically efficient. To illustrate its performance, the 3-D discrete shearlet transform is applied to problems of video denoising and enhancement, and compared against other state-of-the-art multiscale techniques, including curvelets and surfacelets.     

3-D reconstruction of 2-D crystals in real space

A new algorithm for three-dimensional reconstruction of two-dimensional crystals from projections is presented, and its applicability to biological macromolecules imaged using transmission electron microscopy (TEM) is investigated. Its main departures from the traditional approach is that it works in real space, rather than in Fourier space, and it is iterative. This has the advantage of making it convenient to introduce additional constraints (such as the support of the function to be reconstructed, which may be known from alternative measurements) and has the potential of more accurately modeling the TEM image formation process. Phantom experiments indicate the superiority of the new approach even without the introduction of constraints in addition to the projection data.     

Integrating the scene length characteristics of MPEG video bitstreams into a direct broadcast satellite network with return channel system

In order to optimize the network resources, we should incorporate all the available information into the network design. However, incorporating irrelevant information may increase the design complexity and/or decrease the performance of the network. In this paper, we investigate the relevance of integrating the scene length characteristics of moving pictures expert group (MPEG) coded video bitstreams into a direct broadcast satellite (DBS) network with return channel system (DVB‐RCS). Due to the complexity of the studied system, unless disputable simplifications are made, it is hard to achieve a mathematical foundation for this integration. Our analysis relies on extensive set of simulations. Firstly, we achieve the scene length distributions for MPEG bitstreams based on the proposed scene change models and their subjective observations of the actual video. We show that these models may be used to estimate the scene length of MPEG bitstreams. We then integrate this estimation into a DBS network simulator. Finally, we show that the scene length characteristics may be used to improve the DBS network performance under certain conditions. Copyright © 2004 John Wiley & Sons, Ltd.     

基于虚拟现实的三维动态场景重建

针对传统的重建过程在面对动态景观的重建过程中由于采集的数据具有一定的连续性会产生一定的数据干扰以及数据跃迁的问题,提出一种基于虚拟现实的三维动态场景重建方案。使用虚拟现实技术中的单目视觉法进行数据采集处理,保证在数据采集过程中的数据阻断性,避免连接数据的相互跃迁,并最大限度地保证清晰度。对光度立体视觉进行了重新设计,能够保证动态场景的动态呈现,极大地改善视觉阻扰的发生。进行三维重建过程中为了能够保证动态数据的连续性,通过对高阶纹理以及透明度等数据的动态参设,达到最佳的成像状态。经过重建的三维动态场景必须经过去燥处理,这样才能保证成像的完整度。为了验证设计的基于虚拟现实的三维动态场景重建方案的有效性,设计了对比仿真试验。通过实验数据表明,基于虚拟现实的三维动态场景重建方案趋光性明显比较平稳,可以保证动态呈现效果达到最佳状态,能够准确连续的对动态场景进行重建。     

3-D image reconstruction from averaged Fourier transform magnitudeby parameter estimation

An object model and estimation procedure for three-dimensional (3-D) reconstruction of objects from measurements of the spherically averaged Fourier transform magnitudes is described. The motivating application is the 3-D reconstruction of viruses based on solution X-ray scattering data. The object model includes symmetry, positivity and support constraints and has the form of a truncated orthonormal expansion and the parameters are estimated by maximum likelihood methods. Successful 3-D reconstructions based on synthetic and experimental measurements from Cowpea mosaic virus are described.