Object-oriented analysis-synthesis coding of moving images

An object-oriented analysis-synthesis coder is presented which encodes objects instead of blocks of N × N picture elements. The objects are described by three parameter sets defining the motion, shape and colour of an object. The parameter sets are obtained by image analysis based on source models of either moving 2D-objects or moving 3D-objects. Known coding techniques are used to encode the parameter sets. An object-depending parameter coding allows to introduce geometrical distortions instead of quantization errors. Using the transmitted parameter sets an image can be reconstructed by model-based image synthesis.

Experimental results achieved with a first implementation of the coder are given and are discussed.     

Lossy to lossless object-based coding of 3-D MRI data

We propose a fully three-dimensional (3-D) object-based coding system exploiting the diagnostic relevance of the different regions of the volumetric data for rate allocation. The data are first decorrelated via a 3-D discrete wavelet transform. The implementation via the lifting steps scheme allows to map integer-to-integer values, enabling lossless coding, and facilitates the definition of the object-based inverse transform. The coding process assigns disjoint segments of the bitstream to the different objects, which can be independently accessed and reconstructed at any up-to-lossless quality. Two fully 3-D coding strategies are considered: embedded zerotree coding (EZW-3D) and multidimensional layered zero coding (MLZC), both generalized for region of interest (ROI)-based processing. In order to avoid artifacts along region boundaries, some extra coefficients must be encoded for each object. This gives rise to an overheading of the bitstream with respect to the case where the volume is encoded as a whole. The amount of such extra information depends on both the filter length and the decomposition depth. The system is characterized on a set of head magnetic resonance images. Results show that MLZC and EZW-3D have competitive performances. In particular, the best MLZC mode outperforms the others state-of-the-art techniques on one of the datasets for which results are available in the literature.     

Disparity field and depth map coding for multiview 3D image generation

In the present paper techniques are examined for the coding of the depth map and disparity fields for stereo or multiview image communication applications. It is assumed that both the left and right channels of the multiview image sequence are coded using block- or object-based methods. A dynamic programming algorithm is used to estimate a disparity field between each stereo image pair. Depth is then estimated and occlusions are optionally detected, based on the estimated disparity fields. Spatial interpolation techniques are examined based on the disparity/depth information and the detection of occluded regions using either stereoscopic or trinocular camera configurations. It is seen that the presence of a third camera at the transmitter site improves the estimation of disparities, the detection of occlusions and the accuracy of the resulting spatial interpolation at the receiver. Various disparity field and depth map coding techniques are then proposed and evaluated, with emphasis given to the quality of the resulting intermediate images at the receiver site. Block-based and wireframe modeling techniques are examined for the coding of isolated depth or disparity map information. Further, 2D and 3D motion compensation techniques are evaluated for the coding of sequences of depth or disparity maps. The motion fields needed may be available as a byproduct of block-based or object-based coding of the intensity images. Experimental results are given for the evaluation of the performance of the proposed coding and spatial interpolation methods.     

Hybrid model-and-object-based real-time conversational video coding

Bandwidth-constrained real-time conversational video communications (such as mobile teleconferencing) require video codecs with good rate-distortion characteristics at low bit-rates and modest computational complexity. While target-specific object-based and model-based coding methods have been proposed for low bit-rate conversational video coding, difficulties in generalization and high computational complexity hinder their practical utilization. In this paper, we propose a low bit-rate coding method for typical conversational video by combining two-dimensional model-based coding of face regions and object-based coding of non-face head-shoulder regions, achieving high-quality face reconstruction and low overall bit-rate with real-time encoding capability. Experiments on typical conversational test sequences confirm that, compared to other conversational video codecs, our model-and-object-based coding method offers superior rate-distortion performance at low bit-rates.     

非正交多址中继系统复数域网络编码优化设计

无线中继与网络编码是提高通信系统数据可靠性和吞吐率的重要技术手段,重点研究了单中继非正交多址系统复数域网络编码的优化设计问题.首先,在限制源节点和中继节点的总发送功率并假设各个源节点发送功率相同的条件下,证明了当源节点与中继节点发送功率相等时基于复数域网络编码的单中继非正交多址系统误符号率(SEP)性能最佳.其次,以编码增益为目标函数,提出了一种复数域网络编码的优化设计方法.与已有的复数域网络编码系数设计方法相比,此方法能满足任意源节点个数条件下复数域网络编码系数的优化设计要求,同时可以有效地避免因为网络编码系数选择不当造成信号重叠的现象,提升和改进系统的SEP性能.     

Optimization of layered multicast with network coding and genetic algorithm

This article studies the problem of constructing optimal layered multicast with network coding for heterogeneous networks.Based on the flexibility of layered source coding, a global-favorable optimization scheme is proposed, which maximizes the aggregate throughput of heterogeneous sink nodes for layered multicast with network coding by determining the optimal bit rates of the layers. To solve this global-favorable optimization scheme, especially in the large-scale heterogeneous networks, a new problem-specific genetic algorithm (GA) is further proposed. It not only searches efficiently for the optimal allocation of layer bit rates, but also guarantees the validity of candidate solutions in the whole evolutionary process. Simulation results demonstrate that this new GA-based optimization scheme could obtain efficiently the optimal or satisfactorily near-optimal bit rates for layered multicast with network coding, even in the large-scale heterogeneous networks.     

Layered coding for ATM based video distribution systems

This paper discusses packet loss and its protection in an asynchronous transfer mode (ATM) based video distribution system. Packet losses in ATM based networks have such a great impact on the design of coding algorithms and network architectures that they should be exhaustively discussed and resolved. In this paper, first basic configuration of the ATM based video transmission system and its packet-loss protection schemes are discussed. The DCT based layered coding scheme with packet priority classification is proposed as an effective packet-loss protection scheme. Burstiness characteristics of the broadcast video sources are evaluated and modeled to clarify statistical multiplexing performance and packet-loss properties. The quality degradation caused by the packet losses is also evaluated by the SNR, and the superior performance of the proposed layered coding scheme is verified.     

Estimation of point light source parameters for object-based coding

In this paper, the source model of moving rigid 3D objects of an object-based analysis-synthesis coder (OBASC) is extended from diffuse to non-diffuse illumination introducing the explicit illumination model of a distant point light source and ambient diffuse light. For each image of a real image sequence containing moving objects, first, shape and 3D motion parameters describing the objects are estimated assuming an ellipsoid-like smooth shape. Then, the illumination parameters are estimated by a fast iterative maximum-likelihood Gauβ-Newton estimation method. Typically, the illumination parameters converge after very few images close to the true ones. The accurateness depends on the amount of object rotation and the correctness of the shape assumptions. For a real image sequence showing a textured ball covering 20% of image area, rotating about 10 ° per frame, and illuminated by spot and ambient light, the extension of the source model reduces the model failures from 9.9% of the image area to 6.7%. In the area of model failures, the image synthesized from the source model parameters differ significantly from the real image. In this early experiment, source model parameters are coded losslessly. Since model failures are expensive by means of bit-rate, a significant reduction of bit-rate can be expected.     

Object-based analysis-synthesis coding (OBASC) based on the sourcemodel of moving flexible 3-D objects

This correspondence investigates object-based analysis-synthesis coding (OBASC) for the encoding of moving images at very low data rates. According to the source model, each moving object of an image is described and encoded by three parameter sets defining its motion, shape, and surface color. The parameter sets of each object are obtained by model-based image analysis. They are coded by an object-dependent parameter coding. Using the coded parameter sets, an image can be synthesized by model-based image synthesis. Here, OBASC based on the source model of "moving flexible 3-D objects with 3-D motion" (F3D) is introduced. The efficiency of this source model F3D is compared to the efficiency of OBASC based on the source model of "moving rigid 3-D objects with 3-D motion" (R3D). Compared to R3D, F3D requires the additional transmission of flexible-shape parameters. Therefore, the source model F3D is only applied in those areas of the image which cannot be described by the source model R3D. The new source model F3D reduces the bit rate from 64 to 56 kb/s, providing the same picture quality measured by the SNR of the encoded color parameters.     

Adaptive shape and texture intra refreshment schemes for improved error resilience in object-based video coding

Video encoders may use several techniques to improve error resilience. In particular, for video encoders that rely on predictive (inter) coding to remove temporal redundancy, intra coding refreshment is especially useful to stop temporal error propagation when errors occur in the transmission or storage of the coded streams, since these errors may cause the decoded quality to decay very rapidly. In the context of object-based video coding, intra coding refreshment can be applied to both the shape and texture data. In this paper, novel shape and texture intra refreshment schemes are proposed which can be used by object-based video encoders, such as MPEG-4 video encoders, independently or combined. These schemes allow to adaptively determine when the shape and texture of the various video objects in a scene should be refreshed in order to maximize the decoded video quality for a certain total bit rate.     

Scalable broadcast with network coding in heterogeneous networks

This article studies the scalable broadcast scheme realized with the joint application of layered source coding,unequal error protection(UEP)and random network coding from the theoretical point of view.The success probability for any non-source node in a heterogeneous network to recover the most important layers of the source data is deduced.This probability proves that in this broadcast scheme every non-source node with enough capacity can always recover the source data partially or entirely as long as the finite field size is sufficiently large.Furthermore,a special construction for the local encoding kernel at the source node is proposed.With this special construction,an increased success probability for partial decoding at any non-source node is achieved,i.e.,the partial decodability offered by the scalable broadcast scheme is improved.     

Multiresolution coding techniques for digital television: A review

Multiresolution decompositions for video coding are reviewed. Both nonrecursive and recursive coding schemes are considered. In nonrecursive schemes, it is shown that pyramid structures have certain advantages over subband or wavelet techniques, and a specific spatiotemporal pyramid coding of HDTV is discussed in some detail. It is shown that recursive, DPCM like schemes will incur a slight loss of optimality due to a restricted form of prediction if multiresolution decomposition with compatible decoding is required. Compatibility and transmission issues are also discussed. Multiresolution transmission for digital broadcast TV is introduced. This, when combined with multiresolution source coding, achieves spectrum efficiency, robustness and graceful degradation under channel impairments.Invited PaperWork supported in part by the National Science Foundation under grants ECD-88-11111, MIP-90-14189 and Bell Communications Research.Work supported by the National Science Foundation under grants ECD-88-11111. K.M. Uz is now with David Sarnoff Research Center in Princeton, NJ 08543.     

Object-based coding of stereoscopic and 3D image sequences

The author begins by discussing what object based coding is and goes on to consider the structure of object based stereoscopic coders. Different techniques for object-based stereoscopic image sequence coding are reviewed. These techniques basically differ in the way they define models and estimate model parameters. We review the various models used for representing motion and structure. Then we review segmentation techniques, and discuss coding of object parameters and image synthesis     

VIDEO CODING SCHEME FOR ATM NETWORKS

A new variable bit rate layered coding scheme based on subband coding is given in this paper. It can make full use of the merits of ATM transmission, as well as compensation for the quality degradation caused by cell loss, which is an inherent problem of ATM networks, and thus a stable, high quality video transmission is achieved. This paper presents the coding scheme, cell construction and simulation results.     

帧差预测3D-DCT子带分层编码的研究

本文提出了一种利用３Ｄ－ＤＣＴ帧差预测变换域频率系数进行了带划分的分层编码方法。与其它变换系数分层编码相比,３Ｄ－ＤＣＴ变换编码概念的引入,使该方法可以在实现空间方向分层的同时,实现时间方向分层。另外,它比采用时间方向亚抽样和空间滤波器的分辨率分层方法更灵活,并克服了频率混叠现象。该方法可用于实现分层图像通信,适合未通信发展的需要。     

Intra-layer network coding for lossy communication of progressive codes

In this paper we discuss layered multicast (LM) of progressive source codes using network coding. LM is absolutely optimal if different sinks in the network are satisfied up to their max-flow. Since absolutely optimal intra-layer network strategies might not exist for general networks, we present conditions under which an absolutely optimal, intra-layer multicast strategy exists for a given network and how that strategy may be efficiently constructed. We also discuss the problem of designing optimal intra-layer multicast strategies for general directed networks.     

Adaptive coding of moving objects for very low bit rates

This paper describes an object-based video coding system with new ideas in both the motion analysis and source encoding procedures. The moving objects in a video are extracted by means of a joint motion estimation and segmentation algorithm based on the Markov random field (MRF) model. The two important features of the presented technique are the temporal linking of the objects, and the guidance of the motion segmentation with spatial color information. This facilitates several aspects of an object-based coder. First, a new temporal updating scheme greatly reduces the bit rate to code the object boundaries without resorting to crude lossy approximations. Next, the uncovered regions can be extracted and encoded in an efficient manner by observing their revealed contents. The objects are classified adaptively as P objects or I objects and encoded accordingly. Subband/wavelet coding is applied in encoding the object interiors. Simulations at very low bit rates yielded comparable performance in terms of reconstructed PSNR to the H.263 coder. The object-based coder produced visually more pleasing video with less blurriness and devoid of block artifacts, thus confirming the advantages of object-based coding at very low bit-rates     

Object-oriented analysis-synthesis coding based on moving two-dimensional objects

An object-oriented analysis-synthesis coder is presented which encodes arbitrarily shaped objects instead of rectangular blocks. The objects are described by three parameter sets defining their motion, shape and colour. Throughout this contribution, the colour parameters denote the luminance and chrominance values of the object surface. The parameter sets of each object are obtained by image analysis based on source models of moving 2D-objects and coded by an object-dependent parameter coding. Using the coded parameter sets an image can be reconstructed by model-based image synthesis. In order to cut down the generated bit-rate of the parameter coding, the colour updating of an object is suppressed if the modelling of the object by the source model is sufficiently exact, i.e., if only a relatively small colour update information would be needed for an errorless image synthesis. Omitting colour update information, small position errors of objects denoted as geometrical distortions are allowed for image synthesis instead of quantization error distortions. Tolerating geometrical distortions, the image area to be updated by colour coding can be decreased to 4% of the image size without introducing annoying distortions. As motion and shape parameters can efficiently be coded, about 1 bit per pel remains for colour updating in a 64 kbit/s coder compared to about 0.1 bit per pel in the standard reference coder (RM8) of the CCITT. Experimental results concerning the efficient coding of motion and shape parameters are given and discussed. The coding of the colour information will be dealt with in further research.     

视频编码的技术基础及发展方向

现有视频编码采用基于块的混合编码架构,利用预测、变换、量化和熵编码技术实现对视频信号的高效压缩.在现有架构基础上进一步优化,提供针对视频图像信号局部特性的更加灵活的处理和编码.基于机器学习的视频编码技术有望部分或全面地改变现有的混合编码框架,给视频编码带来新的研究思路.未来视频除了现有的二维平面视频,还需要编码面向AR/VR应用的球面视频数据和体视频数据,这些新的视频源数据格式也给视频编码技术研究带来新的机会和挑战.     

Joint source-channel coding for wireless object-based video communications utilizing data hiding.

In recent years, joint source-channel coding for multimedia communications has gained increased popularity. However, very limited work has been conducted to address the problem of joint source-channel coding for object-based video. In this paper, we propose a data hiding scheme that improves the error resilience of object-based video by adaptively embedding the shape and motion information into the texture data. Within a rate-distortion theoretical framework, the source coding, channel coding, data embedding, and decoder error concealment are jointly optimized based on knowledge of the transmission channel conditions. Our goal is to achieve the best video quality as expressed by the minimum total expected distortion. The optimization problem is solved using Lagrangian relaxation and dynamic programming. The performance of the proposed scheme is tested using simulations of a Rayleigh-fading wireless channel, and the algorithm is implemented based on the MPEG-4 verification model. Experimental results indicate that the proposed hybrid source-channel coding scheme significantly outperforms methods without data hiding or unequal error protection.