Disparity field and depth map coding for multiview 3D image generation

In the present paper techniques are examined for the coding of the depth map and disparity fields for stereo or multiview image communication applications. It is assumed that both the left and right channels of the multiview image sequence are coded using block- or object-based methods. A dynamic programming algorithm is used to estimate a disparity field between each stereo image pair. Depth is then estimated and occlusions are optionally detected, based on the estimated disparity fields. Spatial interpolation techniques are examined based on the disparity/depth information and the detection of occluded regions using either stereoscopic or trinocular camera configurations. It is seen that the presence of a third camera at the transmitter site improves the estimation of disparities, the detection of occlusions and the accuracy of the resulting spatial interpolation at the receiver. Various disparity field and depth map coding techniques are then proposed and evaluated, with emphasis given to the quality of the resulting intermediate images at the receiver site. Block-based and wireframe modeling techniques are examined for the coding of isolated depth or disparity map information. Further, 2D and 3D motion compensation techniques are evaluated for the coding of sequences of depth or disparity maps. The motion fields needed may be available as a byproduct of block-based or object-based coding of the intensity images. Experimental results are given for the evaluation of the performance of the proposed coding and spatial interpolation methods.     

Bayesian estimation of Dirichlet mixture model with variational inference

In statistical modeling, parameter estimation is an essential and challengeable task. Estimation of the parameters in the Dirichlet mixture model (DMM) is analytically intractable, due to the integral expressions of the gamma function and its corresponding derivatives. We introduce a Bayesian estimation strategy to estimate the posterior distribution of the parameters in DMM. By assuming the gamma distribution as the prior to each parameter, we approximate both the prior and the posterior distribution of the parameters with a product of several mutually independent gamma distributions. The extended factorized approximation method is applied to introduce a single lower-bound to the variational objective function and an analytically tractable estimation solution is derived. Moreover, there is only one function that is maximized during iterations and, therefore, the convergence of the proposed algorithm is theoretically guaranteed. With synthesized data, the proposed method shows the advantages over the EM-based method and the previously proposed Bayesian estimation method. With two important multimedia signal processing applications, the good performance of the proposed Bayesian estimation method is demonstrated.     

Incentive analysis for cooperative interactive multiview video streaming

In interactive multiview video streaming (IMVS), users can periodically select one out of many captured views available for observation. In single-view video streaming, cooperative strategies where peers share received packets of the same video have proven to be effective in reducing server׳s upload burden, and incentive mechanisms are designed to stimulate user cooperation. However, exploiting user cooperation in higher dimensional IMVS is difficult, since users watching different views makes it difficult to establish partnership, and users switching views frequently and independently makes it difficult to maintain partnership over time. In this paper, we use a multiview video frame structure for IMVS to support cooperative view-switching, where peers may help each other even if they are observing different views. We then model peers׳ interaction as an indirect reciprocity game, where each user is assigned a reputation level. To gain a higher reputation level, users help others, which in turn leads to a higher likelihood to receive others׳ help later. In this work, we focus on how view switching, the key feature of IMVS, affects user cooperation. By modeling users׳ decision making as a Markov decision process, our analysis shows that users tend to cooperate at some views but not others: given peers can predict their future view navigation paths probabilistically, for a peer who is likely to enter a view-switching path not requiring others׳ help, he also has less incentive to cooperate. Furthermore, we observe that the game may have multiple Nash Equilibria corresponding to different cooperation levels, e.g., users cooperate at all views in the full cooperation equilibrium, while users only cooperate at certain views in the partial cooperation equilibrium. The particular equilibrium the game will converge to depends on the initial cooperation level of the game. To stimulate user cooperation at all views, we propose a Pay-for-Cooperation (PfC) scheme at the beginning of the game to drive the game to the full cooperation equilibrium to improve system efficiency. Our simulation results show the effectiveness of PfC.     

Adaptive depth truncation filter for MVC based compressed depth image

In multiview video plus depth (MVD) format, virtual views are generated from decoded texture videos with corresponding decoded depth images through depth image based rendering (DIBR). 3DV-ATM is a reference model for the H.264/AVC based multiview video coding (MVC) and aims at achieving high coding efficiency for 3D video in MVD format. Depth images are first downsampled then coded by 3DV-ATM. However, sharp object boundary characteristic of depth images does not well match with the transform coding based nature of H.264/AVC in 3DV-ATM. Depth boundaries are often blurred with ringing artifacts in the decoded depth images that result in noticeable artifacts in synthesized virtual views. This paper presents a low complexity adaptive depth truncation filter to recover the sharp object boundaries of the depth images using adaptive block repositioning and expansion for increasing the depth values refinement accuracy. This new approach is very efficient and can avoid false depth boundary refinement when block boundaries lie around the depth edge regions and ensure sufficient information within the processing block for depth layers classification. Experimental results demonstrate that the sharp depth edges can be recovered using the proposed filter and boundary artifacts in the synthesized views can be removed. The proposed method can provide improvement up to 3.25 dB in the depth map enhancement and bitrate reduction of 3.06% in the synthesized views.     

面向CIMS的集成多方位建模与仿真框架

首先分析了几种常用建模方法，包括ＩＤＥＦ０、ＩＤＥＦ１、动态建模方法及ＯＭＴ方法。然后提出了一个集成多方位建模与仿真框架，它将功能、信息、动态视图及仿真核心集成在一起，实现了仿真模型的自动生成，它使得在系统功能设计早期就能预见，评价未来系统的有关动态品质。     

End-to-End Multiview Gesture Recognition for Autonomous Car Parking System

The use of hand gestures can be the most intuitive human-machine interaction medium.The early approaches for hand gesture recognition used device-based methods....     

Unbalanced JPEG image steganalysis via multiview data match

Image steganalysis must address the matter of learning from unbalanced training sets where the cover objects (normal images) always greatly outnumber the stego ones. But the research in unbalanced image steganalysis is seldom seen. This work just focuses on the problem of unbalance JPEG images steganalysis. In this paper, we propose a frame of feature dimension reduction based semi-supervised learning for high-dimensional unbalanced JPEG image steganalysis. Our method uses standard steganalysis features, and selects the confident stego images from the unlabeled examples by multiview match resampling method to rebalance the unbalanced training images. Furthermore, weighted Fisher linear discriminant (WFLD) is proposed to find the proper feature subspace where K-means provides the weight factor for WFLD in return. Finally, WFLD and K-means both work in an iterative fashion until convergence. Experimental results on the MBs and nsF5 steganographic methods show the usefulness of the developed scheme over current popular feature spaces.     

基于HBP编码结构的MVC快速算法

针对MVC系统中运算量最大的矢量预测部分——单向搜索和双向搜索,该文提出一种快速算法。根据多视点视频的序列相关性,采用立体-运动约束模型在运动域和视差域进行联合迭代单向搜索,设计可信度因子来自适应地调整修正搜索窗口,同步搜索得到运动和视差矢量。此外,通过分析双向预测宏块的特性,提出一种双向搜索提前中止算法。实验结果证明该文算法的有效性,能在保持编码质量的同时极大减少算法复杂度。     

Reconstruction from Projections Using Grassmann Tensors

In this paper a general procedure is given for reconstruction of a set of feature points in an arbitrary dimensional projective space from their projections into lower dimensional spaces. This extends the methods applied in the well-studied problem of reconstruction of scene points in ℘³ given their projections in a set of images. In this case, the bifocal, trifocal and quadrifocal tensors are used to carry out this computation. It is shown that similar methods will apply in a much more general context, and hence may be applied to projections from ℘ ⁿ to ℘ ^m , which have been used in the analysis of dynamic scenes, and in radial distortion correction. For sufficiently many generic projections, reconstruction of the scene is shown to be unique up to projectivity, except in the case of projections onto one-dimensional image spaces (lines), in which case there are two solutions. Projections from ℘ ⁿ to ℘² have been considered by Wolf and Shashua (in International Journal of Computer Vision 48(1): 53–67, 2002), where they were applied to several different problems in dynamic scene analysis. They analyzed these projections using tensors, but no general way of defining such tensors, and computing the projections was given. This paper settles the general problem, showing that tensor definition and retrieval of the projections is always possible.