未知室内环境的三维景物描述

景物描述是计算机视觉的一个重要内容。提出了一个利用三维信息对未知景物的描述方法。该方法对三维场景进行物体级和结构级两级描述；根据三维物体分割的结果，首先计算单个物体的最大尺度、形状类型和边集等整体特征，继而推测它们之间的相互位置关系。从数学上推导了“简化边框矩”和“平面惯量张量”的计算公式，并给出了实验结果。     

结合场景描述的文本生成图像方法

研究基于场景描述文本生成对应图像的方法,针对生成图像常常出现的对象重叠和缺失问题,提出了一种结合场景描述的生成对抗网络模型。首先,利用掩模生成网络对数据集进行预处理,为数据集中的对象提供分割掩模向量。然后,将生成的对象分割掩模向量作为约束,通过描述文本训练布局预测网络,得到各个对象在场景布局中的具体位置和大小,并将结果送入到级联细化网络模型,完成图像的生成。最后,将场景布局与图像共同引入到布局鉴别器中,弥合场景布局与图像之间的差距,得到更加真实的场景布局。实验结果表明,所提模型能够生成与文本描述更匹配的图像,图像更加自然,同时有效地提高了生成图像的真实性和多样性。     

Temporal stabilization of Video Object Segmentation for 3D-TV applications

Our aim is to insert depth information into an existing 2D video sequence to provide content for 3D-TV applications, which we try to achieve through segmentation of the objects in the given 2D video sequence. To this effect, we present a method for temporal stabilization of video object segmentation algorithms for 3D-TV applications. First, two quantitative measures to evaluate temporal stability without ground-truth are discussed. Then, a pseudo-3D curve evolution method, which spatio-temporally stabilizes the estimated segmentation of a video object is introduced. Temporal stability is achieved by re-distributing existing object segmentation errors such that they will be less disturbing when the scene is rendered and viewed in 3D. Our starting point is the hypothesis that if making segmentation errors is inevitable, these errors should be made in a temporally consistent way for 3D-TV applications. This hypothesis is supported by the experiments, which show that there is significant improvement in segmentation quality both in terms of the objective quantitative measures and in terms of the viewing comfort in subjective perceptual tests. Therefore, it is possible to increase the perceptual object segmentation quality without increasing the actual segmentation accuracy.     

Video representation with three-dimensional entities

Very low bit-rate coding requires new paradigms that go well beyond pixel- and frame-based video representations. We introduce a novel content-based video representation using tridimensional entities: textured object models and pose estimates. The multiproperty object models carry stochastic information about the shape and texture of each object present in the scene. The pose estimates define the position and orientation of the objects for each frame. This representation is compact. It provides alternative means for handling video by manipulating and compositing three-dimensional (3-D) entities. We call this representation tridimensional video compositing, or 3DVC for short. We present the 3DVC framework and describe the methods used to construct incrementally the object models and the pose estimates from unregistered noisy depth and texture measurements. We also describe a method for video frame reconstruction based on 3-D scene assembly, and discuss potential applications of 3DVC to video coding and content-based handling. 3DVC assumes that the objects in the scene are rigid and segmented. By assuming segmentation, we do not address the difficult questions of nonrigid segmentation and multiple object segmentation. In our experiments, segmentation is obtained via depth thresholding. It is important to notice that 3DVC is independent of the segmentation technique adopted. Experimental results with synthetic and real video sequences where compression ratios in the range of 1:150-1:2700 are achieved demonstrate the applicability of the proposed representation to very low bit-rate coding     

Image-based rendering and 3D modeling: A complete framework

Multi-viewpoint synthesis of video data is a key technology for the integration of video and 3D graphics, as necessary for telepresence and augmented-reality applications. This paper describes a number of important techniques which can be employed to accomplish that goal. The techniques presented are based on the analysis of 2D images acquired by two or more cameras. To determine depth information of single objects present in the scene, it is necessary to perform segmentation and disparity estimation. It is shown, how these analysis tools can benefit from each other. For viewpoint synthesis, techniques with different levels of tradeoff between complexity and degrees of freedom are presented. The first approach is disparity-controlled view interpolation, which is capable of generating intermediate views along the interocular axis between two adjacent cameras. The second is the recently introduced incomplete 3D technique, which in a first step extracts the texture of the visible surface of a video object acquired with multiple cameras, and then performs disparity-compensated projection from the surface onto a view plane. In the third and most complex approach, a 3D model of the object is generated, which can be represented by a 3D wire grid. For synthesis, this model can be rotated to arbitrary orientations, and original texture is mapped onto the surface to obtain an arbitrary view of the processed object. The result of this rendering procedure is a virtual image with very natural appearance.     

Region-based image coding using polynomial intensity functions

The vast majority of coded images are real-world images. These images consist of distinct objects within a scene, where each object has its own reflective, textural and lighting characteristics. Region-based image coding encodes these images by partitioning the scene into objects, and then describing each object's characteristics using a set of parameters. The paper uses orthonormal polynomial functions to describe the lighting and reflective characteristics of each object. The coefficients of these polynomials are coded with linear quantisers that have their decision boundaries spaced according to rate-distortion considerations. The textural component of each object is coded using vector quantisation of the autocorrelation coefficients of the residual. The partitioning of the image into distinct objects is achieved with a segmentation algorithm which attempts to maximise the rate-distortion performance of the encoding procedure as a whole. In doing so, the segmentation algorithm partitions the image into distinct objects as well as providing estimates for the optimal bit allocations among the polynomial coefficients. Results generated by this method show reconstructions with quality superior to other region-based methods, both objectively and subjectively     

一种基于相对模糊连通度的交互式序列图像快速分割算法

在基于相对模糊连通度的交互式图像分割方法的基础上提出了一种序列图像分割的快速算法。对于单幅图像,新算法在保持分割精度的同时,运行速度提高了3倍。作为该算法的扩展,一方面将原算法中单目标分割推广到多目标分割,另一方面将单帧图像的分割推广到序列图像的分割,实现了复杂背景下多目标的图像分割和图像序列的批处理分割。并进行了分割结果的后处理,提取出目标的单像素宽度的光滑边缘。用人造图像和实际的医学图像和图像序列所做的测试实验取得了令人满意的分割结果。     

Segmentation and tracking of moving objects for content-based videocoding

To enable content-based functionalities in video coding, a decomposition of the scene into physical objects is required. Such objects are normally not characterised by homogeneous colour, intensity, or optical flow. Therefore, conventional techniques based on these low-level features cannot perform the desired segmentation. The authors address segmentation and tracking of moving objects and present a new video object plane (VOP) segmentation algorithm that extracts semantically meaningful objects. A morphological motion filter detects physical objects by identifying areas that are moving differently from the background. A new filter criterion is introduced that measures the deviation of the estimated local motion from the synthesised global motion. A two-dimensional binary model is derived for the object of interest and tracked throughout the sequence by a Hausdorff object tracker. To accommodate for rotations and changes in shape, the model is updated every frame by a two-stage method that accounts for rigid and non-rigid moving parts of the object. The binary model then guides the actual VOP extraction, whereby a novel boundary post-processor ensures high boundary accuracy. Experimental results demonstrate the performance of the proposed algorithm     

3D Shape Inferencing and Modeling for Video Retrieval

We present a geometry-based indexing approach for the retrieval of video databases. It consists of two modules: 3D object shape inferencing from video data and geometric modeling from the reconstructed shape structure. A motion-based segmentation algorithm employing feature block tracking and principal component split is used for multi-moving-object motion classification and segmentation. After segmentation, feature blocks from each individual object are used to reconstruct its motion and structure through a factorization method. The estimated shape structure and motion parameters are used to generate the implicit polynomial model for the object. The video data is retrieved using the geometric structure of objects and their spatial relationship. We generalize the 2D string to 3D to compactly encode the spatial relationship of objects.     

智能型目标识别系统

介绍了图像目标识别技术中的图像分割，不变性参数提取和目标分类，利用图像目标的均匀性和相应知识自适应地分割和提取图像目标，被提取的每个图像目标的不变性参数由归一化过程和Ｚｅｒｎｉｋｅ矩提取，并利用ＭＰＮＮ模型将图像目标分类，实验结果该识别系统能识别光照不均匀或复杂背景下的图像目标。     

Towards 3-D scene reconstruction from broadcast video

Three-dimensional (3-D) scene reconstruction from broadcast video is a challenging problem with many potential applications, such as 3-D TV, free-view TV, augmented reality or three-dimensionalization of two-dimensional (2-D) media archives. In this paper, a flexible and effective system capable of efficiently reconstructing 3-D scenes from broadcast video is proposed, with the assumption that there is relative motion between camera and scene/objects. The system requires no a priori information and input, other than the video sequence itself, and capable of estimating the internal and external camera parameters and performing a 3-D motion-based segmentation, as well as computing a dense depth field. The system also serves as a showcase to present some novel approaches for moving object segmentation, sparse and dense reconstruction problems. According to the simulations for both synthetic and real data, the system achieves a promising performance for typical TV content, indicating that it is a significant step towards the 3-D reconstruction of scenes from broadcast video.     

视频序列客体分布阵集成检测分层技术

视频图像序列按客体分层在很多领域都具有重要的应用，通常，视频图像可分为背景客体和其它目标客体。本文根据场景视频图像序列中不同客体的特性，针对背景客体研究了分布阵集成检测分层技术，采用两级模式识别及多帧加权估计。实验表明该技术具有很好的分层效果，提高了客体分层的可靠性，极大地抑制了噪声。