Three-Dimensional Shape Knowledge for Joint Image Segmentation and Pose Tracking

In this article we present the integration of 3-D shape knowledge into a variational model for level set based image segmentation and contour based 3-D pose tracking. Given the surface model of an object that is visible in the image of one or multiple cameras calibrated to the same world coordinate system, the object contour extracted by the segmentation method is applied to estimate the 3-D pose parameters of the object. Vice-versa, the surface model projected to the image plane helps in a top-down manner to improve the extraction of the contour. While common alternative segmentation approaches, which integrate 2-D shape knowledge, face the problem that an object can look very differently from various viewpoints, a 3-D free form model ensures that for each view the model can fit the data in the image very well. Moreover, one additionally solves the problem of determining the object’s pose in 3-D space. The performance is demonstrated by numerous experiments with a monocular and a stereo camera system.     

应用多个正交视角轮流逼近3维目标的坐标

为实现目标的快速、精确3维定位和跟踪,提出一种正交摄像机视频定位系统及其坐标轮流逼近的迭代定位算法。系统中平面摄像机光轴按照正交方式布置,拍摄方向均指向原点。不同于现有的大部分计算机视觉方法,本算法中没有影响定位效率和精度问题的图像配准操作。证明了迭代算法的收敛性。数值验证和实际试验表明,本算法计算简单、误差稳定性好,收敛快,因此具有良好的应用潜力。     

多模式3维视频形状编码

目的 具有立体感和高端真实感的3D视频正越来越受到学术界和产业界的关注和重视,未来在3D影视、机器视觉、远程医疗、军事航天等领域将有着广泛的应用前景。对象基3D视频是未来3D视频技术的重要发展趋势,其中高效形状编码是对象基3D视频应用中的关键问题。但现有形状编码方法主要针对图像和视频对象,面向3D视频的形状编码算法还很少。为此,基于对象基3D视频的应用需求,提出一种基于轮廓和链码表示的高效多模式3D视频形状编码方法。方法 对于给定的3D视频形状序列逐帧进行对象轮廓提取并预处理后,进行对象轮廓活动性分析,将形状图像分成帧内模式编码图像和帧间预测模式编码图像。对于帧内编码图像,基于轮廓内链码方向约束和线性特征进行高效编码。对于帧间编码图像,采用基于链码表示的轮廓基运动补偿预测、视差补偿预测、联合运动与视差补偿预测等多种模式进行编码,以充分利用视点内对象轮廓的帧间时域相关性和视点间对象轮廓的空域相关性,从而达到高效编码的目的。结果 实验仿真结果显示所提算法性能优于经典和现有的最新同类方法,压缩效率平均能提高9.3%到64.8%不等。结论 提出的多模式3D视频形状编码方法可以有效去除对象轮廓的帧间和视点间冗余,能够进行高效编码压缩,性能优于现有同类方法,可广泛应用于对象基编码、对象基检索、对象基内容分析与理解等。     

3-D shape recovery using distributed aspect matching

An approach to the recovery of 3-D volumetric primitives from a single 2-D image is presented. The approach first takes a set of 3-D volumetric modeling primitives and generates a hierarchical aspect representation based on the projected surfaces of the primitives; conditional probabilities capture the ambiguity of mappings between levels of the hierarchy. From a region segmentation of the input image, the authors present a formulation of the recovery problem based on the grouping of the regions into aspects. No domain-independent heuristics are used; only the probabilities inherent in the aspect hierarchy are exploited. Once the aspects are recovered, the aspect hierarchy is used to infer a set of volumetric primitives and their connectivity. As a front end to an object recognition system, the approach provides the indexing power of complex 3-D object-centered primitives while exploiting the convenience of 2-D viewer-centered aspect matching; aspects are used to represent a finite vocabulary of 3-D parts from which objects can be constructed     

A multilevel decision fusion approach for urban mapping using very high-resolution multi/hyperspectral imagery

A novel multilevel decision fusion approach is proposed for urban mapping using very-high-resolution (VHR) multi/hyperspectral imagery. The proposed framework consists of three levels: (1) at level I, we first propose a self-dual filter for extracting structural features from the VHR imagery–subsequently, the spectral and structural features are integrated based on a weighted probability fusion; (2) level II extends level I by implementing the spectral–structural fusion in an object-based framework; and (3) at level III, the object-based probabilistic outputs at level II are used to identify unreliable objects, and shape attributes of these unreliable objects are then considered for refinement of classification. At this level, a decision-level object merging is used to improve the initial segmentation, since shape feature extraction is highly dependent on the quality of segmentation. Experiments were conducted on a Hyperspectral Digital Imagery Collection Experiment (HYDICE) DC Mall image and a QuickBird Beijing data set. The results revealed that the proposed approach provided progressively increasing accuracies when the multilevel features were gradually considered in the processing chain.     

Knowledge-driven ultrasonic three-dimensional organ modeling

A representation is described for nonstructured biologic objects which are single-valued distortions of a sphere. The representation is implemented in a model-driven system for extracting three-dimensional (3-D) organ reconstructions from a series of arbitrarily oriented ultrasound slices. A training set of ultrasonic reconstructions of similarly shaped objects is used to give the computer generic knowledge of a given shape class. This knowledge is in the form of local slope constraints defined on an object coordinate system. The combination of constraints, interacting together via a relaxation process on continuous label sets, attempts to capture the essential shape and range of variation for an organ class. An initial tolerance region and ``bestguess' organ surface are established by the interaction of the learned shape knowledge with manually input organ landmarks. A hypothesize-verify paradigm is employed to alternately request new data and to update the tolerance region and bestguess surface. Examples from runs on two balloon classes are presented. These examples show: 1) the local constraints interact to produce a reasonable global depiction of the essential shape and range of variation, 2) the use of shape knowledge permits accurate results from only one third of the available data, and 3) the 3-D shape knowledge provides a two-dimensional (2-D) tolerance region for plan-guided edge detection.     

自由差分运算与直交型Snake模型

自由差分运算能够突破传统微分算子的模板运算模式,把对图像这种二维信号的微分计算转化成一维的方式进行,使计算方向可以根据实际问题灵活选择．直交型Snake模型基于任意曲线可由多边形进行逼近的思想．借助自由差分运算,把本应沿曲线法线方向计算的灰度梯度转化成在多边形各边的直交线上分析灰度的相对变化或绝对变化．在曲线的演化过程中,该模型以目标曲线的特征为指导,以基本多边形为依托,通过在直交线上进行差分操作,从而把轮廓变化的信息和图像灰度变化的信息联系在一起,使得所提取的轮廓曲线的特征不断地逼近目标曲线的特征．与已有的Snake模型相比,直交型Snake模型通过强调高层视觉和底层信息之间的相互联系在曲线演化过程中的作用,使得信息的处理流程处于闭环状态．在对畸形物体的轮廓提取实验中,获得了比较理想的结果．     

Contour based object detection using part bundles

In this paper we propose a novel framework for contour based object detection from cluttered environments. Given a contour model for a class of objects, it is first decomposed into fragments hierarchically. Then, we group these fragments into part bundles, where a part bundle can contain overlapping fragments. Given a new image with set of edge fragments we develop an efficient voting method using local shape similarity between part bundles and edge fragments that generates high quality candidate part configurations. We then use global shape similarity between the part configurations and the model contour to find optimal configuration. Furthermore, we show that appearance information can be used for improving detection for objects with distinctive texture when model contour does not sufficiently capture deformation of the objects.     

A shape prior constraint for implicit active contours

We present a shape prior constraint to guide the evolution of implicit active contours. Our method includes three core techniques. Firstly, a rigid registration is introduced, using a line search method within a level set framework. The method automatically finds the time step for the iterative optimization processes. The order for finding the optimal translation, rotation and scale is derived experimentally. Secondly, a single reconstructed shape is created from a shape distribution of a previously acquired learning set. The reconstructed shape is applied to guide the active contour evolution. Thirdly, our method balances the impact of the shape prior versus the image guidance of the active contour. A mixed stopping condition is defined based on the stationarity of the evolving curve and the shape prior constraint. Our method is completely non-parametric and avoids taking linear combinations of non-linear signed distance functions, which would cause problems because distance functions are not closed under linear operations. Experimental results show that our method is able to extract the desired objects in several circumstances, namely when noise is present in the image, when the objects are in slightly different poses and when parts of the object are invisible in the image.     

Moments of superellipsoids and their application to range image registration

Cartesian moments are frequently used global geometrical features in computer vision for object pose estimation and recognition. We derive a closed form expression for 3-D Cartesian moment of order p+q+r of a superellipsoid in its canonical coordinate system. We also show how 3-D Cartesian moment of a globally deformed superellipsoid in general position and orientation can be computed as a linear combination of 3-D Cartesian moments of the corresponding nondeformed superellipsoid in canonical coordinate system. Additionally, moments of objects that are compositions of superellipsoids can be computed as simple sums of moments of individual parts. To demonstrate practical application of the derived results we register pairs of range images based on moments of recovered compositions of superellipsoids. We use a standard technique to find centers of gravity and principal axes in pairs of range images while third-order moments are used to resolve the four-way ambiguity. Experimental results show expected improvement of recovered rigid transformation based on moments of recovered superellipsoids as compared to the registration based on moments of raw range image data. Besides object pose estimation the presented results can be directly used for object recognition with moments and/or moment invariants as object features.     

Optimal Image and Video Closure by Superpixel Grouping

Detecting independent objects in images and videos is an important perceptual grouping problem. One common perceptual grouping cue that can facilitate this objective is the cue of contour closure, reflecting the spatial coherence of objects in the world and their projections as closed boundaries separating figure from background. Detecting contour closure in images consists of finding a cycle of disconnected contour fragments that separates an object from its background. Searching the entire space of possible groupings is intractable, and previous approaches have adopted powerful perceptual grouping heuristics, such as proximity and co-curvilinearity, to constrain the search. We introduce a new formulation of the problem, by transforming the problem of finding cycles of contour fragments to finding subsets of superpixels whose collective boundary has strong edge support (few gaps) in the image. Our cost function, a ratio of a boundary gap measure to area, promotes spatially coherent sets of superpixels. Moreover, its properties support a global optimization procedure based on parametric maxflow. Extending closure detection to videos, we introduce the concept of spatiotemporal closure. Analogous to image closure, we formulate our spatiotemporal closure cost over a graph of spatiotemporal superpixels. Our cost function is a ratio of motion and appearance discontinuity measures on the boundary of the selection to an internal homogeneity measure of the selected spatiotemporal volume. The resulting approach automatically recovers coherent components in images and videos, corresponding to objects, object parts, and objects with surrounding context, providing a good set of multiscale hypotheses for high-level scene analysis. We evaluate both our image and video closure frameworks by comparing them to other closure detection approaches, and find that they yield improved performance.     

基于先验形状的人工目标识别与轮廓恢复

以具有典型形状特征的操场为例,探讨了形状特征在人工目标自动识别中的应用。基于形状特征的人工目标识别,一般首先将图像分割为多个基元,然后依据人工目标有较规则形状的特点,以形状特征作为指标进行识别。由于高分辨率遥感图像细节信息丰富,这种方法所提取的目标轮廓往往不完整。本文提出了"图像-基元-目标-轮廓恢复"的识别模式,在目标识别后采用加入形状先验知识的主动轮廓模型对目标进行轮廓恢复。实验结果表明,这种方法可以有效地修正目标提取结果中的轮廓缺失。     

三维物体识别及姿态测定的推理系统

本文介绍了三维物体识别及姿态测定的一种新技术，从物体空间域模型出发，通过约束推理及几何推理，在物体三维信息部分给定的条件下，推断预测图象模型，并通过实测的图象数据反馈，推断出隐含在图象中未给定的三维信息，最终实现三维物体识别及姿态测定。整个系统在ＶＩＣＯＭ机上用Ｃ语言完成。     

Contour-based object detection as dominant set computation

Contour-based object detection can be formulated as a matching problem between model contour parts and image edge fragments. We propose a novel solution by treating this problem as the problem of finding dominant sets in weighted graphs. The nodes of the graph are pairs composed of model contour parts and image edge fragments, and the weights between nodes are based on shape similarity. Because of high consistency between correct correspondences, the correct matching corresponds to a dominant set of the graph. Consequently, when a dominant set is determined, it provides a selection of correct correspondences. As the proposed method is able to get all the dominant sets, we can detect multiple objects in an image in one pass. Moreover, since our approach is purely based on shape, we also determine an optimal scale of target object without a common enumeration of all possible scales. Both theoretic analysis and extensive experimental evaluation illustrate the benefits of our approach.     

Three-Dimensional Modeling and Effects on Still Images

Designers and creative artists use computer graphics and image processing effects on stall photographs in application areas such as advertising entertainment broadcasting and the arts Most of the effects available in research arid commercial work are two-dimensional in nature, for example image processing filters [blur, edge enhancement) and creative effects (tilings, reflections) There is almost no usage of information taken from the 3-D world in which the objects appearing an the image are located. In this paper we present a novel method for creating 3-D effects on photographs or in general on any image created by rendering a 3-D world The artist interacts with the image using a set of intuitive direct manipulation interface objects These objects let the user define a 3-D model, display at, and manipulate it in a 3-D space which is correlated with that of the input image. The generated model can be an arbitrarily complex 3-D polyhedron Any texture, including texture taken from the input photograph, can be mapped into any of its faces arid used for special effects We discuss and show examples for effects such as copy and paste, motion blur, model editing and deformations lighting effects, and shadows.     

基于轮廓追踪的字符识别特征提取

字符识别是模式识别的一个重要分支,其关键是特征向量的选择与提取。小波分解和分形在图像处理方面有着广泛的应用,在结合二者特点的基础上提出了一种新的基于轮廓追踪的字符识别特征选取方法。即对于一个输入的字符图像经预处理提取其轮廓,并由轮廓追踪获得边缘点坐标序列,实现了从二维图像数据到一维数据的转化,对得到的一维曲线进行小波分解,计算少数几个分解得到的曲线的分形维数,以它们构成特征向量。并对有关字符做了实验,其效果是令人满意的。