Matching point features under small nonrigid motion

This paper describes a method for matching point features between images of objects that have undergone small nonrigid motion. Feature points are assumed to be available and, given a properly extracted set of feature points, a robust matching is established under the condition that the local nonrigid motion of each point is restricted to a circle of radius δ, where δ is not too large. This is in contrast to other techniques for point matching which assume either rigid motion or nonrigid motion of a known kind. The point matching problem is viewed in terms of weighted bipartite graph matching. In order to account for the possibility that the feature selector can be imprecise, we incorporate a greedy matching strategy with the weighted graph matching algorithm. Our algorithm is robust and insensitive to noise and missing features. The resulting matching can be used with image warping or other techniques for nonrigid motion analysis, image subtraction, etc. We present our experimental results on sequences of mammograms, images of a deformable clay object and satellite cloud images. In the first two cases we provide quantitative comparison with known ground truth.     

Object matching using deformable templates

We propose a general object localization and retrieval scheme based on object shape using deformable templates. Prior knowledge of an object shape is described by a prototype template which consists of the representative contour/edges, and a set of probabilistic deformation transformations on the template. A Bayesian scheme, which is based on this prior knowledge and the edge information in the input image, is employed to find a match between the deformed template and objects in the image. Computational efficiency is achieved via a coarse-to-fine implementation of the matching algorithm. Our method has been applied to retrieve objects with a variety of shapes from images with complex background. The proposed scheme is invariant to location, rotation, and moderate scale changes of the template     

Unsupervised learning of an atlas from unlabeled point-sets

One of the key challenges in deformable shape modeling is the problem of estimating a meaningful average or mean shape from a set of unlabeled shapes. We present a new joint clustering and matching algorithm that is capable of computing such a mean shape from multiple shape samples which are represented by unlabeled point-sets. An iterative bootstrap process is used wherein multiple shape sample point-sets are nonrigidly deformed to the emerging mean shape, with subsequent estimation of the mean shape based on these nonrigid alignments. The process is entirely symmetric with no bias toward any of the original shape sample point-sets. We believe that this method can be especially useful for creating atlases of various shapes present in medical images. We have applied the method to create mean shapes from nine hand-segmented 2D corpus callosum data sets and 10 hippocampal 3D point-sets.     

Learning to Match: Deriving Optimal Template-Matching Algorithms from Probabilistic Image Models

Finding correspondences between images by template matching is a common problem in image understanding. Although a variety of solutions have been proposed, most of them rely on the arbitrary choice of a template and a matching function. Often, different cost functions lead to different results, and the choice of a good cost for a specific application remains an art. Statistical models on the other hand, allow us to derive optimal learning and matching algorithms from modeling assumptions using likelihood maximization principles. The key contribution of this paper is the development of a statistical framework for learning what function to optimize from training examples. We present a family of statistical models for grayscale images, which allow us to derive optimal template-matching algorithms. The intensity at each pixel is described by a random variable whose distribution is encoded by a deformable template. Firstly, we assume the intensity distribution to be Gaussian and derive an intensity-matching algorithm, which is a generalization of the classical sum-of-squared differences. Then, we introduce a hidden segmentation variable in the probabilistic model and derive a segmentation-matching algorithm that can handle photometric variations. Both models are exemplified on the automatic detection of anatomical landmarks in brain Magnetic Resonance Images.     

A Deformable Object Tracking Algorithm Based on the Boundary Element Method that is Robust to Occlusions and Spurious Edges

The manipulation of deformable objects is an important problem in robotics and arises in many applications including biomanipulation, microassembly, and robotic surgery. For some applications, the robotic manipulator itself may be deformable. Vision-based deformable object tracking can provide feedback for these applications. Computer vision is a logical sensing choice for tracking deformable objects because the large amount of data that is collected by a vision system allows many points within the deformable object to be tracked simultaneously. This article introduces a template based deformable object tracking algorithm, based on the boundary element method, that is able to track a wide range of deformable objects. The robustness of this algorithm to occlusions and to spurious edges in the source image is also demonstrated. A robust error measure is used to handle the problem of occlusion and an improved edge detector based on the Canny edge operator is used to suppress spurious edges. This article concludes by quantifying the performance increase provided by the robust error measure and the robust edge detector. The performance of the algorithm is also demonstrated through the tracking of a sequence of cardiac MRI images.     

A fast modal space transform for robust nonrigid shape retrieval

Nonrigid or deformable 3D objects are common in many application domains. Retrieval of such objects in large databases based on shape similarity is still a challenging problem. In this paper, we take advantages of functional operators as characterizations of shape deformation, and further propose a framework to design novel shape signatures for encoding nonrigid geometries. Our approach constructs a context-aware integral kernel operator on a manifold, then applies modal analysis to map this operator into a low-frequency functional representation, called fast functional transform, and finally computes its spectrum as the shape signature. In a nutshell, our method is fast, isometry-invariant, discriminative, smooth and numerically stable with respect to multiple types of perturbations. Experimental results demonstrate that our new shape signature for nonrigid objects can outperform all methods participating in the nonrigid track of the SHREC’11 contest. It is also the second best performing method in the real human model track of SHREC’14.     

A Fast 2D Shape Recovery Approach by Fusing Features and Appearance

In this paper, we present a fusion approach to solve the nonrigid shape recovery problem, which takes advantage of both the appearance information and the local features. We have two major contributions. First, we propose a novel progressive finite Newton optimization scheme for the feature-based nonrigid surface detection problem, which is reduced to only solving a set of linear equations. The key is to formulate the nonrigid surface detection as an unconstrained quadratic optimization problem that has a closed-form solution for a given set of observations. Second, we propose a deformable Lucas-Kanade algorithm that triangulates the template image into small patches and constrains the deformation through the second-order derivatives of the mesh vertices. We formulate it into a sparse regularized least squares problem, which is able to reduce the computational cost and the memory requirement. The inverse compositional algorithm is applied to efficiently solve the optimization problem. We have conducted extensive experiments for performance evaluation on various environments, whose promising results show that the proposed algorithm is both efficient and effective.     

Tracking Motion, Deformation, and Texture Using Conditionally Gaussian Processes

We present a generative model and inference algorithm for 3D nonrigid object tracking. The model, which we call G-flow, enables the joint inference of 3D position, orientation, and nonrigid deformations, as well as object texture and background texture. Optimal inference under G-flow reduces to a conditionally Gaussian stochastic filtering problem. The optimal solution to this problem reveals a new space of computer vision algorithms, of which classic approaches such as optic flow and template matching are special cases that are optimal only under special circumstances. We evaluate G-flow on the problem of tracking facial expressions and head motion in 3D from single-camera video. Previously, the lack of realistic video data with ground truth nonrigid position information has hampered the rigorous evaluation of nonrigid tracking. We introduce a practical method of obtaining such ground truth data and present a new face video data set that was created using this technique. Results on this data set show that G-flow is much more robust and accurate than current deterministic optic-flow-based approaches.     

Bayesian grid matching

A method for locating distorted grid structures in images is presented. The method is based on the theories of template matching and Bayesian image restoration. The grid is modeled as a deformable template. Prior knowledge of the grid is described through a Markov random field (MRF) model which represents the spatial coordinates of the grid nodes. Knowledge of how grid nodes are depicted in the observed image is described through the observation model. The prior consists of a node prior and an arc (edge) prior, both modeled as Gaussian MRFs. The node prior models variations in the positions of grid nodes and the arc prior models variations in row and column spacing across the grid. Grid matching is done by placing an initial rough grid over the image and applying an ensemble annealing scheme to maximize the posterior distribution of the grid. The method can be applied to noisy images with missing grid nodes and grid-node artifacts and the method accommodates a wide range of grid distortions including: large-scale warping, varying row/column spacing, as well as nonrigid random fluctuations of the grid nodes. The methodology is demonstrated in two case studies concerning (1) localization of DNA signals in hybridization filters and (2) localization of knit units in textile samples.     

Vehicle segmentation and classification using deformable templates

This paper proposes a segmentation algorithm using deformable template models to segment a vehicle of interest both from the stationary complex background and other moving vehicles in an image sequence. We define a polygonal template to characterize a general model of a vehicle and derive a prior probability density function to constrain the template to be deformed within a set of allowed shapes. We propose a likelihood probability density function which combines motion information and edge directionality to ensure that the deformable template is contained within the moving areas in the image and its boundary coincides with strong edges with the same orientation in the image. The segmentation problem is reduced to a minimization problem and solved by the Metropolis algorithm. The system was successfully tested on 405 image sequences containing multiple moving vehicles on a highway     

A constraint-based approach to shape management in multimedia databases

Shape management is an important functionality in multimedia databases. Shape information can be used in both image acquisition and image retrieval. Several approaches have been proposed to deal with shape representation and matching. Among them, the data-driven approach supports searches for shapes based on indexing techniques. Unfortunately, efficient data-driven approaches are often defined only for specific types of shape. This is not sufficient in contexts in which arbitrary shapes should be represented. Constraint databases use mathematical theories to finitely represent infinite sets of relational tuples. They have been proved to be very useful in modeling spatial objects. In this paper, we apply constraint-based data models to the problem of shape management in multimedia databases. We first present the constraint model and some constraint languages. Then, we show how constraints can be used to model general shapes. The use of a constraint language as an internal specification and execution language for querying shapes is also discussed. Finally, we show how a constraint database system can be used to efficiently retrieve shapes, retaining the advantages of the already defined approaches.     

Rendering deformable surface reflectance fields

Animation of photorealistic computer graphics models is an important goal for many applications. Image-based modeling has emerged as a promising approach to capture and visualize real-world objects. Animating image-based models, however, is still a largely unsolved problem. In this paper, we extend a popular image-based representation called surface reflectance field to animate and render deformable real-world objects under arbitrary illumination. Deforming the surface reflectance field is achieved by modifying the underlying impostor geometry. We augment the impostor by a local parameterization that allows the correct evaluation of acquired reflectance images, preserving the original light model on the deformed surface. We present a deferred shading scheme to handle the increased amount of data involved in shading the deformable surface reflectance field. We show animations of various objects that were acquired with 3D photography.     

A framework for automatic landmark identification using a newmethod of nonrigid correspondence

A framework for automatic landmark identification is presented based on an algorithm for corresponding the boundaries of two shapes. The auto-landmarking framework employs a binary tree of corresponded pairs of shapes to generate landmarks automatically on each of a set of example shapes. The landmarks are used to train statistical shape models, known as point distribution models. The correspondence algorithm locates a matching pair of sparse polygonal approximations, one for each of a pair of boundaries by minimizing a cost function, using a greedy algorithm. The cost function expresses the dissimilarity in both the shape and representation error (with respect to the defining boundary) of the sparse polygons. Results are presented for three classes of shape which exhibit various types of nonrigid deformation     

Shape retrieval using triangle-area representation and dynamic space warping

In this paper, we present a shape retrieval method using triangle-area representation for nonrigid shapes with closed contours. The representation utilizes the areas of the triangles formed by the boundary points to measure the convexity/concavity of each point at different scales (or triangle side lengths). This representation is effective in capturing both local and global characteristics of a shape, invariant to translation, rotation, and scaling, and robust against noise and moderate amounts of occlusion. In the matching stage, a dynamic space warping (DSW) algorithm is employed to search efficiently for the optimal (least cost) correspondence between the points of two shapes. Then, a distance is derived based on the optimal correspondence. The performance of our method is demonstrated using four standard tests on two well-known shape databases. The results show the superiority of our method over other recent methods in the literature.     

Flexible syntactic matching of curves and its application toautomatic hierarchical classification of silhouettes

Curve matching is one instance of the fundamental correspondence problem. Our flexible algorithm is designed to match curves under substantial deformations and arbitrary large scaling and rigid transformations. A syntactic representation is constructed for both curves and an edit transformation which maps one curve to the other is found using dynamic programming. We present extensive experiments where we apply the algorithm to silhouette matching. In these experiments, we examine partial occlusion, viewpoint variation, articulation, and class matching (where silhouettes of similar objects are matched). Based on the qualitative syntactic matching, we define a dissimilarity measure and we compute it for every pair of images in a database of 121 images. We use this experiment to objectively evaluate our algorithm. First, we compare our results to those reported by others. Second, we use the dissimilarity values in order to organize the image database into shape categories. The veridical hierarchical organization stands as evidence to the quality of our matching and similarity estimation     

A new 3D-matching method of nonrigid and partially similar models using curve analysis

The 3D-shape matching problem plays a crucial role in many applications, such as indexing or modeling, by example. Here, we present a novel approach to matching 3D objects in the presence of nonrigid transformation and partially similar models. In this paper, we use the representation of surfaces by 3D curves extracted around feature points. Indeed, surfaces are represented with a collection of closed curves, and tools from shape analysis of curves are applied to analyze and to compare curves. The belief functions are used to define a global distance between 3D objects. The experimental results obtained on the TOSCA and the SHREC07 data sets show that the system performs efficiently in retrieving similar 3D models.     

Nonrigid Matching of Undersampled Shapes via Medial Diffusion

We introduce medial diffusion for the matching of undersampled shapes undergoing a nonrigid deformation. We construct a diffusion process with respect to the medial axis of a shape, and use the quantity of heat diffusion as a measure which is both tolerant of missing data and approximately invariant to nonrigid deformations. A notable aspect of our approach is that we do not define the diffusion on the shape's medial axis, or similar medial representation. Instead, we construct the diffusion process directly on the shape. This permits the diffusion process to better capture surface features, such as varying spherical and cylindrical parts, as well as combine with other surface‐based diffusion processes. We show how to use medial diffusion to detect intrinsic symmetries, and for computing correspondences between pairs of shapes, wherein shapes contain substantial missing data.     

Polyhedral object recognition by indexing

In computer vision, the indexing problem is the problem of recognizing a few objects in a large database of objects while avoiding the help of the classical image-feature-to-object-feature matching paradigm. In this paper we address the problem of recognizing three-dimensional (3-D) polyhedral objects from 2-D images by indexing. Both the objects to be recognized and the images are represented by weighted graphs. The indexing problem is therefore the problem of determining whether a graph extracted from the image is present or absent in a database of model graphs. We introduce a novel method for performing this graph indexing process which is based both on polynomial characterization of binary and weighted graphs and on hashing. We describe in detail this polynomial characterization and then we show how it can be used in the context of polyhedral object recognition. Next we describe a practical recognition-by-indexing system that includes the organization of the database, the representation of polyhedral objects in terms of 2-D characteristic views, the representation of this views in terms of weighted graphs and the associated image processing. Finally, some experimental results allow the evaluation of the system performance.     

融合图像感知哈希技术的运动目标跟踪

目的 为解决运动目标跟踪时因遮挡、尺度变换等产生的目标丢失以及传统匹配跟踪算法计算复杂度高等问题,提出一种融合图像感知哈希技术的运动目标跟踪算法.方法 本文算法利用感知哈希技术提取目标摘要进行模板图像识别匹配,采用匹配跟踪策略和搜索跟踪策略相配合来准确跟踪目标,并构建模板评价函数和模板更新准则实现目标模板的自适应更新,保证其在目标发生遮挡和尺度变换情况下的适应性.结果 该算法与基于NCC(normalized cross correlation)的模板匹配跟踪算法、Mean-shift跟踪算法以及压缩跟踪算法相比,在目标尺度变换和物体遮挡时,跟踪的连续性和稳定性更好,且具有较低的计算复杂度,能分别降低跟踪系统约6.2%、 6.3%、 9.3%的计算时间.结论 本文算法能有效实现视频场景中目标发生遮挡及尺度变换情况下的跟踪,跟踪的连续性和稳定性良好,且算法具有较低的计算复杂度,有利于实时性跟踪系统的构建.