Figure-ground separation by cue integration

This letter presents an improved cue integration approach to reliably separate coherent moving objects from their background scene in video sequences. The proposed method uses a probabilistic framework to unify bottom-up and top-down cues in a parallel, "democratic" fashion. The algorithm makes use of a modified Bayes rule where each pixel's posterior probabilities of figure or ground layer assignment are derived from likelihood models of three bottom-up cues and a prior model provided by a top-down cue. Each cue is treated as independent evidence for figure-ground separation. They compete with and complement each other dynamically by adjusting relative weights from frame to frame according to cue quality measured against the overall integration. At the same time, the likelihood or prior models of individual cues adapt toward the integrated result. These mechanisms enable the system to organize under the influence of visual scene structure without manual intervention. A novel contribution here is the incorporation of a top-down cue. It improves the system's robustness and accuracy and helps handle difficult and ambiguous situations, such as abrupt lighting changes or occlusion among multiple objects. Results on various video sequences are demonstrated and discussed. (Video demos are available at http://organic.usc.edu:8376/ approximately tangx/neco/index.html .).     

Blended deformable models

This paper develops a new class of parameterized models based on the linear interpolation of two parameterized shapes along their main axes, using a blending function. This blending function specifies the relative contribution of each component shape on the resulting blended shape. The resulting blended shape can have aspects of each of the component shapes. Using a small number of additional parameters, blending extends the coverage of shape primitives while also providing abstraction of shape. In particular, it offers the ability to construct shapes whose genus can change. Blended models are incorporated into a physics-based shape estimation framework which uses dynamic deformable models. Finally, we present experiments involving the extraction of complex shapes from range data including examples of dynamic genus change     

Animation of deformable models

Although kinematic modelling methods are adequate for describing the shapes of static objects, they are insufficient when it comes to producing realistic animation. Physically based modelling remedies this problem by including forces, masses, strain energies and other physical quantities. The paper describes a system for the animation of deformable models. The system uses physically based modelling methods and approaches from elasticity theory for animating the models. Two different formulations, namely the primal formulation and the hybrid formulation, are implemented so that the user can select the one most suitable for an animation depending on the rigidity of the models. Collision of the models with impenetrable obstacles and constraining of the model points to fixed positions in space are implemented for use in the animations.     

Elastically adaptive deformable models

We present a technique for the automatic adaptation of a deformable model's elastic parameters within a Kalman filter framework for shape estimation applications. The novelty of the technique is that the model's elastic parameters are not constant, but spatio-temporally varying. The variation of the elastic parameters depends on the distance of the model from the data and the rate of change of this distance. Each pass of the algorithm uses physics-based modeling techniques to iteratively adjust both the geometric and the elastic degrees of freedom of the model in response to forces that are computed from the discrepancy between the model and the data. By augmenting the state equations of an extended Kalman filter to incorporate these additional variables, we are able to significantly improve the quality of the shape estimation. Therefore, the model's elastic parameters are always initialized to the same value and they are subsequently modified depending on the data and the noise distribution. We present results demonstrating the effectiveness of our method for both two-dimensional and three-dimensional data.     

Fragments based tracking with adaptive cue integration

In this paper, we address the issue of part-based tracking by proposing a new fragments-based tracker. The proposed tracker enhances the recently suggested FragTrack algorithm to employ an adaptive cue integration scheme. This is done by embedding the original tracker into a particle filter framework, associating a reliability value to each fragment that describes a different part of the target object and dynamically adjusting these reliabilities at each frame with respect to the current context. Particularly, the vote of each fragment contributes to the joint tracking result according to its reliability, and this allows us to achieve a better accuracy in handling partial occlusions and pose changes while preserving and even improving the efficiency of the original tracker. In order to demonstrate the performance and the effectiveness of the proposed algorithm we present qualitative and quantitative results on a number of challenging video sequences.     

Heating and melting deformable models

We develop physically-based graphics models of non-rigid objects capable of heat conduction, thermoelasticity, melting and fluid-like behaviour in the molten state. These deformable models feature non-rigid dynamics governed by Lagrangian equations of motion and conductive heat transfer governed by the heat equation for non-homogeneous, non-isotropic media. In its solid state, the discretized model is an assembly of hexahedral finite elements in which thermoelastic units interconnect particles situated in a lattice. The stiffness of a thermoelastic unit decreases as its temperature increases, and the unit fuses when its temperature exceeds the melting point. The molten state of the model involves a molecular dynamics simulation in which ‘fluid’ particles that have broken free from the lattice interact through long-range attraction forces and short-range repulsion forces. We present a physically-based animation of a thermoelastic model in a simulated physical world populated by hot constraint surfaces.     

Dependent multiple cue integration for robust tracking

We propose a new technique for fusing multiple cues to robustly segment an object from its background in video sequences that suffer from abrupt changes of both illumination and position of the target. Robustness is achieved by the integration of appearance and geometric object features and by their estimation using Bayesian filters, such as Kalman or particle filters. In particular, each filter estimates the state of a specific object feature, conditionally dependent on another feature estimated by a distinct filter. This dependence provides improved target representations, permitting to segment it out from the background even in non-stationary sequences. Considering that the procedure of the Bayesian filters may be described by a "hypotheses generation--hypotheses correction" strategy, the major novelty of our methodology compared to previous approaches is that the mutual dependence between filters is considered during the feature observation, i.e, into the "hypotheses correction" stage,instead of considering it when generating the hypotheses. This proves to be much more effective in terms of accuracy and reliability. The proposed method is analytically justified and applied to develop a robust tracking system that adapts online and simultaneously the color space where the image points are represented, the color distributions, the contour of the object and its bounding box. Results with synthetic data and real video sequences demonstrate the robustness and versatility of our method.     

Boundary finding with parametrically deformable models

Segmentation using boundary finding is enhanced both by considering the boundary as a whole and by using model-based global shape information. The authors apply flexible constraints, in the form of a probabilistic deformable model, to the problem of segmenting natural 2-D objects whose diversity and irregularity of shape make them poorly represented in terms of fixed features or form. The parametric model is based on the elliptic Fourier decomposition of the boundary. Probability distributions on the parameters of the representation bias the model to a particular overall shape while allowing for deformations. Boundary finding is formulated as an optimization problem using a maximum a posteriori objective function. Results of the method applied to real and synthetic images are presented, including an evaluation of the dependence of the method on prior information and image quality     

Interactively deformable models for surgery simulation

A methodology that addresses important issues concerned with the underlying graphical models designed for surgical simulation, as well as issues related to the real-time interactivity with, and manipulation of, these models is presented. The specific application of interest is laparoscopic surgery, which is performed using endoscopes that present a video image of the organs to the clinicians. The surgeon then performs the surgery while looking at the video monitor. The particular focus is gall bladder surgery, which involves various gastrointestinal organs. The overall objective is to simulate this environment by creating realistic, manipulable models of these organs. The models are interactively manipulable and exhibit behavior both visually acceptable and physically accurate. The approach is based on the notion of active surfaces. The rationale, mathematical formalism, and visualization techniques encompassed by the methodology are described. Recent results obtained from applying these methods to the problem of endoscopic gall bladder surgery simulation are presented     

Image segmentation by probabilistic bottom-up aggregation and cue integration

We present a bottom-up aggregation approach to image segmentation. Beginning with an image, we execute a sequence of steps in which pixels are gradually merged to produce larger and larger regions. In each step, we consider pairs of adjacent regions and provide a probability measure to assess whether or not they should be included in the same segment. Our probabilistic formulation takes into account intensity and texture distributions in a local area around each region. It further incorporates priors based on the geometry of the regions. Finally, posteriors based on intensity and texture cues are combined using “a mixture of experts” formulation. This probabilistic approach is integrated into a graph coarsening scheme, providing a complete hierarchical segmentation of the image. The algorithm complexity is linear in the number of the image pixels and it requires almost no user-tuned parameters. In addition, we provide a novel evaluation scheme for image segmentation algorithms, attempting to avoid human semantic considerations that are out of scope for segmentation algorithms. Using this novel evaluation scheme, we test our method and provide a comparison to several existing segmentation algorithms.     

Pose estimation through cue integration: a neuroscience-inspired approach

The aim of this paper is to improve the skills of robotic systems in their interaction with nearby objects. The basic idea is to enhance visual estimation of objects in the world through the merging of different visual estimators of the same stimuli. A neuroscience-inspired model of stereoptic and perspective orientation estimators, merged according to different criteria, is implemented on a robotic setup and tested in different conditions. Experimental results suggest that the integration of multiple monocular and binocular cues can make robot sensory systems more reliable and versatile. The same results, compared with simulations and data from human studies, show that the model is able to reproduce some well-recognized neuropsychological effects.     

On deformable models for visual pattern recognition

This paper reviews model-based methods for non-rigid shape recognition. These methods model, match and classify non-rigid shapes, which are generally problematic for conventational algorithms using rigid models. Issues including model representation, optimization criteria formulation, model matching, and classification are examined in detail with the objective to provide interested researchers a roadmap for exploring the field. This paper emphasizes on 2D deformable models. Their potential applications and future research directions, particularly on deformable pattern classification, are discussed.     

Metamorphs: deformable shape and appearance models

This paper presents a new deformable modeling strategy aimed at integrating shape and appearance in a unified space. If we think traditional deformable models as active contours or evolving curve fronts, the new deformable shape and appearance models we propose are deforming disks or volumes. Each model has not only boundary shape but also interior appearance. The model shape is implicitly embedded in a higher dimensional space of distance transforms, thus represented by a distance map image. In this way, both shape and appearance of the model are defined in the pixel space. A common deformation scheme, the Free Form Deformations (FFD), parameterizes warping deformations of the volumetric space in which the model is embedded in, hence deforming both model boundary and interior simultaneously.     

Visual surveillance using deformable models of vehicles

This paper presents recent developments to a vision-based traffic surveillance system which relies extensively on the use of geometrical and scene context. Firstly, a highly parametrised 3-D model is reported, able to adopt the shape of a wide variety of different classes of vehicle (e.g. cars, vans, buses etc.), and its subsequent specialisation to a generic car class which accounts for commonly encountered types of car (including saloon, batchback and estate cars). Sample data collected from video images, by means of an interactive tool, have been subjected to principal component analysis (PCA) to define a deformable model having 6 degrees of freedom.

Secondly, a new pose refinement technique using “active” models is described, able to recover both the pose of a rigid object, and the structure of a deformable model; an assessment of its performance is examined in comparison with previously reported “passive” model-based techniques in the context of traffic surveillance. The new method is more stable, and requires fewer iterations, especially when the number of free parameters increases, but shows somewhat poorer convergence.

Typical applications for this work include robot surveillance and navigation tasks.     

Animation of deformable models using implicit surfaces

The paper presents a general approach for designing and animating complex deformable models with implicit surfaces. Implicit surfaces are introduced as an extra layer coating any kind of structure that moves and deforms over time. Offering a compact definition of a smooth surface around an object, they provide an efficient collision detection mechanism. The implicit layer deforms in order to generate exact contact surfaces between colliding bodies. A simple physically based model approximating elastic behavior is then used for computing collision response. The implicit formulation also eases the control of the object's volume with a new method based on local controllers. We present two different applications that illustrate the benefits of these techniques. First, the animation of simple characters made of articulated skeletons coated with implicit flesh exploits the compactness and enhanced control of the model. The second builds on the specific properties of implicit surfaces for modeling soft inelastic substances capable of separation and fusion that maintain a constant volume when animated     

Image-based collision detection for deformable cloth models

Modeling the natural interaction of cloth and garments with objects in a 3D environment is currently one of the most computationally demanding tasks. These highly deformable materials are subject to a very large number of contact points in the proximity of other moving objects. Furthermore, cloth objects often fold, roll, and drape within themselves, generating a large number of self-collision areas. The interactive requirements of 3D games and physically driven virtual environments make the cloth collisions and self-collision computations more challenging. By exploiting mathematically well-defined smoothness conditions over smaller patches of deformable surfaces and resorting to image-based collision detection tests, we developed an efficient collision detection method that achieves interactive rates while tracking self-interactions in highly deformable surfaces consisting of a large number of elements. The method makes use of a novel technique for dynamically generating a hierarchy of cloth bounding boxes in order to perform object-level culling and image-based intersection tests using conventional graphics hardware support. An efficient backward voxel-based AABB hierarchy method is proposed to handle deformable surfaces which are highly compressed.     

Numerical integration in logistic-normal models

Marginal maximum likelihood estimation is commonly used to estimate logistic-normal models. In this approach, the contribution of random effects to the likelihood is represented as an intractable integral over their distribution. Thus, numerical methods such as Gauss-Hermite quadrature (GH) are needed. However, as the dimensionality increases, the number of quadrature points becomes rapidly too high. A possible solution can be found among the Quasi-Monte Carlo (QMC) methods, because these techniques yield quite good approximations for high-dimensional integrals with a much lower number of points, chosen for their optimal location. A comparison between three integration methods for logistic-normal models: GH, QMC, and full Monte Carlo integration (MC) is presented. It turns out that, under certain conditions, the QMC and MC method perform better than the GH in terms of accuracy and computing time.     

Parallelized deformable part models with effective hypothesis pruning

Computational Visual Media - As a typical machine-learning based detection technique, deformable part models (DPM) achieve great success in detecting complex object categories. The heavy...