Non-Rigid Stereo Factorization

In this paper we address the problem of recovering 3D non-rigid structure from a sequence of images taken with a stereo pair. We have extended existing non-rigid factorization algorithms to the stereo camera case and presented an algorithm to decompose the measurement matrix into the motion of the left and right cameras and the 3D shape, represented as a linear combination of basis-shapes. The added constraints in the stereo camera case are that both cameras are viewing the same structure and that the relative orientation between both cameras is fixed. Our focus in this paper is on the recovery of flexible 3D shape rather than on the correspondence problem. We propose a method to compute reliable 3D models of deformable structure from stereo images. Our experiments with real data show that improved reconstructions can be achieved using this method. The algorithm includes a non-linear optimization step that minimizes image reprojection error and imposes the correct structure to the motion matrix by choosing an appropriate parameterization. We show that 3D shape and motion estimates can be successfully disambiguated after bundle adjustment and demonstrate this on synthetic and real image sequences. While this optimization step is proposed for the stereo camera case, it can be readily applied to the case of non-rigid structure recovery using a monocular video sequence. Electronic supplementary material Electronic supplementary material is available for this article at and accessible for authorised users.     

Generalized Gradients: Priors on Minimization Flows

This paper tackles an important aspect of the variational problem underlying active contours: optimization by gradient flows. Classically, the definition of a gradient depends directly on the choice of an inner product structure. This consideration is largely absent from the active contours literature. Most authors, explicitely or implicitely, assume that the space of admissible deformations is ruled by the canonical L ² inner product. The classical gradient flows reported in the literature are relative to this particular choice. Here, we investigate the relevance of using (i) other inner products, yielding other gradient descents, and (ii) other minimizing flows not deriving from any inner product. In particular, we show how to induce different degrees of spatial consistency into the minimizing flow, in order to decrease the probability of getting trapped into irrelevant local minima. We report numerical experiments indicating that the sensitivity of the active contours method to initial conditions, which seriously limits its applicability and efficiency, is alleviated by our application-specific spatially coherent minimizing flows. We show that the choice of the inner product can be seen as a prior on the deformation fields and we present an extension of the definition of the gradient toward more general priors. Electronic supplementary material  Electronic supplementary material is available for this article at     

3D Structure Recovery and Unwarping of Surfaces Applicable to Planes

The deformation of applicable surfaces such as sheets of paper satisfies the differential geometric constraints of isometry (lengths and areas are conserved) and vanishing Gaussian curvature. We show that these constraints lead to a closed set of equations that allow recovery of the full geometric structure from a single image of the surface and knowledge of its undeformed shape. We show that these partial differential equations can be reduced to the Hopf equation that arises in non-linear wave propagation, and deformations of the paper can be interpreted in terms of the characteristics of this equation. A new exact integration of these equations is developed that relates the 3-D structure of the applicable surface to an image. The solution is tested by comparison with particular exact solutions. We present results for both the forward and the inverse 3D structure recovery problem.     

Simulations of extensional flow in microrheometric devices

We present a detailed numerical study of the flow of a Newtonian fluid through microrheometric devices featuring a sudden contraction–expansion. This flow configuration is typically used to generate extensional deformations and high strain rates. The excess pressure drop resulting from the converging and diverging flow is an important dynamic measure to quantify if the device is intended to be used as a microfluidic extensional rheometer. To explore this idea, we examine the effect of the contraction length, aspect ratio and Reynolds number on the flow kinematics and resulting pressure field. Analysis of the computed velocity and pressure fields show that, for typical experimental conditions used in microfluidic devices, the steady flow is highly three-dimensional with open spiraling vortical structures in the stagnant corner regions. The numerical simulations of the local kinematics and global pressure drop are in good agreement with experimental results. The device aspect ratio is shown to have a strong impact on the flow and consequently on the excess pressure drop, which is quantified in terms of the dimensionless Couette and Bagley correction factors. We suggest an approach for calculating the Bagley correction which may be especially appropriate for planar microchannels. Electronic supplementary material  The online version of this article (doi:) contains supplementary material, which is available to authorized users.     

Shape-From-Silhouette Across Time Part II: Applications to Human Modeling and Markerless Motion Tracking

In Part I of this paper we developed the theory and algorithms for performing Shape-From-Silhouette (SFS) across time. In this second part, we show how our temporal SFS algorithms can be used in the applications of human modeling and markerless motion tracking. First we build a system to acquire human kinematic models consisting of precise shape (constructed using the temporal SFS algorithm for rigid objects), joint locations, and body part segmentation (estimated using the temporal SFS algorithm for articulated objects). Once the kinematic models have been built, we show how they can be used to track the motion of the person in new video sequences. This marker-less tracking algorithm is based on the Visual Hull alignment algorithm used in both temporal SFS algorithms and utilizes both geometric (silhouette) and photometric (color) information.Electronic supplementary material Electronic supplementary material is available for this article at and accessible for authorised users.     

Kernel Density Estimation and Intrinsic Alignment for Shape Priors in Level Set Segmentation

In this paper, we make two contributions to the field of level set based image segmentation. Firstly, we propose shape dissimilarity measures on the space of level set functions which are analytically invariant under the action of certain transformation groups. The invariance is obtained by an intrinsic registration of the evolving level set function. In contrast to existing approaches to invariance in the level set framework, this closed-form solution removes the need to iteratively optimize explicit pose parameters. The resulting shape gradient is more accurate in that it takes into account the effect of boundary variation on the object’s pose. Secondly, based on these invariant shape dissimilarity measures, we propose a statistical shape prior which allows to accurately encode multiple fairly distinct training shapes. This prior constitutes an extension of kernel density estimators to the level set domain. In contrast to the commonly employed Gaussian distribution, such nonparametric density estimators are suited to model aribtrary distributions. We demonstrate the advantages of this multi-modal shape prior applied to the segmentation and tracking of a partially occluded walking person in a video sequence, and on the segmentation of the left ventricle in cardiac ultrasound images. We give quantitative results on segmentation accuracy and on the dependency of segmentation results on the number of training shapes. Electronic supplementary material Electronic supplementary material is available for this article at and accessible for authorised users.     

Space Deformations, Surface Deformations and the Opportunities In-Between

In recent years we have witnessed a large interest in surface deformation techniques. This has been a reaction that can be attributed to the ability to develop techniques which are detail-preserving. Space deformation techniques, on the other hand, received less attention, but nevertheless they have many advantages over surface-based techniques. This paper explores the potential of these two approaches to deformation and discusses the opportunities that the fusion of the two may lead to. Electronic Supplementary Material  The online version of this article (doi:) contains supplementary material, which is available to authorized users.     

Local surface shape estimation of 3-D textured surfaces usingGaussian Markov random fields and stereo windows

The problem of extracting the local shape information of a 3-D texture surface from a single 2-D image by tracking the perceived systematic deformations the texture undergoes by virtue of being present on a 3-D surface and by virtue of being imaged is examined. The surfaces of interest are planar and developable surfaces. The textured objects are viewed as originating by laying a rubber planar sheet with a homogeneous parent texture on it onto the objects. The homogeneous planar parent texture is modeled by a stationary Gaussian Markov random field (GMRF). A probability distribution function for the texture data obtained by projecting the planar parent texture under a linear camera model is derived, which is an explicit function of the parent GMRF parameters, the surface shape parameters. and the camera geometry. The surface shape parameter estimation is posed as a maximum likelihood estimation problem. A stereo-windows concept is introduced to obtain a unique and consistent parent texture from the image data that, under appropriate transformations, yields the observed texture in the image. The theory is substantiated by experiments on synthesized as well as real images of textured surfaces     

Multi-level differential surface representation based on local transformations

One crucial issue of multi-resolution surface representations is how to effectively record and reconstruct geometric details among surface levels. Standard multi-resolution techniques encode details directly as local displacements in the vertices, and may produce unplausible results when the base level endures large deformations. In this paper we propose an alternative detail representation and reconstruction scheme, based on local transformations on a per-triangle basis. While more storage is required, recording details as local transformations favors global coupling of geometric details and allows for large-scale surface manipulations. By modeling the scale components of the surface modifications as a set of deforming factors, we achieved detail-preserving reconstruction results naturally under very large deformations. Comprehensive experimental results verify the efficiency and robustness of our approach. Electronic supplementary material Supplementary material is available in the online version of this article at and is accessible for authorized users.     

Edge-Oriented Spatial Interpolation for Error Concealment of Consecutive Blocks

This paper proposes a low-complexity spatial-domain error concealment （EC） algorithm for recovering consecutive blocks error in still images or intra-coded （I） frames of video sequences. The proposed algorithm works with the following steps. Firstly the Sobel operator is performed on the top and bottom adjacent pixels to detect the most probable edge direction of current block area. After that one-dimensional （1-D） matching is used on the available block boundaries. Displacement between edge direction candidate and most probable edge direction is taken into consideration as an important factor to improve stability of 1-D boundary matching. Then the corrupted pixels are recovered by linear weighting interpolation along the estimated edge direction. Finally the interpolated values are merged to get last recovered picture. Simulation results demonstrate that the proposed algorithms obtain good subjective quality and higher PSNR than the methods in literatures for most images.     

Streaming 3D deforming surfaces with dynamic resolution control

Real‐time streaming of shape deformations in a shared distributed virtual environment is a challenging task due to the difficulty of transmitting large amounts of 3D animation data to multiple receiving parties at a high frame rate. In this paper, we present a framework for streaming 3D shape deformations, which allows shapes with multi‐resolutions to share the same deformations simultaneously in real time. The geometry and motion of deforming mesh or point‐sampled surfaces are compactly encoded, transmitted, and reconstructed using the spectra of the manifold harmonics. A receiver‐based multi‐resolution surface reconstruction approach is introduced, which allows deforming shapes to switch smoothly between continuous multi‐resolutions. On the basis of this dynamic reconstruction scheme, a frame rate control algorithm is further proposed to achieve rendering at interactive rates. We also demonstrate an efficient interpolation‐based strategy to reduce computing of deformation. The experiments conducted on both mesh and point‐sampled surfaces show that our approach achieves efficient performance even if deformations of complex 3D surfaces are streamed. Copyright © 2013 John Wiley & Sons, Ltd.     

Perception of 3-D surfaces from 2-D contours

Inference of 3-D shape from 2-D contours in a single image is an important problem in machine vision. The authors survey classes of techniques proposed in the past and provide a critical analysis. They show that two kinds of symmetries in figures, which are known as parallel and skew symmetries, give significant information about surface shape for a variety of objects. They derive the constraints imposed by these symmetries and show how to use them to infer 3-D shape. They also discuss the zero Gaussian curvature (ZGC) surfaces in depth and show results on the recovery of surface orientation for various ZGC surfaces     

A Projective Framework for Polyhedral Mesh Modelling

We present a novel framework for polyhedral mesh editing with face‐based projective maps that preserves planarity by definition. Such meshes are essential in the field of architectural design and rationalization. By using homogeneous coordinates to describe vertices, we can parametrize the entire shape space of planar‐preserving deformations with bilinear equations. The generality of this space allows for polyhedral geometric processing methods to be conducted with ease. We demonstrate its usefulness in planar‐quadrilateral mesh subdivision, a resulting multi‐resolution editing algorithm, and novel shape‐space exploration with prescribed transformations. Furthermore, we show that our shape space is a discretization of a continuous space of conjugate‐preserving projective transformation fields on surfaces. Our shape space directly addresses planar‐quad meshes, on which we put a focus, and we further show that our framework naturally extends to meshes with faces of more than four vertices as well.     

The Monogenic Scale Space on a Rectangular Domain and its Features

In this paper we present a novel method to implement the monogenic scale space on a rectangular domain. The monogenic scale space is a vector valued scale space based on the Poisson scale space, which establishes a sophisticated alternative to the Gaussian scale space. Previous implementations of the monogenic scale space are Fourier transform based, and therefore suffer from the implicit periodicity in case of finite domains.The features of the monogenic scale space, including local amplitude, local phase, local orientation, local frequency, and phase congruency, are much easier to interpret in terms of image features evolving through scale than in the Gaussian case. Furthermore, applying results from harmonic analysis, relations between the features are obtained which improve the understanding of image analysis. As applications, we present a very simple but still accurate approach to image reconstruction from local amplitude and local phase and a method for extracting the evolution of lines and edges through scale.Electronic supplementary material Electronic supplementary material is available for this article at and accessible for authorised users.First online version published in June, 2005This work has been supported by DFG Grant FE 583/1-2.     

Estimation of Error in Curvature Computation on Multi-Scale Free-Form Surfaces

A novel technique for multi-scale curvature computation on a free-form 3-D surface is presented. This is achieved by convolving local parametrisations of the surface with 2-D Gaussian filters iteratively. In our technique, semigeodesic coordinates are constructed at each vertex of the mesh. Smoothing results are shown for 3-D surfaces with different shapes indicating that surface noise is eliminated and surface details are removed gradually. A number of evolution properties of 3-D surfaces are described. Next, the surface Gaussian and mean curvature values are estimated accurately at multiple scales which are then mapped to colours and displayed directly on the surface. The performance of the technique when selecting different directions as an arbitrary direction for the geodesic at each vertex are also presented. The results indicate that the error observed for the estimation of Gaussian and mean curvatures is quite low after only one iteration. Furthermore, as the surface is smoothed iteratively, the error is further reduced. The results also show that the estimation error of Gaussian curvature is less than that of mean curvature. Our experiments demonstrate that estimation of smoothed surface curvatures are very accurate and not affected by the arbitrary direction of the first geodesic line when constructing semigeodesic coordinates. Our technique is independent of the underlying triangulation and is also more efficient than volumetric diffusion techniques since 2-D rather than 3-D convolutions are employed. Finally, the method presented here is a generalisation of the Curvature Scale Space method for 2-D contours. The CSS method has outperformed comparable techniques within the MPEG-7 evaluation framework. As a result, it has been selected for inclusion in the MPEG-7 package of standards.     

Spectral volume rendering using GPU-based raycasting

Traditional volume rendering does not incorporate a number of optical properties that are typically observed for semi-transparent materials, such as glass or water, in the real world. Therefore, we have extended GPU-based raycasting to spectral volume rendering based on the Kubelka–Munk theory for light propagation in parallel colorant layers of a turbid medium. This allows us to demonstrate the effects of selective absorption and dispersion in refractive materials, by generating volume renderings using real physical optical properties. We show that this extended volume rendering technique can be easily incorporated into a flexible framework for GPU-based volume raycasting. Our implementation shows a promising performance for a number of real data sets. In particular, we obtain up to 100 times the performance of a comparable CPU implementation. Electronic supplementary material Supplementary material is available in the online version of this article at and is accessible for authorized users.     

Barycentric Coordinates on Surfaces

This paper introduces a method for defining and efficiently computing barycentric coordinates with respect to polygons on general surfaces. Our construction is geared towards injective polygons (polygons that can be enclosed in a metric ball of an appropriate size) and is based on replacing the linear precision property of planar coordinates by a requirement in terms of center of mass, and generalizing this requirement to the surface setting. We show that the resulting surface barycentric coordinates can be computed using planar barycentric coordinates with respect to a polygon in the tangent plane. We prove theoretically that the surface coordinates properly generalize the planar coordinates and carry some of their useful properties such as unique reconstruction of a point given its coordinates, uniqueness for triangles, edge linearity, similarity invariance, and smoothness; in addition, these coordinates are insensitive to isometric deformations and can be used to reconstruct isometries. We show empirically that surface coordinates are shape‐aware with consistent gross behavior across different surfaces, are well‐behaved for different polygon types/locations on variety of surface forms, and that they are fast to compute. Finally, we demonstrate effectiveness of surface coordinates for interpolation, decal mapping, and correspondence refinement.