An Intrinsic Framework for Analysis of Facial Surfaces

A statistical analysis of shapes of facial surfaces can play an important role in biometric authentication and other face-related applications. The main difficulty in developing such an analysis comes from the lack of a canonical system to represent and compare all facial surfaces. This paper suggests a specific, yet natural, coordinate system on facial surfaces, that enables comparisons of their shapes. Here a facial surface is represented as an indexed collection of closed curves, called facial curves, that are level curves of a surface distance function from the tip of the nose. Defining the space of all such representations of face, this paper studies its differential geometry and endows it with a Riemannian metric. It presents numerical techniques for computing geodesic paths between facial surfaces in that space. This Riemannian framework is then used to: (i) compute distances between faces to quantify differences in their shapes, (ii) find optimal deformations between faces, and (iii) define and compute average of a given set of faces. Experimental results generated using laser-scanned faces are presented to demonstrate these ideas.     

A Continuum Mechanical Approach to Geodesics in Shape Space

In this paper concepts from continuum mechanics are used to define geodesic paths in the space of shapes, where shapes are implicitly described as boundary contours of objects. The proposed shape metric is derived from a continuum mechanical notion of viscous dissipation. A geodesic path is defined as the family of shapes such that the total amount of viscous dissipation caused by an optimal material transport along the path is minimized. The approach can easily be generalized to shapes given as segment contours of multi-labeled images and to geodesic paths between partially occluded objects. The proposed computational framework for finding such a minimizer is based on the time discretization of a geodesic path as a sequence of pairwise matching problems, which is strictly invariant with respect to rigid body motions and ensures a 1–1 correspondence along the induced flow in shape space. When decreasing the time step size, the proposed model leads to the minimization of the actual geodesic length, where the Hessian of the pairwise matching energy reflects the chosen Riemannian metric on the underlying shape space. If the constraint of pairwise shape correspondence is replaced by the volume of the shape mismatch as a penalty functional, one obtains for decreasing time step size an optical flow term controlling the transport of the shape by the underlying motion field. The method is implemented via a level set representation of shapes, and a finite element approximation is employed as spatial discretization both for the pairwise matching deformations and for the level set representations. The numerical relaxation of the energy is performed via an efficient multi-scale procedure in space and time. Various examples for 2D and 3D shapes underline the effectiveness and robustness of the proposed approach.     

A Riemannian Framework for Tensor Computing

Tensors are nowadays a common source of geometric information. In this paper, we propose to endow the tensor space with an affine-invariant Riemannian metric. We demonstrate that it leads to strong theoretical properties: the cone of positive definite symmetric matrices is replaced by a regular and complete manifold without boundaries (null eigenvalues are at the infinity), the geodesic between two tensors and the mean of a set of tensors are uniquely defined, etc. We have previously shown that the Riemannian metric provides a powerful framework for generalizing statistics to manifolds. In this paper, we show that it is also possible to generalize to tensor fields many important geometric data processing algorithms such as interpolation, filtering, diffusion and restoration of missing data. For instance, most interpolation and Gaussian filtering schemes can be tackled efficiently through a weighted mean computation. Linear and anisotropic diffusion schemes can be adapted to our Riemannian framework, through partial differential evolution equations, provided that the metric of the tensor space is taken into account. For that purpose, we provide intrinsic numerical schemes to compute the gradient and Laplace-Beltrami operators. Finally, to enforce the fidelity to the data (either sparsely distributed tensors or complete tensors fields) we propose least-squares criteria based on our invariant Riemannian distance which are particularly simple and efficient to solve.     

Elastic shapes models for improving segmentation of object boundaries in synthetic aperture sonar images

We present a variational framework for naturally incorporating prior shape knowledge in guidance of active contours for boundary extraction in images. This framework is especially suitable for images collected outside the visible spectrum, where boundary estimation is difficult due to low contrast, low resolution, and presence of noise and clutter. Accordingly, we illustrate this approach using the segmentation of various objects in synthetic aperture sonar (SAS) images of underwater terrains. We use elastic shape analysis of planar curves in which the shapes are considered as elements of a quotient space of an infinite dimensional, non-linear Riemannian manifold. Using geodesic paths under the elastic Riemannian metric, one computes sample mean and covariances of training shapes in each classes and derives statistical models for capturing class-specific shape variability. These models are then used as shape priors in a variational setting to solve for Bayesian estimation of desired contours as follows. In traditional active contour models curves are driven towards minimum of an energy composed of image and smoothing terms. We introduce an additional shape term based on shape models of relevant shape classes. The minimization of this total energy, using iterated gradient-based updates of curves, leads to an improved segmentation of object boundaries. This is demonstrated using a number of shape classes in two large SAS image datasets.     

2D-Shape Analysis Using Conformal Mapping

The study of 2D shapes and their similarities is a central problem in the field of vision. It arises in particular from the task of classifying and recognizing objects from their observed silhouette. Defining natural distances between 2D shapes creates a metric space of shapes, whose mathematical structure is inherently relevant to the classification task. One intriguing metric space comes from using conformal mappings of 2D shapes into each other, via the theory of Teichmüller spaces. In this space every simple closed curve in the plane (a “shape”) is represented by a ‘fingerprint’ which is a diffeomorphism of the unit circle to itself (a differentiable and invertible, periodic function). More precisely, every shape defines to a unique equivalence class of such diffeomorphisms up to right multiplication by a Möbius map. The fingerprint does not change if the shape is varied by translations and scaling and any such equivalence class comes from some shape. This coset space, equipped with the infinitesimal Weil-Petersson (WP) Riemannian norm is a metric space. In this space, the shortest path between each two shapes is unique, and is given by a geodesic connecting them. Their distance from each other is given by integrating the WP-norm along that geodesic. In this paper we concentrate on solving the “welding” problem of “sewing” together conformally the interior and exterior of the unit circle, glued on the unit circle by a given diffeomorphism, to obtain the unique 2D shape associated with this diffeomorphism. This will allow us to go back and forth between 2D shapes and their representing diffeomorphisms in this “space of shapes”. We then present an efficient method for computing the unique shortest path, the geodesic of shape morphing between each two end-point shapes. The group of diffeomorphisms of S¹ acts as a group of isometries on the space of shapes and we show how this can be used to define shape transformations, like for instance ‘adding a protruding limb’ to any shape.     

Overview of the Geometries of Shape Spaces and Diffeomorphism Groups

This article provides an overview of various notions of shape spaces, including the space of parametrized and unparametrized curves, the space of immersions, the diffeomorphism group and the space of Riemannian metrics. We discuss the Riemannian metrics that can be defined thereon, and what is known about the properties of these metrics. We put particular emphasis on the induced geodesic distance, the geodesic equation and its well-posedness, geodesic and metric completeness and properties of the curvature.     

Practical Anisotropic Geodesy

The computation of intrinsic, geodesic distances and geodesic paths on surfaces is a fundamental low‐level building block in countless Computer Graphics and Geometry Processing applications. This demand led to the development of numerous algorithms – some for the exact, others for the approximative computation, some focussing on speed, others providing strict guarantees. Most of these methods are designed for computing distances according to the standard Riemannian metric induced by the surface's embedding in Euclidean space. Generalization to other, especially anisotropic, metrics – which more recently gained interest in several application areas – is not rarely hampered by fundamental problems. We explore and discuss possibilities for the generalization and extension of well‐known methods to the anisotropic case, evaluate their relative performance in terms of accuracy and speed, and propose a novel algorithm, the Short‐Term Vector Dijkstra. This algorithm is strikingly simple to implement and proves to provide practical accuracy at a higher speed than generalized previous methods.     

Discrete surface Ricci flow

This work introduces a unified framework for discrete surface Ricci flow algorithms, including spherical, Euclidean, and hyperbolic Ricci flows, which can design Riemannian metrics on surfaces with arbitrary topologies by user-defined Gaussian curvatures. Furthermore, the target metrics are conformal (angle-preserving) to the original metrics. A Ricci flow conformally deforms the Riemannian metric on a surface according to its induced curvature, such that the curvature evolves like a heat diffusion process. Eventually, the curvature becomes the user defined curvature. Discrete Ricci flow algorithms are based on a variational framework. Given a mesh, all possible metrics form a linear space, and all possible curvatures form a convex polytope. The Ricci energy is defined on the metric space, which reaches its minimum at the desired metric. The Ricci flow is the negative gradient flow of the Ricci energy. Furthermore, the Ricci energy can be optimized using Newton's method more efficiently. Discrete Ricci flow algorithms are rigorous and efficient. Our experimental results demonstrate the efficiency, accuracy and flexibility of the algorithms. They have the potential for a wide range of applications in graphics, geometric modeling, and medical imaging. We demonstrate their practical values by global surface parameterizations.     

Shape analysis of local facial patches for 3D facial expression recognition

In this paper we address the problem of 3D facial expression recognition. We propose a local geometric shape analysis of facial surfaces coupled with machine learning techniques for expression classification. A computation of the length of the geodesic path between corresponding patches, using a Riemannian framework, in a shape space provides a quantitative information about their similarities. These measures are then used as inputs to several classification methods. The experimental results demonstrate the effectiveness of the proposed approach. Using multiboosting and support vector machines (SVM) classifiers, we achieved 98.81% and 97.75% recognition average rates, respectively, for recognition of the six prototypical facial expressions on BU-3DFE database. A comparative study using the same experimental setting shows that the suggested approach outperforms previous work.     

Constant cusp height tool paths as geodesic parallels on an abstract Riemannian manifold

In free-form surface milling, cusps on a part surface need to be regulated. They should be small enough for precision purposes. On the other hand, we should maintain high enough cusps so as not to waste effort making unnecessary cuts. A widely accepted practice is to maintain a constant cusp height over the surface. This paper introduces a new approach to generating constant cusp height tool paths. First, we define a Riemannian manifold by assigning a new metric to a part surface without embedding. This new metric is constructed from the curvature tensors of a part and a tool surface, which we refer to as a cusp-metric. Then, we construct geodesic parallels on the new Riemannian manifold. We prove that a selection from such a family of geodesic parallels constitutes a “rational” approximation of accurate constant cusp height tool paths.     

Tools in the Riemannian geometry of quantum computation

An introduction is first given of recent developments in the Riemannian geometry of quantum computation in which the quantum evolution is represented in the tangent space manifold of the special unitary unimodular group for n qubits. The Riemannian right-invariant metric, connection, curvature, geodesic equation for minimal complexity quantum circuits, Jacobi equation, and the lifted Jacobi equation for varying penalty parameter are reviewed. Sharpened tools for calculating the geodesic derivative are presented. The geodesic derivative may facilitate the numerical investigation of conjugate points and the global characteristics of geodesic paths in the group manifold, the determination of optimal quantum circuits for carrying out a quantum computation, and the determination of the complexity of particular quantum algorithms.     

Exploring the Geometry of the Space of Shells

We prove both in the smooth and discrete setting that the Hessian of an elastic deformation energy results in a proper Riemannian metric on the space of shells (modulo rigid body motions). Based on this foundation we develop a time‐ and space‐discrete geodesic calculus. In particular we show how to shoot geodesics with prescribed initial data, and we give a construction for parallel transport in shell space. This enables, for example, natural extrapolation of paths in shell space and transfer of large nonlinear deformations from one shell to another with applications in animation, geometric, and physical modeling. Finally, we examine some aspects of curvature on shell space.     

Histogram-based embedding for learning on statistical manifolds

A novel binning and learning framework is presented for analyzing and applying large data sets that have no explicit knowledge of distribution parameterizations, and can only be assumed generated by the underlying probability density functions (PDFs) lying on a nonparametric statistical manifold. For models’ discretization, the uniform sampling-based data space partition is used to bin flat-distributed data sets, while the quantile-based binning is adopted for complex distributed data sets to reduce the number of under-smoothed bins in histograms on average. The compactified histogram embedding is designed so that the Fisher–Riemannian structured multinomial manifold is compatible to the intrinsic geometry of nonparametric statistical manifold, providing a computationally efficient model space for information distance calculation between binned distributions. In particular, without considering histogramming in optimal bin number, we utilize multiple random partitions on data space to embed the associated data sets onto a product multinomial manifold to integrate the complementary bin information with an information metric designed by factor geodesic distances, further alleviating the effect of over-smoothing problem. Using the equipped metric on the embedded submanifold, we improve classical manifold learning and dimension estimation algorithms in metric-adaptive versions to facilitate lower-dimensional Euclidean embedding. The effectiveness of our method is verified by visualization of data sets drawn from known manifolds, visualization and recognition on a subset of ALOI object database, and Gabor feature-based face recognition on the FERET database.     

Affine-invariant geodesic geometry of deformable 3D shapes

Natural objects can be subject to various transformations yet still preserve properties that we refer to as invariants. Here, we use definitions of affine-invariant arclength for surfaces in R³ in order to extend the set of existing non-rigid shape analysis tools. We show that by re-defining the surface metric as its equi-affine version, the surface with its modified metric tensor can be treated as a canonical Euclidean object on which most classical Euclidean processing and analysis tools can be applied. The new definition of a metric is used to extend the fast marching method technique for computing geodesic distances on surfaces, where now, the distances are defined with respect to an affine-invariant arclength. Applications of the proposed framework demonstrate its invariance, efficiency, and accuracy in shape analysis.     

Registration of Anatomical Images Using Paths of Diffeomorphisms Parameterized with Stationary Vector Field Flows

Computational Anatomy aims for the study of variability in anatomical structures from images. Variability is encoded by the spatial transformations existing between anatomical images and a template selected as reference. In the absence of a more justified model for inter-subject variability, transformations are considered to belong to a convenient family of diffeomorphisms which provides a suitable mathematical setting for the analysis of anatomical variability. One of the proposed paradigms for diffeomorphic registration is the Large Deformation Diffeomorphic Metric Mapping (LDDMM). In this framework, transformations are characterized as end points of paths parameterized by time-varying flows of vector fields defined on the tangent space of a Riemannian manifold of diffeomorphisms and computed from the solution of the non-stationary transport equation associated to these flows. With this characterization, optimization in LDDMM is performed on the space of non-stationary vector field flows resulting into a time and memory consuming algorithm. Recently, an alternative characterization of paths of diffeomorphisms based on constant-time flows of vector fields has been proposed in the literature. With this parameterization, diffeomorphisms constitute solutions of stationary ODEs. In this article, the stationary parameterization is included for diffeomorphic registration in the LDDMM framework. We formulate the variational problem related to this registration scenario and derive the associated Euler-Lagrange equations. Moreover, the performance of the non-stationary vs the stationary parameterizations in real and simulated 3D-MRI brain datasets is evaluated. Compared to the non-stationary parameterization, our proposal provides similar results in terms of image matching and local differences between the diffeomorphic transformations while drastically reducing memory and time requirements.     

On advances in differential-geometric approaches for 2D and 3D shape analyses and activity recognition

In this paper we summarize recent advances in shape analysis and shape-based activity recognition problems with a focus on techniques that use tools from differential geometry and statistics. We start with general goals and challenges faced in shape analysis, followed by a summary of the basic ideas, strengths and limitations, and applications of different mathematical representations used in shape analyses of 2D and 3D objects. These representations include point sets, curves, surfaces, level sets, deformable templates, medial representations, and other feature-based methods. We discuss some common choices of Riemannian metrics and computational tools used for evaluating geodesic paths and geodesic distances for several of these shape representations. Then, we study the use of Riemannian frameworks in statistical modeling of variability within shape classes.Next, we turn to models and algorithms for activity analysis from various perspectives. We discuss how mathematical representations for human shape and its temporal evolutions in videos lead to analyses over certain special manifolds. We discuss the various choices of shape features, and parametric and non-parametric models for shape evolution, and how these choices lead to appropriate manifold-valued constraints. We discuss applications of these methods in gait-based biometrics, action recognition, and video summarization and indexing.For reader convenience, we also provide a short overview of the relevant tools from geometry and statistics on manifolds in the Appendix.     

Elastic Shape Models for Face Analysis Using Curvilinear Coordinates

This paper studies the problem of analyzing variability in shapes of facial surfaces using a Riemannian framework, a fundamental approach that allows for joint matchings, comparisons, and deformations of faces under a chosen metric. The starting point is to impose a curvilinear coordinate system, named the Darcyan coordinate system, on facial surfaces; it is based on the level curves of the surface distance function measured from the tip of the nose. Each facial surface is now represented as an indexed collection of these level curves. The task of finding optimal deformations, or geodesic paths, between facial surfaces reduces to that of finding geodesics between level curves, which is accomplished using the theory of elastic shape analysis of 3D curves. The elastic framework allows for nonlinear matching between curves and between points across curves. The resulting geodesics between facial surfaces provide optimal elastic deformations between faces and an elastic metric for comparing facial shapes. We demonstrate this idea using examples from FSU face database.

Geodesic distance-weighted shape vector image diffusion

This paper presents a novel and efficient surface matching and visualization framework through the geodesic distance-weighted shape vector image diffusion. Based on conformal geometry, our approach can uniquely map a 3D surface to a canonical rectangular domain and encode the shape characteristics (e.g., mean curvatures and conformal factors) of the surface in the 2D domain to construct a geodesic distance-weighted shape vector image, where the distances between sampling pixels are not uniform but the actual geodesic distances on the manifold. Through the novel geodesic distance-weighted shape vector image diffusion presented in this paper, we can create a multiscale diffusion space, in which the cross-scale extrema can be detected as the robust geometric features for the matching and registration of surfaces. Therefore, statistical analysis and visualization of surface properties across subjects become readily available. The experiments on scanned surface models show that our method is very robust for feature extraction and surface matching even under noise and resolution change. We have also applied the framework on the real 3D human neocortical surfaces, and demonstrated the excellent performance of our approach in statistical analysis and integrated visualization of the multimodality volumetric data over the shape vector image.     

Geometric accuracy analysis for discrete surface approximation

In geometric modeling and processing, computer graphics and computer vision, smooth surfaces are approximated by discrete triangular meshes reconstructed from sample points on the surfaces. A fundamental problem is to design rigorous algorithms to guarantee the geometric approximation accuracy by controlling the sampling density. This paper gives explicit formulae to the bounds of Hausdorff distance, normal distance and Riemannian metric distortion between the smooth surface and the discrete mesh in terms of principle curvature and the radii of geodesic circum-circle of the triangles. These formulae can be directly applied to design sampling density for data acquisitions and surface reconstructions. Furthermore, we prove that the meshes induced from the Delaunay triangulations of the dense samples on a smooth surface are convergent to the smooth surface under both Hausdorff distance and normal fields. The Riemannian metrics and the Laplace–Beltrami operators on the meshes are also convergent to those on the smooth surfaces. These theoretical results lay down the foundation for a broad class of reconstruction and approximation algorithms in geometric modeling and processing.Practical algorithms for approximating surface Delaunay triangulations are introduced based on global conformal surface parameterizations and planar Delaunay triangulations. Thorough experiments are conducted to support the theoretical results.