3D anatomical shape atlas construction using mesh quality preserved deformable models

3D anatomical shape atlas construction has been extensively studied in medical image analysis research, owing to its importance in model-based image segmentation, longitudinal studies and populational statistical analysis, etc. Among multiple steps of 3D shape atlas construction, establishing anatomical correspondences across subjects, i.e., surface registration, is probably the most critical but challenging one. Adaptive focus deformable model (AFDM) [1] was proposed to tackle this problem by exploiting cross-scale geometry characteristics of 3D anatomy surfaces. Although the effectiveness of AFDM has been proved in various studies, its performance is highly dependent on the quality of 3D surface meshes, which often degrades along with the iterations of deformable surface registration (the process of correspondence matching). In this paper, we propose a new framework for 3D anatomical shape atlas construction. Our method aims to robustly establish correspondences across different subjects and simultaneously generate high-quality surface meshes without removing shape details. Mathematically, a new energy term is embedded into the original energy function of AFDM to preserve surface mesh qualities during deformable surface matching. More specifically, we employ the Laplacian representation to encode shape details and smoothness constraints. An expectation–maximization style algorithm is designed to optimize multiple energy terms alternatively until convergence. We demonstrate the performance of our method via a set of diverse applications, including a population of sparse cardiac MRI slices with 2D labels, 3D high resolution CT cardiac images and rodent brain MRIs with multiple structures. The constructed shape atlases exhibit good mesh qualities and preserve fine shape details. The constructed shape atlases can further benefit other research topics such as segmentation and statistical analysis.     

Anomaly detection based on spatio-temporal sparse representation and visual attention analysis

In this paper, we proposed a unified framework for anomaly detection and localization in crowed scenes. For each video frame, we extract the spatio-temporal sparse features of 3D blocks and generate the saliency map using a block-based center-surround difference operator. Two sparse coding strategies including off-line long-term sparse representation and on-line short-term sparse representation are integrated within our framework. Abnormality of each candidate is measured using bottom-up saliency and top-down fixation inference and further used to classify the frames into normal and anomalous ones by a binary classifier. Local abnormal events are localized and segmented based on the saliency map. In the experiments, we compared our method against several state-of-the-art approaches on UCSD data set which is a widely used anomaly detection and localization benchmark. Our method outputs competitive results with near real-time processing speed compared to state-of-the-arts.     

Partwise cross-parameterization via nonregular convex hull domains

In this paper, we propose a novel partwise framework for cross-parameterization between 3D mesh models. Unlike most existing methods that use regular parameterization domains, our framework uses nonregular approximation domains to build the cross-parameterization. Once the nonregular approximation domains are constructed for 3D models, different (and complex) input shapes are transformed into similar (and simple) shapes, thus facilitating the cross-parameterization process. Specifically, a novel nonregular domain, the convex hull, is adopted to build shape correspondence. We first construct convex hulls for each part of the segmented model, and then adopt our convex-hull cross-parameterization method to generate compatible meshes. Our method exploits properties of the convex hull, e.g., good approximation ability and linear convex representation for interior vertices. After building an initial cross-parameterization via convex-hull domains, we use compatible remeshing algorithms to achieve an accurate approximation of the target geometry and to ensure a complete surface matching. Experimental results show that the compatible meshes constructed are well suited for shape blending and other geometric applications.     

On growth and formlets: Sparse multi-scale coding of planar shape

We propose a sparse representation of 2D planar shape through the composition of warping functions, termed formlets, localized in scale and space. Each formlet subjects the 2D space in which the shape is embedded to a localized isotropic radial deformation. By constraining these localized warping transformations to be diffeomorphisms, the topology of shape is preserved, and the set of simple closed curves is closed under any sequence of these warpings. A generative model based on a composition of formlets applied to an embryonic shape, e.g., an ellipse, has the advantage of synthesizing only those shapes that could correspond to the boundaries of physical objects. To compute the set of formlets that represent a given boundary, we demonstrate a greedy coarse-to-fine formlet pursuit algorithm that serves as a non-commutative generalization of matching pursuit for sparse approximations. We evaluate our method by pursuing partially occluded shapes, comparing performance against a contour-based sparse shape coding framework.     

Hierarchical Quad Meshing of 3D Scanned Surfaces

In this paper we present a novel method to reconstruct watertight quad meshes on scanned 3D geometry. There exist many different approaches to acquire 3D information from real world objects and sceneries. Resulting point clouds depict scanned surfaces as sparse sets of positional information. A common downside is the lack of normals, connectivity or topological adjacency data which makes it difficult to actually recover a meaningful surface. The concept described in this paper is designed to reconstruct a surface mesh despite all this missing information. Even when facing varying sample density, our algorithm is still guaranteed to produce watertight manifold meshes featuring quad faces only. The topology can be set‐up to follow superimposed regular structures or align naturally to the point cloud's shape. Our proposed approach is based on an initial divide and conquer subsampling procedure: Surface samples are clustered in meaningful neighborhoods as leafs of a kd‐tree. A representative sample of the surface neighborhood is determined for each leaf using a spherical surface approximation. The hierarchical structure of the binary tree is utilized to construct a basic set of loose tiles and to interconnect them. As a final step, missing parts of the now coherent tile structure are filled up with an incremental algorithm for locally optimal gap closure. Disfigured or concave faces in the resulting mesh can be removed with a constrained smoothing operator.     

FARM: Functional Automatic Registration Method for 3D Human Bodies

We introduce a new method for non-rigid registration of 3D human shapes. Our proposed pipeline builds upon a given parametric model of the human, and makes use of the functional map representation for encoding and inferring shape maps throughout the registration process. This combination endows our method with robustness to a large variety of nuisances observed in practical settings, including non-isometric transformations, downsampling, topological noise and occlusions; further, the pipeline can be applied invariably across different shape representations (e.g. meshes and point clouds), and in the presence of (even dramatic) missing parts such as those arising in real-world depth sensing applications. We showcase our method on a selection of challenging tasks, demonstrating results in line with, or even surpassing, state-of-the-art methods in the respective areas.     

Landmark transfer with minimal graph

We present an efficient and robust algorithm for the landmark transfer on 3D meshes that are approximately isometric. Given one or more custom landmarks placed by the user on a source mesh, our method efficiently computes corresponding landmarks on a family of target meshes. The technique is useful when a user is interested in characterization and reuse of application-specific landmarks on meshes of similar shape (for example, meshes coming from the same class of objects). Consequently, across a set of multiple meshes consistency is assured among landmarks, regardless of landmark geometric distinctiveness. The main advantage of our method over existing approaches is its low computation time. Differently from existing non-rigid registration techniques, our method detects and uses a minimum number of geometric features that are necessary to accurately locate the user-defined landmarks and avoids performing unnecessary full registration. In addition, unlike previous techniques that assume strict consistency with respect to geodesic distances, we adopt histograms of geodesic distance to define feature point coordinates, in order to handle the deviation of isometric deformation. This allows us to accurately locate the landmarks with only a small number of feature points in proximity, from which we build what we call a minimal graph. We demonstrate and evaluate the quality of transfer by our algorithm on a number of Tosca data sets.     

Locality-constrained nonnegative robust shape interaction subspace clustering and its applications

In this paper, we present a locality-constrained nonnegative robust shape interaction (LNRSI) subspace clustering method. LNRSI integrates the local manifold structure of data into the robust shape interaction (RSI) in a unified formulation, which guarantees the locality and the low-rank property of the optimal affinity graph. Compared with traditional low-rank representation (LRR) learning method, LNRSI can not only pursuit the global structure of data space by low-rank regularization, but also keep the locality manifold, which leads to a sparse and low-rank affinity graph. Due to the clear block-diagonal effect of the affinity graph, LNRSI is robust to noise and occlusions, and achieves a higher rate of correct clustering. The theoretical analysis of the clustering effect is also discussed. An efficient solution based on linearized alternating direction method with adaptive penalty (LADMAP) is built for our method. Finally, we evaluate the performance of LNRSI on both synthetic data and real computer vision tasks, i.e., motion segmentation and handwritten digit clustering. The experimental results show that our LNRSI outperforms several state-of-the-art algorithms.     

Consistent quadrangulation for shape collections via feature line co-extraction

This paper presents a method that takes a collection of 3D surface shapes, and produces a consistent and individually feature preserving quadrangulation of each shape. By exploring the correspondence among shapes within a collection, we coherently extract a set of representative feature lines as the key characteristics for the given shapes. Then we compute a smooth cross-field interpolating sparsely distributed directional constraints induced from the feature lines and apply the mixed-integer quadrangulation to generate the quad meshes. We develop a greedy algorithm to extract aligned cut graphs across the shape collection so that the meshes can be aligned in a common parametric domain. Computational results demonstrate that our approach not only produces consistent quad meshes across the entire collection with significant geometry variation but also achieves a trade-off between global structural simplicity for the collection and local geometry fidelity for each shape.     

Data‐Driven Sparse Priors of 3D Shapes

We present a sparse optimization framework for extracting sparse shape priors from a collection of 3D models. Shape priors are defined as point‐set neighborhoods sampled from shape surfaces which convey important information encompassing normals and local shape characterization. A 3D shape model can be considered to be formed with a set of 3D local shape priors, while most of them are likely to have similar geometry. Our key observation is that the local priors extracted from a family of 3D shapes lie in a very low‐dimensional manifold. Consequently, a compact and informative subset of priors can be learned to efficiently encode all shapes of the same family. A comprehensive library of local shape priors is first built with the given collection of 3D models of the same family. We then formulate a global, sparse optimization problem which enforces selecting representative priors while minimizing the reconstruction error. To solve the optimization problem, we design an efficient solver based on the Augmented Lagrangian Multipliers method (ALM). Extensive experiments exhibit the power of our data‐driven sparse priors in elegantly solving several high‐level shape analysis applications and geometry processing tasks, such as shape retrieval, style analysis and symmetry detection.     

Spectral Affine‐Kernel Embeddings

In this paper, we propose a controllable embedding method for high‐ and low‐dimensional geometry processing through sparse matrix eigenanalysis. Our approach is equally suitable to perform non‐linear dimensionality reduction on big data, or to offer non‐linear shape editing of 3D meshes and pointsets. At the core of our approach is the construction of a multi‐Laplacian quadratic form that is assembled from local operators whose kernels only contain locally‐affine functions. Minimizing this quadratic form provides an embedding that best preserves all relative coordinates of points within their local neighborhoods. We demonstrate the improvements that our approach brings over existing nonlinear dimensionality reduction methods on a number of datasets, and formulate the first eigen‐based as‐rigid‐as‐possible shape deformation technique by applying our affine‐kernel embedding approach to 3D data augmented with user‐imposed constraints on select vertices.     

Key-components: detection of salient regions on 3D meshes

In this paper, we present a method to detect stable components on 3D meshes. A component is a salient region on the mesh which contains discriminative local features. Our goal is to represent a 3D mesh with a set of regions, which we called key-components, that characterize the represented object and therefore, they could be used for effective matching and recognition. As key-components are features in coarse scales, they are less sensitive to mesh deformations such as noise. In addition, the number of key-components is low compared to other local representations such as keypoints, allowing us to use them in efficient subsequent tasks. A desirable characteristic of a decomposition is that the components should be repeatable regardless shape transformations. We show in the experiments that the key-components are repeatable and robust under several transformations using the SHREC’2010 feature detection benchmark. In addition, we discover the connection between the theory of saliency of visual parts from the cognitive science and the results obtained with our technique.     

整合原始人脸图像和其虚拟样本的人脸分类算法

人脸识别作为最具吸引力的生物识别技术之一,由于会受到不同的照明条件、面部表情、姿态和环境的影响,仍然是一个具有挑战性的任务.众所周知,一幅人脸图像是对人脸的一次采样,它不应该被看作是脸部的绝对精确表示.然而在实际应用中很难获得足够多的人脸样本.随着稀疏表示方法在图像重建问题中的成功应用,研究人员提出了一种特殊的分类方法,即基于稀疏表示的分类方法.受此启发,提出了在稀疏表示框架下的整合原始人脸图像和虚拟样本的人脸分类算法.首先,通过合成虚拟训练样本来减少面部表示的不确定性.然后,在原始训练样本和虚拟样本组成的混合样本中通过计算来消除对分类影响较小的类别和单个样本,在系数分解的过程中采用最小误差正交匹配追踪(Error-Constrained Orthogonal Matching Pursuit,OMP)方法,进而选出贡献程度大的类别样本并进行分类.实验结果表明,提出的方法不仅能获得较高的人脸识别的精度,而且还具有更低的计算复杂性.     

Automatic sub-category partitioning and parts localization for learning a robust object model

This paper introduces a novel topic model for learning a robust object model. In this hierarchical model, the layout topic is used to capture the local relationships among a limited number of parts when the part topic is used to locate the potential part regions. Naturally, an object model is represented as a probability distribution over a set of parts with certain layouts. Rather than a monolithic model, our object model is composed of multiple sub-category models designed to capture the significant variations in appearance and shape of an object category. Given a set of object instances with a bounding box, an iterative learning process is proposed to divide them into several sub-categories and learn the corresponding sub-category models without any supervision. Through an experiment in object detection, the learned object model is examined and the results highlight the advantages of our present method compared with others.     

基于协作低秩分层稀疏和LC-KSVD的人脸表情识别

为了降低人脸表情识别对待识别个体的依赖程度,控制识别字典规模,增加识别准确度,提出了一种基于协作低秩和分层稀疏的表情识别字典构建方法.通过协作低秩和分层稀疏表示(C-HiSLR)有效分离与待识别个体相关部分,保留表情变化部分,并结合标签一致区分字典学习(LC-KSVD)算法,进行相应待训练表情序列的重构识别和对应类别字典的区分程度的优化学习.该方法在CK+数据集上进行验证,识别效果较一般基于稀疏表示模型算法有明显的提升.     

结合稀疏表示和深度学习的视频中3D人体姿态估计

目的 2D姿态估计的误差是导致3D人体姿态估计产生误差的主要原因,如何在2D误差或噪声干扰下从2D姿态映射到最优、最合理的3D姿态,是提高3D人体姿态估计的关键。本文提出了一种稀疏表示与深度模型联合的3D姿态估计方法,以将3D姿态空间几何先验与时间信息相结合,达到提高3D姿态估计精度的目的。方法 利用融合稀疏表示的3D可变形状模型得到单帧图像可靠的3D初始值。构建多通道长短时记忆MLSTM（multi-channel long short term memory）降噪编/解码器,将获得的单帧3D初始值以时间序列形式输入到其中,利用MLSTM降噪编/解码器学习相邻帧之间人物姿态的时间依赖关系,并施加时间平滑约束,得到最终优化的3D姿态。结果 在Human3.6M数据集上进行了对比实验。对于两种输入数据：数据集给出的2D坐标和通过卷积神经网络获得的2D估计坐标,相比于单帧估计,通过MLSTM降噪编/解码器优化后的视频序列平均重构误差分别下降了12.6%,13%;相比于现有的基于视频的稀疏模型方法,本文方法对视频的平均重构误差下降了6.4%,9.1%。对于2D估计坐标数据,相比于现有的深度模型方法,本文方法对视频的平均重构误差下降了12.8%。结论 本文提出的基于时间信息的MLSTM降噪编/解码器与稀疏模型相结合,有效利用了3D姿态先验知识,视频帧间人物姿态连续变化的时间和空间依赖性,一定程度上提高了单目视频3D姿态估计的精度。     

3D human pose estimation from image using couple sparse coding

Recent studies have demonstrated that high-level semantics in data can be captured using sparse representation. In this paper, we propose an approach to human body pose estimation in static images based on sparse representation. Given a visual input, the objective is to estimate 3D human body pose using feature space information and geometrical information of the pose space. On the assumption that each data point and its neighbors are likely to reside on a locally linear patch of the underlying manifold, our method learns the sparse representation of the new input using both feature and pose space information and then estimates the corresponding 3D pose by a linear combination of the bases of the pose dictionary. Two strategies for dictionary construction are presented: (i) constructing the dictionary by randomly selecting the frames of a sequence and (ii) selecting specific frames of a sequence as dictionary atoms. We analyzed the effect of each strategy on the accuracy of pose estimation. Extensive experiments on datasets of various human activities show that our proposed method outperforms state-of-the-art methods.