EM-GPA: Generalized Procrustes analysis with hidden variables for 3D shape modeling

Aligning shapes is essential in many computer vision problems and generalized Procrustes analysis (GPA) is one of the most popular algorithms to align shapes. However, if some of the shape data are missing, GPA cannot be applied. In this paper, we propose EM-GPA, which extends GPA to handle shapes with hidden (missing) variables by using the expectation-maximization (EM) algorithm. For example, 2D shapes can be considered as 3D shapes with missing depth information due to the projection of 3D shapes into the image plane. For a set of 2D shapes, EM-GPA finds scales, rotations and 3D shapes along with their mean and covariance matrix for 3D shape modeling. A distinctive characteristic of EM-GPA is that it does not enforce any rank constraint often appeared in other work and instead uses GPA constraints to resolve the ambiguity in finding scales, rotations, and 3D shapes. The experimental results show that EM-GPA can recover depth information accurately even when the noise level is high and there are a large number of missing variables. By using the images from the FRGC database, we show that EM-GPA can successfully align 2D shapes by taking the missing information into consideration. We also demonstrate that the 3D mean shape and its covariance matrix are accurately estimated. As an application of EM-GPA, we construct a 2D + 3D AAM (active appearance model) using the 3D shapes obtained by EM-GPA, and it gives a similar success rate in model fitting compared to the method using real 3D shapes. EM-GPA is not limited to the case of missing depth information, but it can be easily extended to more general cases.     

Nonrigid structure-from-motion: estimating shape and motion with hierarchical Priors

This paper describes methods for recovering time-varying shape and motion of non-rigid 3D objects from uncalibrated 2D point tracks. For example, given a video recording of a talking person, we would like to estimate the 3D shape of the face at each instant, and learn a model of facial deformation. Time-varying shape is modeled as a rigid transformation combined with a non-rigid deformation. Reconstruction is ill-posed if arbitrary deformations are allowed, and thus additional assumptions about deformations are required. We first suggest restricting shapes to lie within a low-dimensional subspace, and describe estimation algorithms. However, this restriction alone is insufficient to constrain reconstruction. To address these problems, we propose a reconstruction method using a Probabilistic Principal Components Analysis (PPCA) shape model, and an estimation algorithm that simultaneously estimates 3D shape and motion for each instant, learns the PPCA model parameters, and robustly fills-in missing data points. We then extend the model to model temporal dynamics in object shape, allowing the algorithm to robustly handle severe cases of missing data.     

SRF-Net: Spatial Relationship Feature Network for Tooth Point Cloud Classification

3D scanned point cloud data of teeth is popular used in digital orthodontics. The classification and semantic labelling for point cloud of each tooth is a key and challenging task for planning dental treatment. Utilizing the priori ordered position information of tooth arrangement, we propose an effective network for tooth model classification in this paper. The relative position and the adjacency similarity feature vectors are calculated for tooth 3D model, and combine the geometric feature into the fully connected layers of the classification training task. For the classification of dental anomalies, we present a dental anomalies processing method to improve the classification accuracy. We also use FocalLoss as the loss function to solve the sample imbalance of wisdom teeth. The extensive evaluations, ablation studies and comparisons demonstrate that the proposed network can classify tooth models accurately and automatically and outperforms state-of-the-art point cloud classification methods.     

三维牙颌模型的牙齿形状建模方法

针对三维牙颌模型由于齿间粘连导致单颗牙齿局部形状缺失的问题,提出一种综合性的牙齿形状建模方法.根据三维牙颌模型的形态特征,采用基于曲面最小主曲率的方法对粘连区域进行准确的检测提取;对删除齿间粘连区域后生成的齿间开口利用局部最优化权值规则进行剖分并细分优化,生成"中间"恢复曲面;根据牙齿的生理医学特征,以齿间开口的边界信息为约束条件,采用基于最小化曲面能量函数的方法对"中间"恢复曲面进行变形调整,实现单颗牙齿缺失形状的准确恢复.实验结果表明,采用文中方法能取得满足临床口腔医学要求的形状建模结果.     

Joint estimation of segmentation and structure from motion

We present a novel optimisation framework for the estimation of the multi-body motion segmentation and 3D reconstruction of a set of point trajectories in the presence of missing data. The proposed solution not only assigns the trajectories to the correct motion but it also solves for the 3D location of multi-body shape and it fills the missing entries in the measurement matrix. Such a solution is based on two fundamental principles: each of the multi-body motions is controlled by a set of metric constraints that are given by the specific camera model, and the shape matrix that describes the multi-body 3D shape is generally sparse. We jointly include such constraints in a unique optimisation framework which, starting from an initial segmentation, iteratively enforces these set of constraints in three stages. First, metric constraints are used to estimate the 3D metric shape and to fill the missing entries according to an orthographic camera model. Then, wrongly segmented trajectories are detected by using sparse optimisation of the shape matrix. A final reclassification strategy assigns the detected points to the right motion or discards them as outliers. We provide experiments that show consistent improvements to previous approaches both on synthetic and real data.     

Dimensionality reduction for multivariate time-series data mining

A multivariate time series is one of the most important objects of research in data mining. Time and variables are two of its distinctive characteristics that add the complication of the algorithms applied to data mining. Reduction in the dimensionality is often regarded as an effective way to address these issues. In this paper, we propose a method based on principal component analysis (PCA) to effectively reduce the dimensionality. We call it “piecewise representation based on PCA” (PPCA), which segments multivariate time series into several sequences, calculates the covariance matrix for each of them in terms of the variables, and employs PCA to obtain the principal components in an average covariance matrix. The results of the experiments, including retained information analysis, classification, and a comparison of the central processing unit time consumption, demonstrate that the PPCA method used to reduce the dimensionality in multivariate time series is superior to the prior methods.

     

Individual tooth segmentation from CT images using level set method with shape and intensity prior

3D visualization of teeth from CT images provides important assistance for dentists performing orthodontic surgery and treatment. However, dental CT images present several major challenges for the segmentation of tooth, which touches with adjacent teeth as well as surrounding periodontium and jaw bones. Moreover, tooth contour suffers from topological changes and splits into several branches. In this work, we focus on the segmentation of individual teeth with complete crown and root parts. To this end, we propose adaptive active contour tracking algorithms: single level set method tracking for root segmentation to handle the complex image conditions as well as the root branching problem, and coupled level set method tracking for crown segmentation in order to separate the touching teeth and create the virtual common boundaries between them. Furthermore, we improve the variational level set method in several aspects: gradient direction is introduced into the level set framework to prevent catching the surrounding object boundaries; in addition to the shape prior, intensity prior is introduced to provide adaptive shrinking or expanding forces in order to deal with the topological changes. The test results for both tooth segmentation and 3D reconstruction show that the proposed method can visualize individual teeth with high accuracy and efficiency.     

Estimating 3D shape from degenerate sequences with missing data

Reconstructing a 3D scene from a moving camera is one of the most important issues in the field of computer vision. In this scenario, not all points are known in all images (e.g. due to occlusion), thus generating missing data. On the other hand, successful 3D reconstruction algorithms like Tomasi & Kanade’s factorization method, require an orthographic model for the data, which is adequate in close-up views. The state-of-the-art handles the missing points in this context by enforcing rank constraints on the point track matrix. However, quite frequently, close-up views tend to capture planar surfaces producing degenerate data. Estimating missing data using the rank constraint requires that all known measurements are “full rank” in all images of the sequence. If one single frame is degenerate, the whole sequence will produce high errors on the reconstructed shape, even though the observation matrix verifies the rank 4 constraint. In this paper, we propose to solve the structure from motion problem with degenerate data, introducing a new factorization algorithm that imposes the full scaled-orthographic model in one single optimization procedure. By imposing all model constraints, a unique (correct) 3D shape is estimated regardless of the data degeneracies. Experiments show that remarkably good reconstructions are obtained with an approximate models such as orthography.     

Choosing an appropriate number of factors in factor analysis with incomplete data

When we conduct factor analysis, the number of factors is often unknown in advance. Among many decision rules for an appropriate number of factors, it is easy to find approaches that make use of the estimated covariance matrix. When data include missing values, the estimated covariance matrix using either complete cases or available cases may not accurately represent the true covariance matrix, and decision based on the estimated covariance matrix may be misleading. We discuss how to apply model selection techniques using AIC or BIC to choose an appropriate number of factors when data include missing values. In the simulation study, it is shown that the suggested methods select the correct number of factors for simulated data with known number of factors.     

水平集活动轮廓模型的3维牙齿重建

目的 建立患者口腔每颗牙齿独立的3维模型对于计算机口腔修复的精确定位和量化评估具有十分重要的意义。针对口腔CT图像序列的牙齿形态变化和排列特点,提出一种新颖的基于水平集活动轮廓模型的3维牙齿重建方法。方法 基于层间映射机制,对不同部位牙层切片采用不同的分割模型：利用先验形状约束能量、基于Flux模型的边缘梯度能量、基于先验灰度的局部区域能量相结合构造的单相混合水平集模型分割牙根层切片轮廓;利用结合区域竞争约束的双相混合水平集模型分割牙冠层切片轮廓;最后利用这些层间轮廓重建出牙齿3维模型。结果 对不同位置的牙齿CT图像进行分割实验,结果表明,与现有的方法相比,本文方法具有较好的分割效果和较高的准确率,平均分割相似系数达到96%。结论 本文的混合水平集模型能有效克服牙髓腔和牙槽骨的干扰以及图像灰度不均匀等问题,较准确地重建出每颗牙齿独立的3维模型,从而为制定口腔修复规划、生物力学分析等奠定坚实的基础。     

Multi-stage 3D reconstruction under circular motion

In this paper, we propose a new method for 3D reconstruction from an image sequence captured by a camera with constant intrinsic parameters undergoing circular motion. We introduce a method, called circular projective reconstruction, for enforcing the circular constraint in a factorization-based projective reconstruction. To deal with the missing data problem, our method uses a multi-stage approach to reconstructing the objects and cameras, which first computes a circular projective reconstruction of a sub-sequence and then extends the reconstruction to the complete sequence. Camera matrix, rotation angles, and 3D structure are computed iteratively in a way that the 2D reprojection error is minimized. The algorithm is evaluated using real image sequences.     

Low complexity adaptive algorithms for Principal and Minor Component Analysis

This article introduces new low cost algorithms for the adaptive estimation and tracking of principal and minor components. The proposed algorithms are based on the well-known OPAST method which is adapted and extended in order to achieve the desired MCA or PCA (Minor or Principal Component Analysis). For the PCA case, we propose efficient solutions using Givens rotations to estimate the principal components out of the weight matrix given by OPAST method. These solutions are then extended to the MCA case by using a transformed data covariance matrix in such a way the desired minor components are obtained from the PCA of the new (transformed) matrix. Finally, as a byproduct of our PCA algorithm, we propose a fast adaptive algorithm for data whitening that is shown to overcome the recently proposed RLS-based whitening method.     

A novel 3D morphing approach for tooth occlusal surface reconstruction

Reconstruction of the tooth occlusal surface is an important aspect of dental CAD systems. The design surface should fit to the existing tooth articulation and keep the morphological features of the generic teeth. This paper presents the use of a 3D morphing technique to reconstruct the missing inlay or crown occlusal surface. This method involves three necessary steps of tooth cavity contour extraction, feature points identification and tooth surface deformation. Because the missing occlusal surface is often determined by the outline of the cavity inside the remaining tooth, the cavity contour is tracked firstly by a 3D optimal path searching algorithm. Then through identifying a number of corresponding feature points on the standard tooth and the remaining tooth by a Snake model algorithm, the method uses a mapping function called Radial Basis Function (RBF) to define the spatial relations of all points in both teeth, and interpolate the intermediate feature positions, such that the standard tooth surface is deformed into alignment, and the missing surface is determined. This presents a direct and simple method to automatically reconstruct the occlusal surface. The corresponding features are fully considered in the morphing algorithm, consequently the morphological features of the tooth are well retained.     

Local fuzzy PCA based GMM with dimension reduction on speaker identification

To reduce the high dimensionality required for training of feature vectors in speaker identification, we propose an efficient GMM based on local PCA with fuzzy clustering. The proposed method firstly partitions the data space into several disjoint clusters by fuzzy clustering, and then performs PCA using the fuzzy covariance matrix on each cluster. Finally, the GMM for speaker is obtained from the transformed feature vectors with reduced dimension in each cluster. Compared to the conventional GMM with diagonal covariance matrix, the proposed method shows faster result with less storage maintaining same performance.     

Fusing multiview and photometric stereo for 3D reconstruction under uncalibrated illumination

We propose a method to obtain a complete and accurate 3D model from multiview images captured under a variety of unknown illuminations. Based on recent results showing that for Lambertian objects, general illumination can be approximated well using low-order spherical harmonics, we develop a robust alternating approach to recover surface normals. Surface normals are initialized using a multi-illumination multiview stereo algorithm, then refined using a robust alternating optimization method based on the l(1) metric. Erroneous normal estimates are detected using a shape prior. Finally, the computed normals are used to improve the preliminary 3D model. The reconstruction system achieves watertight and robust 3D reconstruction while neither requiring manual interactions nor imposing any constraints on the illumination. Experimental results on both real world and synthetic data show that the technique can acquire accurate 3D models for Lambertian surfaces, and even tolerates small violations of the Lambertian assumption.     

Principal component analysis for data containing outliers and missing elements

Two approaches are presented to perform principal component analysis (PCA) on data which contain both outlying cases and missing elements. At first an eigendecomposition of a covariance matrix which can deal with such data is proposed, but this approach is not fit for data where the number of variables exceeds the number of cases. Alternatively, an expectation robust (ER) algorithm is proposed so as to adapt the existing methodology for robust PCA to data containing missing elements. According to an extensive simulation study, the ER approach performs well for all data sizes concerned. Using simulations and an example, it is shown that by virtue of the ER algorithm, the properties of the existing methods for robust PCA carry through to data with missing elements.     

A Bayesian hybrid Huberized support vector machine and its applications in high-dimensional medical data

A hybrid Huberized support vector machine (HHSVM) with an elastic-net penalty has been developed for cancer tumor classification based on thousands of gene expression measurements. In this paper, we develop a Bayesian formulation of the hybrid Huberized support vector machine for binary classification. For the coefficients of the linear classification boundary, we propose a new type of prior, which can select variables and group them together simultaneously. Our proposed prior is a scale mixture of normal distributions and independent gamma priors on a transformation of the variance of the normal distributions. We establish a direct connection between the Bayesian HHSVM model with our special prior and the standard HHSVM solution with the elastic-net penalty. We propose a hierarchical Bayes technique and an empirical Bayes technique to select the penalty parameter. In the hierarchical Bayes model, the penalty parameter is selected using a beta prior. For the empirical Bayes model, we estimate the penalty parameter by maximizing the marginal likelihood. The proposed model is applied to two simulated data sets and three real-life gene expression microarray data sets. Results suggest that our Bayesian models are highly successful in selecting groups of similarly behaved important genes and predicting the cancer class. Most of the genes selected by our models have shown strong association with well-studied genetic pathways, further validating our claims.     

1种基于遗失数据重构的软测量方法

提出1种遗失数据重构思想下的软测量方法:先采用主元分析(PCA)离线建立所有变量(包括难测变量)的主元模型,实际应用时,将实时的难测变量看作遗失数据,通过遗失数据重构方法估计出难测变量,增加了软测量方法的灵活性.更进一步,在重构遗失数据时,使用马氏距离取代欧几里德距离作为指标,更准确地反映了过程变量之间的相关关系,由此...