基于尺寸建立个性化人体模型的方法

近年来,人体模型定制已成为计算机图形学领域的重要研究课题之一。文中提出一种基于少量人体尺寸生成个性化人体模型的方法——分块优化法。首先,根据MPI人体扫描模型数据库获得人体外形形变参数与尺寸参数,采用线性相关分析方法实现由若干尺寸恢复完整人体尺寸集。其次,通过参数优化的线性回归方法分析各部位三维人体外形参数与二维尺寸数据之间的关系,并根据输入尺寸对人体模型进行进一步精调。实验表明,上述方法能够生成准确反映人体外形的人体模型。     

A methodology for shape matching of non-rigid structures based on integrated graphical information

Shape matching is a long-studied problem and lies at the core of many applications in statistical shape analysis, virtual reality and human–computer interaction. This paper presents an automatic dense correspondence method to match the mesh vertices of two 3D shapes under near-isometric and non-rigid deformations. The goal is achieved by combining three types of graphic structure information. The method includes three major steps: first, we describe the vertices based on three types of graphical information, Euclidean structure information, Riemannian structure information, and conformal structure information; second, the match between two shapes is formulated as an optimization problem and a novel objective function is proposed; third, we resolve the optimal solution by using the projected descent optimization procedure to solve the objective function. The method is tested on various shape pairs with different poses, surface details, and topological noises. We demonstrate the performance of our approach through an extensive quantitative and qualitative evaluation on several challenging 3D shape matching datasets where we achieve superior performance to existing methods.     

Recursive non-rigid structure from motion with online learned shape prior

Most existing approaches in structure from motion for deformable objects focus on non-incremental solutions utilizing batch type algorithms. All data is collected before shape and motion reconstruction take place. This methodology is inherently unsuitable for applications that require real-time learning. Ideally the online system is capable of incrementally learning and building accurate shapes using current measurement data and past reconstructed shapes. Estimation of 3D structure and camera position is done online. To rely only on the measurements up until that moment is still a challenging problem.     

Automatic body landmark identification for various body figures

In this paper, we presented automatic body landmark identification algorithms that deals flexibly with the difference in body shapes and reduces the inconsistency resulting from the differences in body shapes. First, the landmark search range was defined using the statistical analysis. Next, body scan direction was identified and it was segmented. Next, automatic landmark identification algorithms were developed for each of the six landmarks and the accuracy was examined for each body shape. The scans were extracted from 5th Size Korea database. This algorithms were successfully tested on various body shapes and improved the robustness.

Relevance to industry

In automatic body measurement systems, the landmark location error occurring at nonstandard body shapes nullifies the advantage of saving time. It also makes the 3D scan measurements unreliable. The improvement of reliability and accuracy of the automatic 3D body measurement algorithm for various human body shapes will reduce the time for performing measurements and be practical for use in human-size-related production processes.     

Three-dimensional human shape inference from silhouettes: reconstruction and validation

Silhouettes are robust image features that provide considerable evidence about the three-dimensional (3D) shape of a human body. The information they provide is, however, incomplete and prior knowledge has to be integrated to reconstruction algorithms in order to obtain realistic body models. This paper presents a method that integrates both geometric and statistical priors to reconstruct the shape of a subject assuming a standardized posture from a frontal and a lateral silhouette. The method is comprised of three successive steps. First, a non-linear function that connects the silhouette appearances and the body shapes is learnt and used to create a first approximation. Then, the body shape is deformed globally along the principal directions of the population (obtained by performing principal component analysis over 359 subjects) to follow the contours of the silhouettes. Finally, the body shape is deformed locally to ensure it fits the input silhouettes as well as possible. Experimental results showed a mean absolute 3D error of 8 mm with ideal silhouettes extraction. Furthermore, experiments on body measurements (circumferences or distances between two points on the body) resulted in a mean error of 11 mm.     

自监督深度残差函数映射网络的3维模型对应关系计算

目的 针对传统非刚性3维模型的对应关系计算方法需要模型间真实对应关系监督的缺点,提出一种自监督深度残差函数映射网络（self-supervised deep residual functional maps network,SSDRFMN）。方法 首先将局部坐标系与直方图结合以计算3维模型的特征描述符,即方向直方图签名（signature of histograms of orientations,SHOT）描述符;其次将源模型与目标模型的SHOT描述符输入SSDRFMN,利用深度函数映射（deep functional maps,DFM）层计算两个模型间的函数映射矩阵,并通过模糊对应层将函数映射关系转换为点到点的对应关系;最后利用自监督损失函数计算模型间的测地距离误差,对计算出的对应关系进行评估。结果 实验结果表明,在MPI-FAUST数据集上,本文算法相比于有监督的深度函数映射（supervised deep functional maps,SDFM）算法,人体模型对应关系的测地误差减小了1.45;相比于频谱上采样（spectral upsampling,SU）算法减小了1.67。在TOSCA数据集上,本文算法相比于SDFM算法,狗、猫和狼等模型的对应关系的测地误差分别减小了3.13、0.98和1.89;相比于SU算法分别减小了2.81、2.22和1.11,并有效克服了已有深度函数映射方法需要模型间的真实对应关系来监督的缺点,使得该方法可以适用于不同的数据集,可扩展性大幅增强。结论 本文通过自监督深度残差函数映射网络训练模型的方向直方图签名描述符,提升了模型对应关系的准确率。本文方法可以适应于不同的数据集,相比传统方法,普适性较好。     

Bayes Reconstruction of Missing Teeth

We propose a method for restoring the surface of tooth crowns in a 3D model of a human denture, so that the pose and anatomical features of the tooth will work well for chewing. This is achieved by including information about the position and anatomy of the other teeth in the mouth. Our system contains two major parts: A statistical model of a selection of tooth shapes and a reconstruction of missing data. We use a training set consisting of 3D scans of dental cast models obtained with a laser scanner, and we have build a model of the shape variability of the teeth, their neighbors, and their antagonists, using the eigenstructure of the covariance matrix, also known as Principle Component Analysis (PCA). PCA is equivalent to fitting a multivariate Gaussian distribution to the data and the principle directions constitute a linear model for stochastic data and is used both for a data reduction or equivalently noise elimination and for data analysis. However for small sets of high dimensional data, the log-likelihood estimator for the covariance matrix is often far from convergence, and therefore reliable models must be obtained by use of prior information. We propose a natural and intrinsic regularization of the log-likelihood estimate based on differential geometrical properties of teeth surfaces, and we show general conditions under which this may be considered a Bayes prior. Finally we use Bayes method to propose the reconstruction of missing data, for e.g. finding the most probable shape of a missing tooth based on the best match with our shape model on the known data, and we superior improved reconstructions of our full system.     

EM-GPA: Generalized Procrustes analysis with hidden variables for 3D shape modeling

Aligning shapes is essential in many computer vision problems and generalized Procrustes analysis (GPA) is one of the most popular algorithms to align shapes. However, if some of the shape data are missing, GPA cannot be applied. In this paper, we propose EM-GPA, which extends GPA to handle shapes with hidden (missing) variables by using the expectation-maximization (EM) algorithm. For example, 2D shapes can be considered as 3D shapes with missing depth information due to the projection of 3D shapes into the image plane. For a set of 2D shapes, EM-GPA finds scales, rotations and 3D shapes along with their mean and covariance matrix for 3D shape modeling. A distinctive characteristic of EM-GPA is that it does not enforce any rank constraint often appeared in other work and instead uses GPA constraints to resolve the ambiguity in finding scales, rotations, and 3D shapes. The experimental results show that EM-GPA can recover depth information accurately even when the noise level is high and there are a large number of missing variables. By using the images from the FRGC database, we show that EM-GPA can successfully align 2D shapes by taking the missing information into consideration. We also demonstrate that the 3D mean shape and its covariance matrix are accurately estimated. As an application of EM-GPA, we construct a 2D + 3D AAM (active appearance model) using the 3D shapes obtained by EM-GPA, and it gives a similar success rate in model fitting compared to the method using real 3D shapes. EM-GPA is not limited to the case of missing depth information, but it can be easily extended to more general cases.     

A breast modeler based on analysis of breast scans

The advent of 3D scanning technology has allowed effective measurement and analysis of breast size and shape, attracting interests by plastic surgeons, brassier designers, etc. Much work remains, however, before 3D scanning systems can be successfully used in automated analysis and synthesis of the breast—filtering noise, filling holes, and, in case a statistical analysis is desired, finding correspondence among each scan data. Moreover, analysis of a sagged breast is difficult to obtain, due to occlusions. In this paper, we address the problems and specific issues of using 3D scan data for the analysis and synthesis of breast models. The goal of our work is to build a breast modeler which can help both surgeons and garment designers in analyzing breast volume and surface measurements. Given enough samples of scanned breasts, our modeler can generate highly realistic breast shape, with some expected and consistent variability. Copyright © 2007 John Wiley & Sons, Ltd.     

An efficient human model customization method based on orthogonal-view monocular photos

Human body modeling is a central task in computer graphics. In this paper, we propose an intelligent model customization method, in which customer’s detailed geometric characteristics can be reconstructed using limited size features extracted from the customer’s orthogonal-view photos. To realize model customization, we first propose a comprehensive shape representation to describe the geometrical shape characteristics of a human body. The shape representation has a layered structure and corresponds to important feature curves that define clothing size. Next, we identify and model a novel relationship model between 2D size features and 3D shape features for each cross-section using real subject scanned data. We predict a customer’s cross-sectional 3D shape based on size features extracted from the customer’s photos, and then we reconstruct the customer’s shape representation using predicted cross-sections. We develop a new deformation algorithm that deforms a template model into a customized shape using the reconstructed 3D shape representation. A total of 30 subjects, male and female, with varied body shapes have been recruited to verify the model customization method. The customized models show high degree of resemblance of the subjects, with accurate body sizes; the accuracy of the models is comparable to scan. It shows that the method is a feasible and efficient solution for human model customization that fulfills the specific needs of the clothing industry.     

Data‐Driven Sparse Priors of 3D Shapes

We present a sparse optimization framework for extracting sparse shape priors from a collection of 3D models. Shape priors are defined as point‐set neighborhoods sampled from shape surfaces which convey important information encompassing normals and local shape characterization. A 3D shape model can be considered to be formed with a set of 3D local shape priors, while most of them are likely to have similar geometry. Our key observation is that the local priors extracted from a family of 3D shapes lie in a very low‐dimensional manifold. Consequently, a compact and informative subset of priors can be learned to efficiently encode all shapes of the same family. A comprehensive library of local shape priors is first built with the given collection of 3D models of the same family. We then formulate a global, sparse optimization problem which enforces selecting representative priors while minimizing the reconstruction error. To solve the optimization problem, we design an efficient solver based on the Augmented Lagrangian Multipliers method (ALM). Extensive experiments exhibit the power of our data‐driven sparse priors in elegantly solving several high‐level shape analysis applications and geometry processing tasks, such as shape retrieval, style analysis and symmetry detection.     

ShapeSynth: Parameterizing model collections for coupled shape exploration and synthesis

Recent advances in modeling tools enable non‐expert users to synthesize novel shapes by assembling parts extracted from model databases. A major challenge for these tools is to provide users with relevant parts, which is especially difficult for large repositories with significant geometric variations. In this paper we analyze unorganized collections of 3D models to facilitate explorative shape synthesis by providing high‐level feedback of possible synthesizable shapes. By jointly analyzing arrangements and shapes of parts across models, we hierarchically embed the models into low‐dimensional spaces. The user can then use the parameterization to explore the existing models by clicking in different areas or by selecting groups to zoom on specific shape clusters. More importantly, any point in the embedded space can be lifted to an arrangement of parts to provide an abstracted view of possible shape variations. The abstraction can further be realized by appropriately deforming parts from neighboring models to produce synthesized geometry. Our experiments show that users can rapidly generate plausible and diverse shapes using our system, which also performs favorably with respect to previous modeling tools.     

Semi-Supervised 3D Shape Recognition via Multimodal Deep Co-training

3D shape recognition has been actively investigated in the field of computer graphics. With the rapid development of deep learning, various deep models have been introduced and achieved remarkable results. Most 3D shape recognition methods are supervised and learn only from the large amount of labeled shapes. However, it is expensive and time consuming to obtain such a large training set. In contrast to these methods, this paper studies a semi-supervised learning framework to train a deep model for 3D shape recognition by using both labeled and unlabeled shapes. Inspired by the co-training algorithm, our method iterates between model training and pseudo-label generation phases. In the model training phase, we train two deep networks based on the point cloud and multi-view representation simultaneously. In the pseudo-label generation phase, we generate the pseudo-labels of the unlabeled shapes using the joint prediction of two networks, which augments the labeled set for the next iteration. To extract more reliable consensus information from multiple representations, we propose an uncertainty-aware consistency loss function to combine the two networks into a multimodal network. This not only encourages the two networks to give similar predictions on the unlabeled set, but also eliminates the negative influence of the large performance gap between the two networks. Experiments on the benchmark ModelNet40 demonstrate that, with only 10% labeled training data, our approach achieves competitive performance to the results reported by supervised methods.     

Category‐Specific Salient View Selection via Deep Convolutional Neural Networks

In this paper, we present a new framework to determine up front orientations and detect salient views of 3D models. The salient viewpoint to human preferences is the most informative projection with correct upright orientation. Our method utilizes two Convolutional Neural Network (CNN) architectures to encode category‐specific information learnt from a large number of 3D shapes and 2D images on the web. Using the first CNN model with 3D voxel data, we generate a CNN shape feature to decide natural upright orientation of 3D objects. Once a 3D model is upright‐aligned, the front projection and salient views are scored by category recognition using the second CNN model. The second CNN is trained over popular photo collections from internet users. In order to model comfortable viewing angles of 3D models, a category‐dependent prior is also learnt from the users. Our approach effectively combines category‐specific scores and classical evaluations to produce a data‐driven viewpoint saliency map. The best viewpoints from the method are quantitatively and qualitatively validated with more than 100 objects from 20 categories. Our thumbnail images of 3D models are the most favoured among those from different approaches.     

A high-sensitivity 3D shape measurement method using a microscope

Optical microscopes generally have magnifications ranging from several tens to several thousands and they are often used to observe micro-specimens. Three-dimensional (3D) shape measurements of specimen surfaces are used in a wide range of fields including medicine, pharmacy, life science, and materials science. Conventional methods invariably employ 3D measurement techniques that involve adjusting the focal length of a microscope, which requires a complex automatic adjustment mechanism. Furthermore, since the depth is determined by controlling the focal length, 3D measurements have a low sensitivity. To realize a 3D measurement system with a simple configuration and a high measurement accuracy, we propose a high-sensitivity 3D shape measurement method that employs a microscope and is based on a pattern projection technique. The measurement system consists simply of a conventional optical microscope, a line laser, and a computer. The 3D measurement method employs slit pattern projection. A slit pattern produced by the line laser beam is projected onto the target surface and a reflected image is obtained using a camera installed on the microscope. 3D shape information of the target is obtained using image processing based on the triangulation method. We obtain 3D shape information of the target surface by scanning the slit projection pattern across most of the target surfaces by translating the stage on which the specimen is mounted. The experimental results demonstrate the effectiveness of the proposed method.     

Estimating body shape of dressed humans

The paper presents a method to estimate the detailed 3D body shape of a person even if heavy or loose clothing is worn. The approach is based on a space of human shapes, learned from a large database of registered body scans. Together with this database we use as input a 3D scan or model of the person wearing clothes and apply a fitting method, based on ICP (iterated closest point) registration and Laplacian mesh deformation. The statistical model of human body shapes enforces that the model stays within the space of human shapes. The method therefore allows us to compute the most likely shape and pose of the subject, even if it is heavily occluded or body parts are not visible. Several experiments demonstrate the applicability and accuracy of our approach to recover occluded or missing body parts from 3D laser scans.     

Non-Rigid Stereo Factorization

In this paper we address the problem of recovering 3D non-rigid structure from a sequence of images taken with a stereo pair. We have extended existing non-rigid factorization algorithms to the stereo camera case and presented an algorithm to decompose the measurement matrix into the motion of the left and right cameras and the 3D shape, represented as a linear combination of basis-shapes. The added constraints in the stereo camera case are that both cameras are viewing the same structure and that the relative orientation between both cameras is fixed. Our focus in this paper is on the recovery of flexible 3D shape rather than on the correspondence problem. We propose a method to compute reliable 3D models of deformable structure from stereo images. Our experiments with real data show that improved reconstructions can be achieved using this method. The algorithm includes a non-linear optimization step that minimizes image reprojection error and imposes the correct structure to the motion matrix by choosing an appropriate parameterization. We show that 3D shape and motion estimates can be successfully disambiguated after bundle adjustment and demonstrate this on synthetic and real image sequences. While this optimization step is proposed for the stereo camera case, it can be readily applied to the case of non-rigid structure recovery using a monocular video sequence. Electronic supplementary material Electronic supplementary material is available for this article at and accessible for authorised users.     

Global structure constrained local shape prior estimation for medical image segmentation

Organ shape plays an important role in clinical diagnosis, surgical planning and treatment evaluation. Shape modeling is a critical factor affecting the performance of deformable model based segmentation methods for organ shape extraction. In most existing works, shape modeling is completed in the original shape space, with the presence of outliers. In addition, the specificity of the patient was not taken into account. This paper proposes a novel target-oriented shape prior model to deal with these two problems in a unified framework. The proposed method measures the intrinsic similarity between the target shape and the training shapes on an embedded manifold by manifold learning techniques. With this approach, shapes in the training set can be selected according to their intrinsic similarity to the target image. With more accurate shape guidance, an optimized search is performed by a deformable model to minimize an energy functional for image segmentation, which is efficiently achieved by using dynamic programming. Our method has been validated on 2D prostate localization and 3D prostate segmentation in MRI scans. Compared to other existing methods, our proposed method exhibits better performance in both studies.