适用于家庭服务机器人的倒地人体检测方法

人体检测是家庭服务机器人的一项基本功能.本文针对复杂家庭环境,倒地人体面临地上杂物的干扰、遮挡等情况下,提出一种结合三维点云分割和局部特征匹配的倒地人体检测方法.该方法对点云进行分割之后将每个物体横向切分成若干段,对每段点云采用局部特征匹配并分类,并根据匹配段数来判断是否为倒地人体.实验结果表明,该方法在0.3秒的检测时间内,实现平均误识别率低于10%的高检测率,满足服务机器人实时性要求的同时具有良好的鲁棒性,即使人体部分被遮挡,依然可以检测到各种倒地姿态的人.     

Multilevel active registration for kinect human body scans: from low quality to high quality

Registration of 3D human body has been a challenging research topic for over decades. Most of the traditional human body registration methods require manual assistance, or other auxiliary information such as texture and markers. The majority of these methods are tailored for high-quality scans from expensive scanners. Following the introduction of the low-quality scans from cost-effective devices such as Kinect, the 3D data capturing of human body becomes more convenient and easier. However, due to the inevitable holes, noises and outliers in the low-quality scan, the registration of human body becomes even more challenging. To address this problem, we propose a fully automatic active registration method which deforms a high-resolution template mesh to match the low-quality human body scans. Our registration method operates on two levels of statistical shape models: (1) the first level is a holistic body shape model that defines the basic figure of human; (2) the second level includes a set of shape models for every body part, aiming at capturing more body details. Our fitting procedure follows a coarse-to-fine approach that is robust and efficient. Experiments show that our method is comparable with the state-of-the-art methods for high-quality meshes in terms of accuracy and it outperforms them in the case of low-quality scans where noises, holes and obscure parts are prevalent.     

Markerless Multiview Motion Capture with 3D Shape Model Adaptation

In this paper, we address simultaneous markerless motion and shape capture from 3D input meshes of partial views onto a moving subject. We exploit a computer graphics model based on kinematic skinning as template tracking model. This template model consists of vertices, joints and skinning weights learned a priori from registered full‐body scans, representing true human shape and kinematics‐based shape deformations. Two data‐driven priors are used together with a set of constraints and cues for setting up sufficient correspondences. A Gaussian mixture model‐based pose prior of successive joint configurations is learned to soft‐constrain the attainable pose space to plausible human poses. To make the shape adaptation robust to outliers and non‐visible surface regions and to guide the shape adaptation towards realistically appearing human shapes, we use a mesh‐Laplacian‐based shape prior. Both priors are learned/extracted from the training set of the template model learning phase. The output is a model adapted to the captured subject with respect to shape and kinematic skeleton as well as the animation parameters to resemble the observed movements. With example applications, we demonstrate the benefit of such footage. Experimental evaluations on publicly available datasets show the achieved natural appearance and accuracy.     

FARM: Functional Automatic Registration Method for 3D Human Bodies

We introduce a new method for non-rigid registration of 3D human shapes. Our proposed pipeline builds upon a given parametric model of the human, and makes use of the functional map representation for encoding and inferring shape maps throughout the registration process. This combination endows our method with robustness to a large variety of nuisances observed in practical settings, including non-isometric transformations, downsampling, topological noise and occlusions; further, the pipeline can be applied invariably across different shape representations (e.g. meshes and point clouds), and in the presence of (even dramatic) missing parts such as those arising in real-world depth sensing applications. We showcase our method on a selection of challenging tasks, demonstrating results in line with, or even surpassing, state-of-the-art methods in the respective areas.     

CustomCut: On‐demand Extraction of Customized 3D Parts with 2D Sketches

Several applications in shape modeling and exploration require identification and extraction of a 3D shape part matching a 2D sketch. We present CustomCut, an on‐demand part extraction algorithm. Given a sketched query, CustomCut automatically retrieves partially matching shapes from a database, identifies the region optimally matching the query in each shape, and extracts this region to produce a customized part that can be used in various modeling applications. In contrast to earlier work on sketch‐based retrieval of predefined parts, our approach can extract arbitrary parts from input shapes and does not rely on a prior segmentation into semantic components. The method is based on a novel data structure for fast retrieval of partial matches: the randomized compound k‐NN graph built on multi‐view shape projections. We also employ a coarse‐to‐fine strategy to progressively refine part boundaries down to the level of individual faces. Experimental results indicate that our approach provides an intuitive and easy means to extract customized parts from a shape database, and significantly expands the design space for the user. We demonstrate several applications of our method to shape design and exploration.     

A Continuum Mechanical Approach to Geodesics in Shape Space

In this paper concepts from continuum mechanics are used to define geodesic paths in the space of shapes, where shapes are implicitly described as boundary contours of objects. The proposed shape metric is derived from a continuum mechanical notion of viscous dissipation. A geodesic path is defined as the family of shapes such that the total amount of viscous dissipation caused by an optimal material transport along the path is minimized. The approach can easily be generalized to shapes given as segment contours of multi-labeled images and to geodesic paths between partially occluded objects. The proposed computational framework for finding such a minimizer is based on the time discretization of a geodesic path as a sequence of pairwise matching problems, which is strictly invariant with respect to rigid body motions and ensures a 1–1 correspondence along the induced flow in shape space. When decreasing the time step size, the proposed model leads to the minimization of the actual geodesic length, where the Hessian of the pairwise matching energy reflects the chosen Riemannian metric on the underlying shape space. If the constraint of pairwise shape correspondence is replaced by the volume of the shape mismatch as a penalty functional, one obtains for decreasing time step size an optical flow term controlling the transport of the shape by the underlying motion field. The method is implemented via a level set representation of shapes, and a finite element approximation is employed as spatial discretization both for the pairwise matching deformations and for the level set representations. The numerical relaxation of the energy is performed via an efficient multi-scale procedure in space and time. Various examples for 2D and 3D shapes underline the effectiveness and robustness of the proposed approach.     

Deformable model for estimating clothed and naked human shapes from a single image

Estimation of human shape from images has numerous applications ranging from graphics to surveillance. A single image provides insufficient constraints (e.g. clothing), making human shape estimation more challenging. We propose a method to simultaneously estimate a person’s clothed and naked shapes from a single image of that person wearing clothing. The key component of our method is a deformable model of clothed human shape. We learn our deformable model, which spans variations in pose, body, and clothes, from a training dataset. These variations are derived by the non-rigid surface deformation, and encoded in various low-dimension parameters. Our deformable model can be used to produce clothed 3D meshes for different people in different poses, which neither appears in the training dataset. Afterward, given an input image, our deformable model is initialized with a few user-specified 2D joints and contours of the person. We optimize the parameters of the deformable model by pose fitting and body fitting in an iterative way. Then the clothed and naked 3D shapes of the person can be obtained simultaneously. We illustrate our method for texture mapping and animation. The experimental results on real images demonstrate the effectiveness of our method.     

3D Body Shapes Estimation from Dressed‐Human Silhouettes

Estimation of 3D body shapes from dressed‐human photos is an important but challenging problem in virtual fitting. We propose a novel automatic framework to efficiently estimate 3D body shapes under clothes. We construct a database of 3D naked and dressed body pairs, based on which we learn how to predict 3D positions of body landmarks (which further constrain a parametric human body model) automatically according to dressed‐human silhouettes. Critical vertices are selected on 3D registered human bodies as landmarks to represent body shapes, so as to avoid the time‐consuming vertices correspondences finding process for parametric body reconstruction. Our method can estimate 3D body shapes from dressed‐human silhouettes within 4 seconds, while the fastest method reported previously need 1 minute. In addition, our estimation error is within the size tolerance for clothing industry. We dress 6042 naked bodies with 3 sets of common clothes by physically based cloth simulation technique. To the best of our knowledge, We are the first to construct such a database containing 3D naked and dressed body pairs and our database may contribute to the areas of human body shapes estimation and cloth simulation.     

Fast Retrieval of Isolated Visual Shapes

Similarity-based retrieval from databases of isolated visual shapes has become an important information retrieval problem. The goal of the current work is to achieve high retrieval speed with reasonable retrieval effectiveness, and support for partial and occluded shape queries. In the proposed method, histograms of local shape parts are coded as index vectors. To increase retrieval accuracy, a rich set of parts at all scales of the shape is used; specifically, the parts are defined as connected sequences of regions in curvature scale space. To increase efficiency, structural indexing is used to compare the index vectors of the query and database shapes. In experimental evaluations, the method retrieved at least one similar shape in the top three retrieved items 99–100% of the time, depending on the database. Average retrieval times ranged from 0.7 ms on a 131-shape database to 7 ms on a 1310-shape database. The method is thus suitable for fast, approximate shape retrieval in comparison with more accurate but more costly structural matching.     

Designing Personalized Garments with Body Movement

The standardized sizes used in the garment industry do not cover the range of individual differences in body shape for most people, leading to ill-fitting clothes, high return rates and overproduction. Recent research efforts in both industry and academia, therefore, focus on virtual try-on and on-demand fabrication of individually fitting garments. We propose an interactive design tool for creating custom-fit garments based on 3D body scans of the intended wearer. Our method explicitly incorporates transitions between various body poses to ensure a better fit and freedom of movement. The core of our method focuses on tools to create a 3D garment shape directly on an avatar without an underlying sewing pattern, and on the adjustment of that garment's rest shape while interpolating and moving through the different input poses. We alternate between cloth simulation and rest shape adjustment based on stretch to achieve the final shape of the garment. At any step in the real-time process, we allow for interactive changes to the garment. Once the garment shape is finalized for production, established techniques can be used to parameterize it into a 2D sewing pattern or transform it into a knitting pattern.     

三维鼻形:一种新的生物特征识别模式

鼻子是脸部最显著的器官之一,首次用三维鼻形作为生物特征进行身份识别,并提出了有效的三维鼻形识别方法,包括基于特征点的鼻子区域定位、利用ICP方法进行鼻形对齐、相似度量定义3个步骤.在FRGC 1.0数据库上获得3.15%的等错误率和92.1%的rank-1识别率,实验结果表明,采用三维鼻形有较强的判别能力.     

Estimating 3D human shapes from measurements

Recent advances in 3D imaging technologies give rise to databases of human shapes, from which statistical shape models can be built. These statistical models represent prior knowledge of the human shape and enable us to solve shape reconstruction problems from partial information. Generating human shape from traditional anthropometric measurements is such a problem, since these 1D measurements encode 3D shape information. Combined with a statistical shape model, these easy-to-obtain measurements can be leveraged to create 3D human shapes. However, existing methods limit the creation of the shapes to the space spanned by the database and thus require a large amount of training data. In this paper, we introduce a technique that extrapolates the statistically inferred shape to fit the measurement data using non-linear optimization. This method ensures that the generated shape is both human-like and satisfies the measurement conditions. We demonstrate the effectiveness of the method and compare it to existing approaches through extensive experiments, using both synthetic data and real human measurements.     

A Statistical Model of Human Pose and Body Shape

Generation and animation of realistic humans is an essential part of many projects in today's media industry. Especially, the games and special effects industry heavily depend on realistic human animation. In this work a unified model that describes both, human pose and body shape is introduced which allows us to accurately model muscle deformations not only as a function of pose but also dependent on the physique of the subject. Coupled with the model's ability to generate arbitrary human body shapes, it severely simplifies the generation of highly realistic character animations. A learning based approach is trained on approximately 550 full body 3D laser scans taken of 114 subjects. Scan registration is performed using a non-rigid deformation technique. Then, a rotation invariant encoding of the acquired exemplars permits the computation of a statistical model that simultaneously encodes pose and body shape. Finally, morphing or generating meshes according to several constraints simultaneously can be achieved by training semantically meaningful regressors.     

Global structure constrained local shape prior estimation for medical image segmentation

Organ shape plays an important role in clinical diagnosis, surgical planning and treatment evaluation. Shape modeling is a critical factor affecting the performance of deformable model based segmentation methods for organ shape extraction. In most existing works, shape modeling is completed in the original shape space, with the presence of outliers. In addition, the specificity of the patient was not taken into account. This paper proposes a novel target-oriented shape prior model to deal with these two problems in a unified framework. The proposed method measures the intrinsic similarity between the target shape and the training shapes on an embedded manifold by manifold learning techniques. With this approach, shapes in the training set can be selected according to their intrinsic similarity to the target image. With more accurate shape guidance, an optimized search is performed by a deformable model to minimize an energy functional for image segmentation, which is efficiently achieved by using dynamic programming. Our method has been validated on 2D prostate localization and 3D prostate segmentation in MRI scans. Compared to other existing methods, our proposed method exhibits better performance in both studies.     

Dynamic shape outlier detection for human locomotion

Dynamic human shape in video contains rich perceptual information, such as the body posture, identity, and even the emotional state of a person. Human locomotion activities, such as walking and running, have familiar spatiotemporal patterns that can easily be detected in arbitrary views. We present a framework for detecting shape outliers for human locomotion using a dynamic shape model that factorizes the body posture, the viewpoint, and the individual’s shape style. The model uses a common embedding of the kinematic manifold of the motion and factorizes the shape variability with respect to different viewpoints and shape styles in the space of the coefficients of the nonlinear mapping functions that are used to generate the shapes from the kinematic manifold representation. Given a corrupted input silhouette, an iterative procedure is used to recover the body posture, viewpoint, and shape style. We use the proposed outlier detection approach to fill in the holes in the input silhouettes, and detect carried objects, shadows, and abnormal motions.     

Constructing 3D human model from front and side images

Constructing 3D human model from 2D images provides a cost-effective approach to visualize digital human in virtual environment. This paper presents a systematic approach for constructing 3D human model using the front and side images of a person. The silhouettes of human body are first detected and the feature points on the silhouettes are subsequently identified. The feature points are further used to obtain the body dimensions that are necessary for identifying a template 3D human model. The shape of the template human model can be modified by the free-form deformation method. Moreover, the proposed approach has been applied for constructing the 3D human models of 30 subjects. The comparisons between the constructed 3D models and the 3D scanning models of the 30 subjects indicate that the proposed system is very effective and robust.     

DeepGarment : 3D Garment Shape Estimation from a Single Image

3D garment capture is an important component for various applications such as free‐view point video, virtual avatars, online shopping, and virtual cloth fitting. Due to the complexity of the deformations, capturing 3D garment shapes requires controlled and specialized setups. A viable alternative is image‐based garment capture. Capturing 3D garment shapes from a single image, however, is a challenging problem and the current solutions come with assumptions on the lighting, camera calibration, complexity of human or mannequin poses considered, and more importantly a stable physical state for the garment and the underlying human body. In addition, most of the works require manual interaction and exhibit high run‐times. We propose a new technique that overcomes these limitations, making garment shape estimation from an image a practical approach for dynamic garment capture. Starting from synthetic garment shape data generated through physically based simulations from various human bodies in complex poses obtained through Mocap sequences, and rendered under varying camera positions and lighting conditions, our novel method learns a mapping from rendered garment images to the underlying 3D garment model. This is achieved by training Convolutional Neural Networks (CNN‐s) to estimate 3D vertex displacements from a template mesh with a specialized loss function. We illustrate that this technique is able to recover the global shape of dynamic 3D garments from a single image under varying factors such as challenging human poses, self occlusions, various camera poses and lighting conditions, at interactive rates. Improvement is shown if more than one view is integrated. Additionally, we show applications of our method to videos.     

Automatic body landmark identification for various body figures

In this paper, we presented automatic body landmark identification algorithms that deals flexibly with the difference in body shapes and reduces the inconsistency resulting from the differences in body shapes. First, the landmark search range was defined using the statistical analysis. Next, body scan direction was identified and it was segmented. Next, automatic landmark identification algorithms were developed for each of the six landmarks and the accuracy was examined for each body shape. The scans were extracted from 5th Size Korea database. This algorithms were successfully tested on various body shapes and improved the robustness.

Relevance to industry

In automatic body measurement systems, the landmark location error occurring at nonstandard body shapes nullifies the advantage of saving time. It also makes the 3D scan measurements unreliable. The improvement of reliability and accuracy of the automatic 3D body measurement algorithm for various human body shapes will reduce the time for performing measurements and be practical for use in human-size-related production processes.     

基于关键帧的人体行为识别方法

提出了一种基于关键帧的人体行为识别方法。利用关于弧长的方向函数对人体形状进行描述以及通过形状空间中两点间的测地线距离对形状间的相似性进行度量,提取视频序列中的关键帧,通过模板匹配的方法实现对行为的识别。对CASIA单人行为数据库和Weismann数据库进行测试,实验结果表明该方法具有较高的识别率。