Reconstruction of 3D human body pose from stereo image sequences based on top-down learning

This paper presents a novel method for reconstructing a 3D human body pose from stereo image sequences based on a top-down learning method. However, it is inefficient to build a statistical model using all training data. Therefore, the training data is hierarchically divided into several clusters to reduce the complexity of the learning problem. In the learning stage, the human body model database is hierarchically constructed by classifying the training data into several sub-clusters with silhouette images. The data of each cluster in the bottom level is represented by a linear combination of examples. In the reconstruction stage, the proposed method hierarchically searches a cluster for the best matching silhouette image using a silhouette history image (SHI). Then, the 3D human body pose is reconstructed from a depth image using a linear combination of examples method. By using depth information to reconstruct 3D human body pose, the similar poses in silhouette images are estimated as different 3D human body poses. The experimental results demonstrate that the proposed method is efficient and effective for reconstructing 3D human body poses.     

3-D human pose recovery using nonrigid point set registration and body part tracking of depth data

In this paper, we present a novel approach for recovering a 3-D pose from a single human body depth silhouette using nonrigid point set registration and body part tracking. In our method, a human body depth silhouette is presented as a set of 3-D points and matched to another set of 3-D points using point correspondences. To recognize and maintain body part labels, we initialize the first set of points to corresponding human body parts, resulting in a body part-labeled map. Then, we transform the points to a sequential set of points based on point correspondences determined by nonrigid point set registration. After point registration, we utilize the information from tracked body part labels and registered points to create a human skeleton model. A 3-D human pose gets recovered by mapping joint information from the skeleton model to a 3-D synthetic human model. Quantitative and qualitative evaluation results on synthetic and real data show that complex human poses can be recovered more reliably with lower errors compared to other conventional techniques for 3-D pose recovery.     

Synthesis of Silhouettes and Visual Hull Reconstruction for Articulated Humans

In this paper, we propose a complete framework for improved synthesis and understanding of the human pose from a limited number of silhouette images. It combines the active image-based visual hull (IBVH) algorithm and a contour-based body part segmentation technique. We derive a simple, approximate algorithm to decide the extrinsic parameters of a virtual camera, and synthesize the turntable image collection of the person using the IBVH algorithm by actively moving the virtual camera on a properly computed circular trajectory around the person. Using the turning function distance as the silhouette similarity measurement, this approach can be used to generate the desired pose-normalized images for recognition applications. In order to overcome the inability of the visual hull (VH) method to reconstruct concave regions, we propose a contour-based human body part localization algorithm to segment the silhouette images into convex body parts. The body parts observed from the virtual view are generated separately from the corresponding body parts observed from the input views and then assembled together for a more accurate VH reconstruction. Furthermore, the obtained turntable image collection helps to improve the body part segmentation and identification process. By using the inner distance shape context (IDSC) measurement, we are able to estimate the body part locations more accurately from a synthesized view where we can localize the body part more precisely. Experiments show that the proposed algorithm can greatly improve body part segmentation and hence shape reconstruction results.      

人体运动分析的实例学习方法

目的 面向实时、准确、鲁棒的人体运动分析应用需求,从运动分析的特征提取和运动建模问题出发,本文人体运动分析的实例学习方法。方法 在构建人体姿态实例库基础上,首先,采用运动检测方法得到视频每帧的人体轮廓;其次,基于形状上下文轮廓匹配方法,从实例库中检索得到每帧视频的候选姿态集;最后,通过统计建模和转移概率建模实现人体运动分析。结果 对步行、跑步、跳跃等测试视频进行实验,基于轮廓的形状上下文特征表示和匹配方法具有良好的表达能力;本文方法运动分析结果,关节夹角平均误差在5°左右,与其他算法相比,有效提高了运动分析的精度。结论 本文人体运动分析的实例学习方法,能有效分析单目视频中的人体运动,并克服了映射的深度歧义,对运动的视角变化鲁棒,具有良好的计算效率和精度。     

Computer Vision-Based Human Body Segmentation and Posture Estimation

This paper proposes a new method for vision-based human body posture estimation using body silhouette and skin-color information. A moving object segmentation algorithm is first proposed to distinguish the human body from the background using a sequence of images. This algorithm uses a fast Euler number computation technique to automatically determine the threshold of both frame and background differences. After segmentation, a sequence of image processing approaches then creates a complete silhouette of the human body. The objective of posture estimation is to locate five significant body points, including the head, tips of the feet, and tips of the hands. These significant points are first selected from convex points on a defined distance curve. A number of heuristic rules based on body shape characteristics are used to select the proper points among these convex candidates. These rules use features like the principal and minor axes of the human body, their interactions with the silhouette contour, the relative distances between convex points, and the curvature of convex points. An auxiliary skin-color feature is used when the silhouette shape features alone are not sufficient to estimate the significant points. Experimental results show that the proposed approach can efficiently and effectively locate the significant body points for most postures.      

一种基于视点距离的三维模型特征提取算法

如何对三维模型进行特征提取是近年来出现的三维模型检索中的主要问题.文章给出了一种基于视点距离的特征提取算法,该算法利用正规化后的三维模型表面到观察点的距离信息生成六幅距离图像,然后对图像进行二维傅立叶变换并对变换后的频域信息进行低频采样从而得到三维模型的特征向量.该算法克服了基于三维投影的二维图像轮廓算法中丢失模型空域信息、缺乏对图像内部信息进行描述的缺点.实验结果表明,该算法比基于轮廓算法的检索精确度提高了19%.     

Volumetric stereo and silhouette fusion for image-based modeling

This paper presents a volumetric stereo and silhouette fusion algorithm for acquiring high quality models from multiple calibrated photographs. Our method is based on computing and merging depth maps. Different from previous methods of this category, the silhouette information is also applied in our algorithm to recover the shape information on the textureless and occluded areas. The proposed algorithm starts by computing visual hull using a volumetric method in which a novel projection test method is proposed for visual hull octree construction. Then, the depth map of each image is estimated by an expansion-based approach that returns a 3D point cloud with outliers and redundant information. After generating an oriented point cloud from stereo by rejecting outlier, reducing scale, and estimating surface normal for the depth maps, another oriented point cloud from silhouette is added by carving the visual hull octree structure using the point cloud from stereo to restore the textureless and occluded surfaces. Finally, Poisson Surface Reconstruction approach is applied to convert the oriented point cloud both from stereo and silhouette into a complete and accurate triangulated mesh model. The proposed approach has been implemented and the performance of the approach is demonstrated on several real data sets, along with qualitative comparisons with the state-of-the-art image-based modeling techniques according to the Middlebury benchmark.     

3D human pose estimation in motion based on multi-stage regression

3D human pose estimation in motion is a hot research direction in the field of computer vision. However, the performance of the algorithm is affected by the complexity of 3D spatial information, self-occlusion of human body, mapping uncertainty and other problems. In this paper, we propose a 3D human joint localization method based on multi-stage regression depth network and 2D to 3D point mapping algorithm. First of all, we use a single RGB image as the input, through the introduction of heatmap and multi-stage regression to constantly optimize the coordinates of human joint points. Then we input the 2D joint points into the mapping network for calculation, and get the coordinates of 3D human body joint points, and then to complete the 3D human body pose estimation task. The MPJPE of the algorithm in Human3.6 M dataset is 40.7. The evaluation of dataset shows that our method has obvious advantages.     

Estimation of 3-D human body posture via co-registration of 3-D human model and sequential stereo information

In this paper, we present a technique for estimating three-dimensional (3-D) human body posture from a set of sequential stereo images. We estimated the pixel displacements of stereo image pairs to reconstruct 3-D information. We modeled the human body with a set of ellipsoids connected by kinematic chains and parameterized with rotational angles at each body joint. To estimate human posture from the 3-D data, we developed a new algorithm based on expectation maximization (EM) with two-step iterations, assigning the 3-D data to different body parts and refining the kinematic parameters to fit the 3-D model to the data. The algorithm is iterated until it converges on the correct posture. Experimental results with synthetic and real data demonstrate that our method is capable of reconstructing 3-D human posture from stereo images. Our method is robust and generic; any useful information for locating the body parts can be integrated into our framework to improve the outcomes.     

基于图像语义分割和CNN模型的老人跌倒检测

随着老龄化社会的到来,独居老人的安全问题越来越引人关注.其中,跌倒是老人在家中最常见也是危害最大的风险之一.当前已经有许多关于老人跌倒检测的算法,它们大多应用在摄像头固定的场景下,并主要采用前景提取方法来获取人体轮廓.采用固定摄像头意味着需要为家中每一处独立的空间都安装监控设备才能保证对于老人的全面监控,这显然不实用.基于此,本文采用图像语义分割算法和CNN分类模型,提出了一种可用于移动摄像头上的老人跌倒检测算法.首先采用当前流行的全卷积神经网络（fully convolutional network）语义分割算法^[1]分割出图像中的人体,对于满足面积比例条件的情况,直接通过宽高比特征判断人体是否处于跌倒状态;否则,提出一种融合的CNN人体姿态判别模型,将人体区域分成Stand、Fall、Half-Lying三种情况分别进行检测,最后根据三者的分类结果判定图像中是否包含跌倒人体.实验结果显示,文中的算法在具有较高的识别准确率（91.32%）的同时,具有较低的误报率（1.66%）.     

Real-time 3D human pose recovery from a single depth image using principal direction analysis

In this paper, we present a novel approach to recover a 3D human pose in real-time from a single depth image using principal direction analysis (PDA). Human body parts are first recognized from a human depth silhouette via trained random forests (RFs). PDA is applied to each recognized body part, which is presented as a set of points in 3D, to estimate its principal direction. Finally, a 3D human pose is recovered by mapping the principal direction to each body part of a 3D synthetic human model. We perform both quantitative and qualitative evaluations of our proposed 3D human pose recovering methodology. We show that our proposed approach has a low average reconstruction error of 7.07 degrees for four key joint angles and performs more reliably on a sequence of unconstrained poses than conventional methods. In addition, our methodology runs at a speed of 20 FPS on a standard PC, indicating that our system is suitable for real-time applications. Our 3D pose recovery methodology is applicable to applications ranging from human computer interactions to human activity recognition.     

Shape from silhouettes based on a centripetal pentahedron model

In this paper we present a novel volumetric shape from silhouette (SfS) algorithm based on a centripetal pentahedron model (pent-model). The pent-model is an object-centered volumetric model composed of a set of pentahedrons cut from the centripetal triangular pyramids, which together partition the 3D space. The SfS algorithm first computes the pyramids by constructing a geodesic sphere. These pyramids are then projected onto the image planes of all cameras. The intersections between the projected pyramids and the silhouettes, which are a set of hexagons, are computed. This process can be performed very efficiently with pre-computed polar silhouette graphs (PSGs) and reduced PSGs. The hexagons are then back-projected into the 3D space, where the intersections are calculated and the pent-model is derived. After that, a mesh surface model can be extracted by marching pentahedrons. Our algorithm has the combined advantages of robustness, speediness and preciseness. Experimental results based on both synthetic images and real photos are presented.     

关节轴角先验对3维人体重建结果的影响

目的 3维人体重建的目标在于建立真实可靠的3维人体模型。但目前基于SMPL（skinned multi-person linear model）模型重建3维人体的实验和一些公开数据集中，常常会出现预测的姿势角度值不符合真实人体关节角度规则的现象。针对这一问题，本文提出设置关节旋转角值域，使得重建的结果真实性更强、更符合人体关节机械结构。方法 根据人体关节的联接结构将各个关节的运动进行划分。根据划分结果计算关节运动自由度，并结合实际情况提出基于SMPL模型的关节旋转值域。提出一个简单的重建方法来验证值域分析的正确性。结果 使用3维人体数据集UP-3D进行相关实验，并对比以往直接根据学习结果生成重建模型的数据。在使用轴角作为损失参数的情况下，重建精度提高显著，平均误差降低15.1%。在使用所有损失函数后，平均误差比直接根据预测值生成重建模型的两段式重建方法降低7.0%。重建结果与UP-3D数据集进行真实性对比有显著的关节联动性效果。结论 本文提出的关节旋转角值域设置对基于SMPL模型进行3维人体重建的方法在进行关节点旋转角回归的过程中起到了很大作用，重建的模型也更符合人体关节运动联动性。     

Three-Dimensional Human Body Model Acquisition from Multiple Views

We present a novel approach to the three-dimensional human body model acquisition from three mutually orthogonal views. Our technique is based on the spatiotemporal analysis of the deforming apparent contour of a human moving according to a protocol of movements. For generality and robustness our technique does not use a prior model of the human body and a prior body part segmentation is not assumed. Therefore, our technique applies to humans of any anthropometric dimension. To parameterize and segment over time a deforming apparent contour, we introduce a new shape representation technique based on primitive composition. The composed deformable model allows us to represent large local deformations and their evolution in a compact and intuitive way. In addition, this representation allows us to hypothesize an underlying part structure and test this hypothesis against the relative motion (due to forces exerted from the image data) of the defining primitives of the composed model. Furthermore, we develop a Human Body Part Decomposition Algorithm (HBPDA) that recovers all the body parts of a subject by monitoring the changes over time to the shape of the deforming silhouette. In addition, we modularize the process of simultaneous two-dimensional part determination and shape estimation by employing the Supervisory Control Theory of Discrete Event Systems. Finally, we present a novel algorithm which selectively integrates the (segmented by the HBPDA) apparent contours from three mutually orthogonal viewpoints to obtain a three-dimensional model of the subject's body parts. The effectiveness of the approach is demonstrated through a series of experiments where a subject performs a set of movements according to a protocol that reveals the structure of the human body.     

An illumination distribution preserved colour substitution algorithm based on dichromatic reflection model

In this paper, we introduce a novel colour substitution algorithm based on the dichromatic reflection model. We separate the object colour and scene illumination information in the target area of the original image. The object colour was subsequently substituted while keeping the illumination information unchanged. A new method was employed to estimate the object colour. We also developed a set of new parameters to adjust the intensity distribution on the resultant image. Our algorithm is totally automatic and can achieve more realistic effects compared with other colour substitution methods.     

A Prism‐Free Method for Silhouette Rendering in Inverse Displacement Mapping

Silhouette is a key feature that distinguishes displacement mapping from normal mapping. However the silhouette rendering in the GPU implementation of displacement mapping (which is often called inversed displacement mapping) is tricky. Previous approaches rely mostly on construction of additional extruding prism‐like geometry, which slows down the rendering significantly. In this paper, we proposed a method for solving the silhouette rendering problem in inverse displace mapping without using any extruding prism‐like geometry. At each step of intersection finding, we continuously bends the viewing ray according to the current local tangent space associated with the surface. Thus, it allows mapping a displacement map onto an arbitrary curved surface with more accurate silhouette. While our method is simple, it offers surprisingly good results over Curved Relief Map (CRM) [ [OP05] ] in many difficult or degenerated cases.     

Generating Animatable 3D Virtual Humans from Photographs

We present an easy, practical and efficient full body cloning methodology. This system utilizes photos taken from the front, side and back of a person in any given imaging environment without requiring a special background or a controlled illuminating condition. A seamless generic body specified in the VRML H-Anim 1.1 format is used to generate an individualized virtual human. The system is composed of two major components: face-cloning and body-cloning. The face-cloning component uses feature points on front and side images and then applies DFFD for shape modification. Next a fully automatic seamless texture mapping is generated for 360° coloring on a 3D polygonal model. The body-cloning component has two steps: (i feature points specification, which enables automatic silhouette detection in an arbitrary background (ii two-stage body modification by using feature points and body silhouette respectively. The final integrated human model has photo-realistic animatable face, hands, feet and body. The result can be visualized in any VRML compliant browser.