Experimenting with nonintrusive motion capture in a virtual environment

A growing number of promising applications requires recognizing human posture and motion. Conventional techniques require us to attach foreign objects to the body, which in some applications is disturbing or even impossible. New, nonintrusive motion capture approaches are called for. The well-known shape-from-silhouette technique for understanding 3D shapes could also be effective for human bodies. We present a novel technique for model-based motion capture that uses silhouettes extracted from multiple views. A 3D reconstruction of the performer can be computed from a silhouette with a technique known as volume intersection. We can recover the posture by fitting a model of the human body to the reconstructed volume. The purpose of this work is to test the effectiveness of this approach in a virtual environment by investigating the precision of the posture and motion obtained with various numbers and arrangements of stationary cameras. An average 1% position error has been obtained with five cameras.     

3D Posture Reconstruction and Human Animation from 2D Feature Points

An optimal approach is proposed in this paper for posture reconstruction and human animation from 2D feature points extracted from the monocular images containing human motions. Biomechanical constraints are encoded in every joint of the adopted 3D skeletal human model to make sure that each state of the joints represents a physically valid posture. Size of the human model is adjusted to be consistent with the human figure represented by feature points. Energy Function is defined to represent the residuals between the extracted 2D feature points and the corresponding features resulted from projection of the 3D human model. Local Adjustment and Global Adjustment procedures are proposed to place the joints and body segments into proper locations and orientations in 3D space to create the posture with the minimum value of Energy Function. To find the optimal solution of the ill‐posed recovery problem from 2D to 3D, Genetic Algorithm is employed in the high‐dimensional parameter space by considering all the parameters simultaneously. Smooth and continuous changes between consecutive frames are considered in development of the human animation procedure. The proposed approach produces optimal reconstruction results of any possible human postures and movements. It is different from classical kinematics and dynamics formulations, and is an attempt to bridge the gap between computer vision and computer animation in human motion study.     

三维人体点云模型多约束肢体分割

针对人体点云模型的肢体分割这一动作识别和虚拟重建领域的重要问题,提出了一种基于分类骨架线、测地距离、特征点和姿态分析的多约束肢体分割算法,通过生成点云模型的分类骨架线,配合测地距离获得人体各部位粗分割点云集,利用测地路径方法实现关键特征点的定位,并利用曲线拟合方式进行定位优化,针对头颈、上肢、下肢和躯干之间关联部位的解剖学特征,构造多种约束条件,对各部位粗分割点云集进行了优化再分割。实验结果表明,所提算法对站姿条件下的不同动作、不同体型、不同精度人体点云模型均能取得与视觉理解相吻合的分割效果。通过该算法得到的肢体各部分点云数据可用于姿态分析等后续研究。     

Detailed wrinkle generation of virtual garments from a single image

The presence of proper wrinkles is important while modeling realistic virtual garments. Unlike previously used full 3D information methods, our approach achieves detailed garment generation from a single image. First, we retrieve a garment image similar to the initial virtual garment based on content-based image retrieval (CBIR) method. Then, we preprocess the image with a combination of human body reshaping, image segmentation and shape recovery, to obtain the 3D wrinkle details. Finally, the garment height are synthesized into the virtual garment. For better suit the posture of the human body, excess garment energy are released to remove the unmatched wrinkles. We apply our method to various styles of virtual garments, and it enable virtual characters in general pose to be dressed in these garments and complete wrinkle generation. Compared with existing garment modeling methods, the experimental results show that the proposed method could quickly capture the realistic wrinkles of virtual garments with less manual operation and achieve more realistic wrinkles for virtual garments.

     

人体表面点云数据的拓扑特征检测与自动分割

对标准站立测量姿态下的人体表面点云数据的拓扑特征检测与自动分割进行了研究,提出基于全景深度图像表示的人体点云表面拓扑特征检测和自动分割新方法。首先把人体表面的点云数据转换为圆柱极坐标形式,获得人体扫描表面的全景深度图像表示,根据全景深度图像中的层次信息自动检测人体表面的拓扑特征,并根据拓扑特征把人体分割成5个功能结构。实验证明这种方法改进了人体表面点云数据的拓扑特征检测和自动分割的效率和精度。     

Analyzing human gait and posture by combining feature selection and kernel methods

This paper evaluates a set of computational algorithms for the automatic estimation of human postures and gait properties from signals provided by an inertial body sensor. The use of a single sensor device imposes limitations for the automatic estimation of relevant properties, like step length and gait velocity, as well as for the detection of standard postures like sitting or standing. Moreover, the exact location and orientation of the sensor are also a common restriction that is relaxed in this study.Based on accelerations provided by a sensor, known as the ‘9×2’, three approaches are presented extracting kinematic information from the user motion and posture. First, a two-phases procedure implementing feature extraction and support vector machine based classification for daily living activity monitoring is presented. Second, support vector regression is applied on heuristically extracted features for the automatic computation of spatiotemporal properties during gait. Finally, sensor information is interpreted as an observation of a particular trajectory of the human gait dynamical system, from which a reconstruction space is obtained, and then transformed using standard principal components analysis, finally support vector regression is used for prediction.Daily living activities are detected and spatiotemporal parameters of human gait are estimated using methods sharing a common structure based on feature extraction and kernel methods. The approaches presented are susceptible to be used for medical purposes.     

基于足底压力的人体姿态检测和行为分析方法

人体的运动姿态检测和行为分析具有广泛应用价值.设计了一种基于足底压力的人体姿态检测和行为分析系统,系统由压力采集模块、数据处理模块、无线通信模块和行为分析软件组成.姿态检测系统获取足底压力数据,采用蓝牙方式进行数据通信,行为分析软件使用支持向量机多分类方法实现坐、站、走、跑和爬楼5个经典人体姿态的区分.实验证明该系统对人体姿态具有较好的识别精度和可靠性,可以用于人体姿态检测和行为分析.     

基于姿态校正与姿态融合的2D/3D骨架动作识别方法

针对现有的人体骨架动作识别方法对肢体信息挖掘不足以及时间特征提取不足的问题,提出了一种基于姿态校正模块与姿态融合模块的模型PTF-SGN,实现了对骨架图关键时空信息的充分利用。首先,对骨架图数据进行预处理,挖掘肢体和关节点的位移信息并提取特征;然后,姿态校正模块通过无监督学习的方式获取姿态调整因子,并对人体姿态进行自适应调整,增强了模型在不同环境下的鲁棒性;其次,提出一种基于时间注意力机制的姿态融合模块,学习骨架图中的短时刻特征与长时刻特征并融合长短时刻特征,加强了对时间特征的表征能力;最后,将骨架图的全局时空特征输入到分类网络中得到动作识别结果。在NTU60 RGB+D、NTU120 RGB+D两个3D骨架数据集和Penn-Action、HARPET两个2D骨架数据集上的实验结果表明,该模型能够有效地识别骨架时序数据的动作。     

关节轴角先验对3维人体重建结果的影响

目的 3维人体重建的目标在于建立真实可靠的3维人体模型。但目前基于SMPL（skinned multi-person linear model）模型重建3维人体的实验和一些公开数据集中，常常会出现预测的姿势角度值不符合真实人体关节角度规则的现象。针对这一问题，本文提出设置关节旋转角值域，使得重建的结果真实性更强、更符合人体关节机械结构。方法 根据人体关节的联接结构将各个关节的运动进行划分。根据划分结果计算关节运动自由度，并结合实际情况提出基于SMPL模型的关节旋转值域。提出一个简单的重建方法来验证值域分析的正确性。结果 使用3维人体数据集UP-3D进行相关实验，并对比以往直接根据学习结果生成重建模型的数据。在使用轴角作为损失参数的情况下，重建精度提高显著，平均误差降低15.1%。在使用所有损失函数后，平均误差比直接根据预测值生成重建模型的两段式重建方法降低7.0%。重建结果与UP-3D数据集进行真实性对比有显著的关节联动性效果。结论 本文提出的关节旋转角值域设置对基于SMPL模型进行3维人体重建的方法在进行关节点旋转角回归的过程中起到了很大作用，重建的模型也更符合人体关节运动联动性。     

Human Body Posture Classification by a Neural Fuzzy Network and Home Care System Application

A new classification approach for human body postures based on a neural fuzzy network is proposed in this paper, and the approach is applied to detect emergencies that are caused by accidental falls. Four main body postures are used for posture classification, including standing, bending, sitting, and lying. After the human body is segmented from the background, the classification features are extracted from the silhouette. The body silhouette is projected onto horizontal and vertical axes, and then, a discrete Fourier transform is applied to each projected histogram. Magnitudes of significant Fourier transform coefficients together with the silhouette length-width ratio are used as features. The classifier is designed by a neural fuzzy network. The four postures can be classified with high accuracy according to experimental results. Classification results are also applicable to home care emergency detection of a person who suddenly falls and remains in the lying posture for a period of time due to experiments that were performed.     

Nonlinear inverse optimization approach for determining the weights of objective function in standing reach tasks

This paper presents a nonlinear inverse optimization approach to determine the weights for the joint displacement function in standing reach tasks. This inverse optimization problem can be formulated as a bi-level highly nonlinear optimization problem. The design variables are the weights of a cost function. The cost function is the weighted summation of the differences between two sets of joint angles (predicted posture and the actual standing reach posture). Constraints include the normalized weights within limits and an inner optimization problem to solve for joint angles (predicted standing reach posture). The weight linear equality constraints, obtained through observations, are also implemented in the formulation to test the method. A 52 degree-of-freedom (DOF) human whole body model is used to study the formulation and visualize the prediction. An in-house motion capture system is used to obtain the actual standing reach posture. A total of 12 subjects (three subjects for each percentile in stature of 5th percentile female, 50th percentile female, 50th percentile male and 95th percentile male) are selected to run the experiment for 30 tasks. Among these subjects one is Turkish, two are Chinese, and the rest subjects are Americans. Three sets of weights for the general standing reach tasks are obtained for the three zones by averaging all weights in each zone for all subjects and all tasks. Based on the obtained sets of weights, the predicted standing reach postures found using the direct optimization-based approach have good correlation with the experimental results. Sensitivity of the formulation has also been investigated in this study. The presented formulation can be used to determine the weights of cost function within any multi-objective optimization (MOO) problems such as any types of posture prediction and motion prediction.     

Nonlinear inverse optimization approach for determining the weights of objective function in standing reach tasks

This paper presents a nonlinear inverse optimization approach to determine the weights for the joint displacement function in standing reach tasks. This inverse optimization problem can be formulated as a bi-level highly nonlinear optimization problem. The design variables are the weights of a cost function. The cost function is the weighted summation of the differences between two sets of joint angles (predicted posture and the actual standing reach posture). Constraints include the normalized weights within limits and an inner optimization problem to solve for joint angles (predicted standing reach posture). The weight linear equality constraints, obtained through observations, are also implemented in the formulation to test the method. A 52 degree-of-freedom (DOF) human whole body model is used to study the formulation and visualize the prediction. An in-house motion capture system is used to obtain the actual standing reach posture. A total of 12 subjects (three subjects for each percentile in stature of 5th percentile female, 50th percentile female, 50th percentile male and 95th percentile male) are selected to run the experiment for 30 tasks. Among these subjects one is Turkish, two are Chinese, and the rest subjects are Americans. Three sets of weights for the general standing reach tasks are obtained for the three zones by averaging all weights in each zone for all subjects and all tasks. Based on the obtained sets of weights, the predicted standing reach postures found using the direct optimization-based approach have good correlation with the experimental results. Sensitivity of the formulation has also been investigated in this study. The presented formulation can be used to determine the weights of cost function within any multi-objective optimization (MOO) problems such as any types of posture prediction and motion prediction.     

基于半监督聚类的3维肢体分割算法

行为分析已经成为计算机视觉研究领域的热点,行为主体的肢体部件分割是行为分析中很重要的一部分同时也是一个难点问题,为了对3维肢体进行有效分割,提出了一种基于半监督聚类的肢体分割算法。该算法首先利用前一帧姿势估计反馈的时域信息来对3维主体进行初始的肢体部件分割;然后根据人体结构信息进一步确定行为主体上各个点与肢体部件之间的关系来得到半监督聚类的初始值;之后基于各个肢体部件的形状信息进行半监督聚类,迭代求解肢体部件分割的最优解;最后利用分割后的各个肢体部件进行行为主体的姿势参数估计。通过对IXMAS数据库中6种行为下的6个行为主体进行的肢体部件的分割实验结果证明,该新算法具有主体适应性、视角适应性以及行为适应性。     

异常步态3维人体建模和可变视角识别

目的 运用视觉和机器学习方法对步态进行研究已成为当前热点,但多集中在身份识别领域。本文从不同的视角对其进行研究,探讨一种基于点云数据和人体语义特征模型的异常步态3维人体建模和可变视角识别方法。方法 运用非刚性变形和蒙皮方法,构建基于形体和姿态语义特征的参数化3维人体模型;以红外结构光传感器获取的人体异常步态点云数据为观测目标,构建其对应形体和姿态特征的3维人体模型。通过ConvGRU（convolution gated necurrent unit）卷积循环神经网络来提取其投影深度图像的时空特征,并将样本划分为正样本、负样本和自身样本三元组,对异常步态分类器进行训练,以提高分类器对细小差异的鉴别能力。同时对异常步态数据获取难度大和训练视角少的问题,提出了一种基于形体、姿态和视角变换的训练样本扩充方法,以提高模型在面对视角变化时的泛化能力。结果 使用CSU（Central South University）3维异常步态数据库和DHA（depth-included human action video）深度人体行为数据库进行实验,并对比了不同异常步态或行为识别方法的效果。结果表明,本文方法在CSU异常步态库实验中,0°、45°和90°视角下对异常步态的综合检测识别率达到了96.6%,特别是在90°到0°交叉和变换视角实验中,比使用DMHI（difference motion history image）和DMM-CNN（depth motion map-convolutional neural network）等步态动作特征要高出25%以上。在DHA深度人体运动数据库实验中,本文方法识别率接近98%,比DMM等相关算法高出2%~3%。结论 提出的3维异常步态识别方法综合了3维人体先验知识、循环卷积网络的时空特性和虚拟视角样本合成方法的优点,不仅能提高异常步态在面对视角变换时的识别准确性,同时也为3维异常步态检测和识别提供一种新思路。     

Subjective response of standing persons exposed to fore-aft,lateral and vertical whole-body vibration

The aims of this study were to propose multiply scale factors for evaluation of discomfort of standing persons and to investigate whether there exist differences between multiplying factors used for evaluation of discomfort of standing persons and those of seated persons exposed to WBV. Twelve male subjects were exposed to twenty-seven stimuli that comprise three acceleration magnitudes (0.2, 0.4, and 0.8 m/s² r.m.s.) along fore-aft (x), lateral (y) or vertical (z) direction. The subjects with seated or standing posture on the platform of the vibration test rig rated the subjective discomfort for each stimulus that has frequency contents ranging from 1.0 Hz to 20 Hz with a constant power spectrum density. The order of presentation of the test stimuli was fully randomized and each stimulus was repeated three times. The subjective scale for discomfort was calculated by using the category judgment method. The best combinations of multiplying factors were determined by calculating correlation coefficients of regression curves in-between subjective ratings and vibration magnitudes. In all the directions, body posture significantly influenced on subjective discomfort scales. Particularly in the fore-aft and lateral direction, the upper limit of all the categories for the standing posture resulted in higher vibration acceleration magnitudes than those for the seated posture. In contrast, in the vertical direction, only the upper limit of category “1: Not uncomfortable” for standing posture was observed to be higher than that for seated posture. The best agreement for ISO-weighted vibration acceleration occurred at x factor of 1.8 and y factor of 1.8 in the standing posture and x factor of 2.8 and y factor of 1.8 in the seated posture. The results suggest that seated people respond more sensitively and severely in perception of discomfort to fore-aft and lateral vibration than standing people do while standing people respond more sensitively and severely to vertical vibration than seated people do. Thus the effects of body postures on multiplying factors should be considered in evaluation of discomfort caused by whole-body vibration.Relevance to industryThis study reports differences in subjective response of standing persons to fore-aft, lateral and vertical whole-body vibration. The results obtained in this study propose the fundamental data on the sensitivity to whole-body vibration exposed with standing posture.     

面向服装CAD的多因素驱动人体模型变形技术研究

在以人体模型为基础的3维服装CAD系统中,为了获得设计师所需形体尺寸各异的人体模型,提出了一种多因素驱动人体模型变形的方法.该方法将人体模型的变形驱动因素分为尺寸因素、姿态因素和局部体形因素3类,并相应地提出了尺寸驱动、姿态驱动及局部体形驱动的人体模型变形算法.其中尺寸驱动采用基于截面环的 算法和基于模板插值的算法,通过对人体特征尺寸进行改变来驱动人体变形;姿态驱动是通过关节变形算法来实现特定姿态的变形,并通过建立姿态库,以实现快速姿态匹配;局部体形驱动包括基于特征面尺寸和基于特征面形状的调整,以得到具有局部特征的人体.实例表明,该方法可对已有的人体进行快速变形,以获得用户所需的人体模型,从而满足了服装CAD中对人体模型的多样性需求.     

三维人体姿态估计研究综述

三维人体姿态估计在本质上是一个分类问题和回归问题,主要通过图像估计人体的三维姿态。基于传统方法和深度学习方法的三维人体姿态估计是当前研究的主流方法。按照传统方法到深度学习方法的顺序对近年来三维人体姿态估计方法进行系统介绍,从而了解传统方法通过生成和判别等方法得到人体姿态的众多要素完成三维人体姿态的估计。基于深度学习的三维人体姿态估计方法主要通过构建神经网络,从图像特征中回归出人体姿态信息,大致可以分为基于直接回归方法、基于2D信息方法和基于混合方法的三维人体姿态估计这三类。最后对当前三维人体姿态估计研究所面临的困难与挑战进行阐述,并对未来的研究趋势做出展望。     

基于sift特征匹配的人体上半身三维运动跟踪

研究人体姿态与视频优化跟踪问题,单目视频缺少深度信息,使得单目视频的人体运动跟踪难以实现三维姿态恢复问题。为解决上述问题,提出了一种利用sift特征尺度不变性的优点进行人体上半身三维运动跟踪的算法。在跟踪过程中先计算初始匹配sift特征点对,然后反复迭代出除误匹配点,消除误差,最后求解由两个匹配sift特征组成的方程组得到胸部关节的位姿,根据人体骨骼模型采用深度遍历依次恢复其它关节的姿态。实验结果表明,系统能够对人体上半身运动进行比较准确的三维运动跟踪。     

一种新的异常行为检测算法

行人异常行为的自动检测与识别是计算机视觉领域的重点和难点,同时也是智能监控系统中研究的热点问题。针对这一问题,提出了一种基于人体形态特征的异常检测算法。利用轮廓信息将目标从视频序列中分割出来,再对分割出来的目标进行轮廓拟合,根据所得到的拟合信息提取文中所定义的形态特征因子,将特征因子经过行为分类器的判定,从而决策出该行为是否异常。实验结果表明该方法实现简单,具有较好的实时性与鲁棒性,可以作为实时监控系统中异常行为检测的有效方法。     

Estimation of 3-D human body posture via co-registration of 3-D human model and sequential stereo information

In this paper, we present a technique for estimating three-dimensional (3-D) human body posture from a set of sequential stereo images. We estimated the pixel displacements of stereo image pairs to reconstruct 3-D information. We modeled the human body with a set of ellipsoids connected by kinematic chains and parameterized with rotational angles at each body joint. To estimate human posture from the 3-D data, we developed a new algorithm based on expectation maximization (EM) with two-step iterations, assigning the 3-D data to different body parts and refining the kinematic parameters to fit the 3-D model to the data. The algorithm is iterated until it converges on the correct posture. Experimental results with synthetic and real data demonstrate that our method is capable of reconstructing 3-D human posture from stereo images. Our method is robust and generic; any useful information for locating the body parts can be integrated into our framework to improve the outcomes.