卷积神经网络用于关节角度识别与姿势评估

为了快速准确地输出各种工作姿势风险评估结果，提出采用Kinect v2与卷积神经网络识别人体各关节角度，并输出标准姿势风险的评估得分。首先使用亚像素角点提取的棋盘标定算法标定Kinect两个摄像头，其次使用改进后的双边滤波对深度图像去噪，使用卷积神经网络识别人体关节二维位置，结合深度信息获取实际三维坐标并计算人体关节角度，最后输出姿势风险评估得分。通过两种实验分别验证了提出的Kinect角度识别与姿势评估的准确性，表明该方法关节角度识别与姿势风险评估的准确率均较高，是一种低成本、高可靠性的姿势评价方法，具有一定的科学意义和工程应用价值。     

Ergonomic postural assessment using a new open-source human pose estimation technology (OpenPose)

Observational ergonomic postural assessment methods have been commonly used to evaluate the risks of musculoskeletal disorders. Researchers have proposed semi-automatic methods using Kinect, known for limitations with body occlusions and non-frontal tracking. Meanwhile, new human pose estimation methods have been actively developed, and a popular open-source technology is OpenPose. This study aims to propose the OpenPose-based system for computing joint angles and RULA/REBA scores and validate against the reference motion capture system, and compare its performance to the Kinect-based system. Recordings of 10 participants performing 12 experimental tasks under different conditions: with/without body occlusions and tracked from frontal/non-frontal views were analyzed. OpenPose showed good performance under all task conditions, whereas Kinect performed significantly worse than OpenPose especially at cases with body occlusions or non-frontal tracking. The findings suggested that OpenPose could be a promising technology to measure joint angles and conduct semi-automatic ergonomic postural assessments in the real workspace where the conditions are often non-ideal.     

基于Kinect的三维重建技术在三维显示中的应用

三维显示技术能为用户提供更为真实的感官效果,在很多领域具有潜在的应用前景,现在高速发展的美容行业更是如此.提出将三维重建应用到高科技美容产品中,实现人脸检测结果的二维显示到三维显示的转换.首先需要对从Kinect获取的数据做预处理,包括修复、插值,再与彩色数据一起插值并进行2D、3D转换以实现数据对齐,最后将数据滤波并进行前后背景分离,利用PCL点云库进行点云显示.实验结果表明,基于Kinect的三维重建技术能够较好地恢复人脸的三维模型.     

基于深度扫描仪的高辨识度三维人体模型重建方法

3D照相打印馆人像的打印质量取决于3D扫描获得的三维人体模型的辨识度.然而,由于现有3D人体扫描仪价格昂贵、操作复杂等原因,使得3D人像打印成本高、耗时长和打印精度较低.针对这些缺点提出一种基于深度扫描仪重建高辨识度三维人体模型方法.利用多组深度扫描仪分工协作、优势互补,分别获取高辨识度的人体面部五官点云数据,上半身与全身表面轮廓点云数据.然后,通过引入特征点和改进的最近点迭代法将采集到的三组点云数据进行对齐、替换、拼接,将拼接后的无拓扑关系的点云数据进行曲面重构即可获得高辨识度的三维人体模型.该方法的扫描时间较短,以较低的成本构建了具有高辨识度的三维人像模型.     

Nrityabodha: Towards understanding Indian classical dance using a deep learning approach

Indian classical dance has existed since over 5000 years and is widely practised and performed all over the world. However, the semantic meaning of the dance gestures and body postures as well as the intricate steps accompanied by music and recital of poems is only understood fully by the connoisseur. The common masses who watch a concert rarely appreciate or understand the ideas conveyed by the dancer. Can machine learning algorithms aid a novice to understand the semantic intricacies being expertly conveyed by the dancer? In this work, we aim to address this highly challenging problem and propose deep learning based algorithms to identify body postures and hand gestures in order to comprehend the intended meaning of the dance performance. Specifically, we propose a convolutional neural network and validate its performance on standard datasets for poses and hand gestures as well as on constrained and real-world datasets of classical dance. We use transfer learning to show that the pre-trained deep networks can reduce the time taken during training and also improve accuracy. Interestingly, we show with experiments performed using Kinect in constrained laboratory settings and data from Youtube that it is possible to identify body poses and hand gestures of the performer to understand the semantic meaning of the enacted dance piece.     

Towards fully automatic reliable 3D acquisition: From designing imaging network to a complete and accurate point cloud

This paper describes a novel system for accurate 3D digitization of complex objects. Its main novelties can be seen in the new approach, which brings together different systems and tools in a unique platform capable of automatically generating an accurate and complete model for an object of interest. This is performed through generating an approximate model of the object, designing a stereo imaging network for the object with this model and capturing the images at the designed postures through exploiting an inverse kinematics method for a non-standard six degree of freedom robot. The images are then used for accurate and dense 3D reconstruction using photogrammetric multi-view stereo method in two modes, including resolving scale with baseline and with control points. The results confirm the feasibility of using Particle Swarm Optimization in solving inverse kinematics for this non-standard robot. The system provides this opportunity to test the effect of incidence angle on imaging network design and shows that the matching algorithms work effectively for incidence angle of 10°. The accuracy of the final point cloud generated with the system was tested in two modes through a comparison with a dataset generated with a close range 3D colour laser scanner.     

Comparison between low-cost marker-less and high-end marker-based motion capture systems for the computer-aided assessment of working ergonomics

The paper deals with the comparison between a high-end marker-based acquisition system and a low-cost marker-less methodology for the assessment of the human posture during working tasks. The low-cost methodology is based on the use of a single Microsoft Kinect V1 device. The high-end acquisition system is the BTS SMART that requires the use of reflective markers to be placed on the subject's body. Three practical working activities involving object lifting and displacement have been investigated. The operational risk has been evaluated according to the lifting equation proposed by the American National Institute for Occupational Safety and Health. The results of the study show that the risk multipliers computed from the two acquisition methodologies are very close for all the analysed activities. In agreement to this outcome, the marker-less methodology based on the Microsoft Kinect V1 device seems very promising to promote the dissemination of computer-aided assessment of ergonomics while maintaining good accuracy and affordable costs.     

Calibration of multiple kinects with little overlap regions

When using multiple Kinects, there must be enough distances among neighboring Kinects to avoid spoiled range data caused by the interference of their infrared speckle patterns. In the arrangement, their overlapped regions are too small to apply existing calibration methods using correspondences between their observations straightforwardly. Therefore, we propose a method to calibrate Kinects without large overlapped regions. In our method, first, we add extra RGB cameras in an environment to compensate overlapped regions. Thanks to them, we can estimate their camera parameters by obtaining correspondences between color images. Next, for accurate calibration, which considers range data as well as color images of Kinects, we optimize the estimated parameters by minimizing both the errors of correspondences between color images and those of range data of planar regions, which exist in a general environment such as walls and floors. Although our method consists of conventional techniques, its combination is optimized to achieve the calibration. © 2015 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.     

Region based foreground segmentation combining color and depth sensors via logarithmic opinion pool decision

In this paper we present a novel foreground segmentation system that combines color and depth sensors information to perform a more complete Bayesian segmentation between foreground and background classes. The system shows a combination of spatial-color and spatial-depth region-based models for the foreground as well as color and depth pixel-wise models for the background in a Logarithmic Opinion Pool decision framework used to correctly combine the likelihoods of each model. A posterior enhancement step based on a trimap analysis is also proposed in order to correct the precision errors that the depth sensor introduces. The results presented in this paper show that our system is robust in front of color and depth camouflage problems between the foreground object and the background, and also improves the segmentation in the area of the objects’ contours by reducing the false positive detections that appear due to the lack of precision of the depth sensors.     

低分辨率KINECT传感器采集三维点云的人脸识别

针对传统的三维人脸识别算法成本较高且不能很好地处理带有光照、表情等变化人脸识别的问题,设计了低分辨率Kinect传感器采集三维点云的鲁棒人脸识别系统。首先,通过鼻尖检测、人脸剪裁、姿势校正、对称填充及平滑采样得到规范的纹理图像;然后,在纹理图像上运用判别色彩空间变换,从而最大化类与类之间的分离性;最后,利用多模态稀疏编码有效地重建误差以得到查询图像与训练集之间的相似度,并利用Z-得分技术完成最终的人脸识别。在通用人脸数据库CurtinFaces、PIE及AR上的识别率可高达96.7%,实验结果表明,相比其它几种人脸识别算法,本文算法取得了更好的识别效果。