A memory-based model for planning target reach postures in the presence of obstructions

Existing posture prediction and motion simulation models generally lack the capability of simulating human obstruction avoidance during target reach. This compromises the utility of digital human models for ergonomics, as many design problems involve interactions between humans and obstructions. To address this problem, this paper presents a novel memory-based posture planning (MBPP) model, which plans reach postures that avoid obstructions. In this model, the task space is partitioned into small regions called cells. For a given human figure, each cell is linked to a memory that stores various alternative postures for reaching the cell. When a posture planning problem is given in terms of a target and an obstruction configuration, the model examines postures belonging to the relevant cell, selects collision-free ones and modifies them to exactly meet the hand target acquisition constraint. Simulation results showed that the MBPP model is capable of rapidly and robustly planning reach postures for various scenarios.     

Animating rising up from various lying postures and environments

Generating rising up motions is an important problem but has less been addressed in computer animation. This problem is challenging as rising motions involve complex motor skills and exhibit wide varieties due to various lying postures and environments. In this paper, we present an approach that utilizes motion planning and dynamics filtering to produce physically plausible rising motions. Our motion planning algorithm connects a given posture to a closest posture in a database of 14 rising motions. Then the dynamics filtering generates a physically plausible motion from a planned motion path. Our experiments show that a variety of motions of rising from various lying postures and different environments with obstacles can be generated easily by our approach.     

Real-Time Optimal Reach-Posture Prediction in a New Interactive Virtual Environment

Human posture prediction is a key factor for the design and evaluation of workspaces, in a virtual environment using virtual humans. This work presents a new interface and virtual environment for the direct human optimized posture prediction （D-HOPP） approach to predicting realistic reach postures of digital humans, where reach postures entail the use of the torso, arms, and neck. D-HOPP is based on the contention where depending on what type of task is being completed, and human posture is governed by different human performance measures. A human performance measure is a physics-based metric, such as energy or discomfort, and serves as an objective function in an optimization formulation. The problem is formulated as a single-objective optimization （SO0） problem with a single performance measure and as multiobjective-optimization （MOO） problem with multiple combined performance measures. We use joint displacement, change in potential energy, and musculoskeletal discomfort as performance measures. D-HOPP is equipped with an extensive yet intuitive user-interface, and the results are presented in an interactive virtual environment.     

Robotic milling posture adjustment under composite constraints: A weight-sequence identification and optimization strategy

Industrial robots are widely used for milling complex parts in restricted spaces owing to their multiple degrees of freedom and flexible postures. To plan posture trajectory for robot machining with high precision under multiple constraints, this study establishes composite constraint models with constraint boundary solutions. An improved gray relation analysis model is adopted to identify the weight-sequences among the composite constraints. The correlation degrees of the postures of the robot can be dynamically quantified between arbitrary cutter locations by applying weight sequence identification, which is conducive to fulfilling attractive orientations in artificial potential fields. In addition, this study proposes an initial posture determination strategy based on the optimization principle of minimizing the rotated energy in global postures. Consequently, an artificial potential planning model is applied to the implement posture adjustment of the robot end effector. During simulation and experimental validation, the proposed posture adjustment strategies with optimized initial postures and identified weight-sequences achieve a significant improvement in both the six-joint motion performance and machining precision quality in robotic milling.     

A statistical method for predicting automobile driving posture

A new model for predicting automobile driving posture is presented. The model, based on data from a study of 68 men and women in 18 vehicle package and seat conditions, is designed for use in posturing the human figure models that are increasingly used for vehicle interior design. The model uses a series of independent regression models, coupled with data-guided inverse kinematics, to fit a whole-body linkage. An important characteristic of the new model is that it places greatest importance on prediction accuracy for the body locations that are most important for vehicle interior design: eye location and hip location. The model predictions were compared with the driving postures of 120 men and women in five vehicles. Errors in mean eye location predictions in the vehicles were typically less than 10 mm. Prediction errors were largely independent of anthropometric variables and vehicle layout. Although the average posture of a group of people can be predicted accurately, individuals' postures cannot be predicted precisely because of interindividual posture variance that is unrelated to key anthropometric variables. The posture prediction models developed in this research can be applied to posturing computer-rendered human models to improve the accuracy of ergonomic assessments of vehicle interiors.     

低光照场景下基于序列增强的移动机器人人体检测与姿态识别

灾难救援、地下空间开发利用等场景均存在低光照、甚至完全黑暗的问题,导致机器人目标搜索与识别困难。为此,本文面向低光照场景提出基于红外深度相机图像序列的人体检测和姿态识别方法。首先,利用基于YOLO v4的AlphaPose算法检测人体框和关键点。然后,提出基于特征点匹配的漏检人体框恢复算法,降低人体漏检率,同时使用D-S（Dempster-Shafer）证据理论融合人体框和关键点的检测结果,从而降低人体误检率。最后,设计一种基于图像序列信息的人体姿态分层识别方法,在不同的识别层提取不同的人体躯干特征,利用连续多帧躯干向量特征组成的特征序列对人体姿态进行精准的识别并进行实验验证。实验结果表明本文算法能够在低光照条件下实现准确的人体检测与姿态识别,姿态识别准确率高达95.36%。     

Inverse Kinematics of 7-DOF Robots and Limbs by Decomposition and Approximation

The paper proposes a novel method for extremely fast inverse kinematics computation suitable for fast-moving manipulators and their path planning and for the animation of anthropomorphic limbs. In a preprocessing phase, the workspace of the robot is decomposed into small cells, and data sets for joint angle vectors (configurations) and hand positions/orientations (postures) are generated randomly in each cell using the forward kinematics. Due to the existence of multiple solutions for a desired posture, data classification is utilized to identify various solutions. The generated and classified data are used to determine the parameters of a simple linear or quadratic model that closely approximates the inverse kinematics within a cell. These parameters are stored in a lookup file. During the online phase, given the desired posture, the index of the appropriate cell is found, the model parameters are retrieved, and the joint angle vectors are computed. The advantages of the proposed method over the existing approaches are discussed. Data resulting from many trial runs are compiled for a manipulator and an anthropomorphic arm to demonstrate the performance of the proposed method.     

Optimization-based posture reconstruction for digital human models

Digital human modeling provides a valuable tool for designers when implemented early in the design process. Motion capture experiments offer a means of validation of the digital human simulation models. However, there is a gap between the motion capture experiments and the simulation models, as the motion capture results are marker positions in Cartesian space and the simulation model is based on joint space. Therefore, it is necessary to map the motion capture data to simulation models by employing a posture reconstruction algorithm. Posture reconstruction is an inherently redundant problem where the collective distance error between experimental joint centers and simulation joint centers is minimized. This paper presents an optimization-based method for determining an accurate and efficient solution to the posture reconstruction problem. The procedure is used to recreate 120 experimental postures. For each posture, the algorithm minimizes the distance between the simulation model joint centers and the corresponding experimental subject joint centers which is called the mean measurement error.     

面向服装CAD的多因素驱动人体模型变形技术研究

在以人体模型为基础的3维服装CAD系统中,为了获得设计师所需形体尺寸各异的人体模型,提出了一种多因素驱动人体模型变形的方法.该方法将人体模型的变形驱动因素分为尺寸因素、姿态因素和局部体形因素3类,并相应地提出了尺寸驱动、姿态驱动及局部体形驱动的人体模型变形算法.其中尺寸驱动采用基于截面环的 算法和基于模板插值的算法,通过对人体特征尺寸进行改变来驱动人体变形;姿态驱动是通过关节变形算法来实现特定姿态的变形,并通过建立姿态库,以实现快速姿态匹配;局部体形驱动包括基于特征面尺寸和基于特征面形状的调整,以得到具有局部特征的人体.实例表明,该方法可对已有的人体进行快速变形,以获得用户所需的人体模型,从而满足了服装CAD中对人体模型的多样性需求.     

Nonlinear inverse optimization approach for determining the weights of objective function in standing reach tasks

This paper presents a nonlinear inverse optimization approach to determine the weights for the joint displacement function in standing reach tasks. This inverse optimization problem can be formulated as a bi-level highly nonlinear optimization problem. The design variables are the weights of a cost function. The cost function is the weighted summation of the differences between two sets of joint angles (predicted posture and the actual standing reach posture). Constraints include the normalized weights within limits and an inner optimization problem to solve for joint angles (predicted standing reach posture). The weight linear equality constraints, obtained through observations, are also implemented in the formulation to test the method. A 52 degree-of-freedom (DOF) human whole body model is used to study the formulation and visualize the prediction. An in-house motion capture system is used to obtain the actual standing reach posture. A total of 12 subjects (three subjects for each percentile in stature of 5th percentile female, 50th percentile female, 50th percentile male and 95th percentile male) are selected to run the experiment for 30 tasks. Among these subjects one is Turkish, two are Chinese, and the rest subjects are Americans. Three sets of weights for the general standing reach tasks are obtained for the three zones by averaging all weights in each zone for all subjects and all tasks. Based on the obtained sets of weights, the predicted standing reach postures found using the direct optimization-based approach have good correlation with the experimental results. Sensitivity of the formulation has also been investigated in this study. The presented formulation can be used to determine the weights of cost function within any multi-objective optimization (MOO) problems such as any types of posture prediction and motion prediction.     

Nonlinear inverse optimization approach for determining the weights of objective function in standing reach tasks

This paper presents a nonlinear inverse optimization approach to determine the weights for the joint displacement function in standing reach tasks. This inverse optimization problem can be formulated as a bi-level highly nonlinear optimization problem. The design variables are the weights of a cost function. The cost function is the weighted summation of the differences between two sets of joint angles (predicted posture and the actual standing reach posture). Constraints include the normalized weights within limits and an inner optimization problem to solve for joint angles (predicted standing reach posture). The weight linear equality constraints, obtained through observations, are also implemented in the formulation to test the method. A 52 degree-of-freedom (DOF) human whole body model is used to study the formulation and visualize the prediction. An in-house motion capture system is used to obtain the actual standing reach posture. A total of 12 subjects (three subjects for each percentile in stature of 5th percentile female, 50th percentile female, 50th percentile male and 95th percentile male) are selected to run the experiment for 30 tasks. Among these subjects one is Turkish, two are Chinese, and the rest subjects are Americans. Three sets of weights for the general standing reach tasks are obtained for the three zones by averaging all weights in each zone for all subjects and all tasks. Based on the obtained sets of weights, the predicted standing reach postures found using the direct optimization-based approach have good correlation with the experimental results. Sensitivity of the formulation has also been investigated in this study. The presented formulation can be used to determine the weights of cost function within any multi-objective optimization (MOO) problems such as any types of posture prediction and motion prediction.     

Applying incremental Deep Neural Networks-based posture recognition model for ergonomics risk assessment in construction

Monitoring and assessing awkward postures is a proactive approach for Musculoskeletal Disorders (MSDs) prevention in construction. Machine Learning models have shown promising results when used in recognition of workers’ posture from Wearable Sensors. However, there is a need to further investigate: i) how to enable Incremental Learning, where trained recognition models continuously learn new postures from incoming subjects while controlling the forgetting of learned postures; ii) the validity of ergonomics risk assessment with recognized postures. The research discussed in this paper seeks to address this need through an adaptive posture recognition model– the incremental Convolutional Long Short-Term Memory (CLN) model. The paper discusses the methodology used to develop and validate this model’s use as an effective Incremental Learning strategy. The evaluation was based on real construction workers’ natural postures during their daily tasks. The CLN model with “shallow” (up to two) convolutional layers achieved high recognition performance (Macro F1 Score) under personalized (0.87) and generalized (0.84) modeling. Generalized CLN model, with one convolutional layer, using the “Many-to-One” Incremental Learning scheme can potentially balance the performance of adaptation and controlling forgetting. Applying the ergonomics rules on recognized and ground truth postures yielded comparable risk assessment results. These findings support that the proposed incremental Deep Neural Networks model has a high potential for adaptive posture recognition. They can be deployed alongside ergonomics rules for effective MSDs risk assessment.     

Human shape correspondence with automatically predicted landmarks

We consider the problem of computing accurate point-to-point correspondences among a set of human bodies in similar posture using a landmark-free approach. The approach learns the locations of the anthropometric landmarks present in a database of human models in similar postures and uses this knowledge to automatically predict the locations of these anthropometric landmarks on a newly available scan. The predicted landmarks are then used to compute point-to-point correspondences between a template model and the newly available scan. This study conducts a large-scale evaluation to examine the accuracy of the computed correspondences. Furthermore, we show that the correspondences are accurate enough for the application of motion transfer.     

Human-and-Humanoid Postures Under External Disturbances: Modeling,Simulation, and Robustness. Part 1: Modeling

It is a well-known fact that the growth of technology has radically changed our approach to biosciences and medicine. What is interesting is that in the last decade we have witnessed a reverse influence—a trend towards “biologically inspired” solutions to technical problems. This leads to a true symbiosis between bio and technical sciences. A good example is the intersection and overlapping of three distinct fields: sports, medicine, and robotics. This paper intends to apply sophisticated methods developed for mathematical modeling of humanoid robots in real human motions, particularly in posture stabilization and selection of appropriate postures for different situation in sports and every day life. A general simulation system is realized: following a deductive principle, the algorithm considers particular human/humanoid motions (like those occurring in different sports) as being just special cases of a general motion and impact theory. Simulation includes the interaction with the environment. Simulating a human/humanoid dynamics in a given task, all relevant characteristics could be found: trajectories, velocities and accelerations, loads of joints, power requirements, energy consumption, contact forces including ground reactions, impact effects, etc. Simulation is used in solving a problem that is important for both humans and humanoid robots, namely, the behavior of a posture (keeping stability or collapsing) when subject to different disturbances. Although “posture” is mainly a static term, maintaining its balance in the presence of disturbances is a truly dynamic problem. Typical postures from every day life and sports are considered, such as: upright standing, squat (and partial squat), and three karate postures. Two sorts of disturbances are applied to eventually, compromise the posture: external impulse and permanent external force. This paper does not aim to suggest some new control strategy but to develop the dynamic model and simulation algorithm, and apply them to compare the robustness of different postures to external disturbances.     

Dynamic grasp planning of multifingered robot hands based onasymptotic stability

We describe an approach for planning grasps of multifingered robot hands based on a small vibration model. Using features of the grasp configuration, we analyze asymptotic stability, contact situations, and uniaxial fingertip force constraints for the combined planning of finger posture and finger position, and characterize the generalized mass, damping, and stiffness. Choosing the largest time constant of the vibration model as an optimization criterion for planning finger postures and positions, the original problem of dynamic grasp planning is formulated as a nonlinear program. Simulation examples for a three-fingered robot hand grasping a spherical object demonstrate the effectiveness of the approach.     

Accumulated risk of body postures in assembly line balancing problem and modeling through a multi-criteria fuzzy-genetic algorithm

Monotonous body postures during repetitive jobs negatively affect assembly-line workers with the developing of Work-related Musculoskeletal Disorders (WMSDs). Ergonomics specialists have offered auxiliary posture diversity to deal with the lack of varieties, especially for high-risk ones. Meanwhile, Assembly Line Balancing (ALB) problem has been recognized as a prior thinking to (re)configure assembly lines via the balancing of their tasks among their workstations. Some conventional criteria, cycle time and overall workload are often considered during the balancing. This paper presents a novel model of ALB problem that incorporates assembly worker postures into the balancing. In addition to the conventional ALB criteria, a new criterion of posture diversity is defined and contributes to enhance the model. The proposed model suggests configurations of assembly lines via the balancing; so that the assigned workers encounter the opportunities of changing their body postures, regularly. To address uncertainties in the conventional criteria, a fuzzy goal programming is used, and an appropriate genetic algorithm is developed to deal with the model. Various computational tests are performed on the different models made with combinations of the three criteria mentioned above. Comparing the pay-offs among the combinations, results show that well balanced task allocation can be obtained through the proposed model.     

基于长短时记忆网络的人体姿态检测方法

针对在循环神经网络（RNN）网络结构下较为遥远的历史信号无法传递至当前时刻的问题,长短时记忆（LSTM）网络作为RNN的一种变体被提出,在继承RNN对时间序列优秀的记忆能力的前提下,LSTM克服了这种时间序列的长期依赖问题,并在自然语言处理与语音识别领域有较好的表现。对于人体行为动作中也存在作为时间序列的长期依赖问题与使用传统滑窗算法采集数据时造成的无法实时检测的问题,将LSTM扩展应用到人体姿态检测,提出了基于LSTM的人体姿态检测方法。通过目前智能手机中一般都带有的加速度传感器、陀螺仪、气压计和方向传感器实时采集的时序数据,制作了包含3336条带有人工标注数据的人体姿态数据集,对行走、奔跑、上楼梯、下楼梯和平静五种日常持续性行为姿态与跌倒、起立、坐下和跳跃这四个突发行为姿态进行预测分类。对比LSTM网络与该研究领域内常用的浅层学习算法、深度学习全连接神经网络与卷积神经网络,实验结果表明,所提方法使用端对端的深度学习的方法相比基于所制作数据集的人体姿态检测算法模型的正确率提高了4.49个百分点,验证了该网络结构的泛化能力且更适合姿态检测。     

Recovering 3D human pose based on biomechanical constraints,postures comfort and image shading

This paper presents a new model to identify 3D human poses in pictures, given a single input image. The proposed approach is based on a well known model found in the literature, including improvements in terms of biomechanical restrictions aiming to reduce the number of 3D possible postures that correctly represent the pose in the 2D image. Since the generated set of poses can have more than one possible posture, we propose a ranking system in order to suggest the best generated postures according to a “comfort” criterion and shading characteristics in the image as well. The comfort criterion adopts assumptions in terms of pose equilibrium, while the shading criterion eliminates the ambiguities of postures taken into account the image illumination. We must emphasize that the removal of ambiguous 3D poses related to a single image is the main focus of this work. The achieved results were analyzed w.r.t. visual inspection of users as well as a state of the art technique and indicate that our model contributed in some way to the solution of that challenge problem.     

基于改进分段铰链变换的人体重建技术

目前,基于单帧图像的人体建模还不能有效地处理手臂、衣服等对身体部位的遮 挡,以及因视角带来的自我身体遮挡等复杂的遮挡问题。为此,利用SMPL 模型骨骼关节分布 特点,提出改进传统分段铰链变换模型的人体重建方法。该方法运用骨骼关节的精确标注确定 模型变换的节点,结合图像轮廓边界约束图,提出前向分段回归概率期望最小化(FPR-PEM)的 柔性配准方法。通过迭代模型对变形关节处结合薄板样条进行线性插值,保证模型表面点云形 状的独立性,有效地注册各种姿势下的非刚性变形模型,较好地解决了复杂遮挡带来的重建挑 战,并进行模型姿态回归调整,实现准确的人体建模。实验结果表明,方法可以有效实现精细 和平滑模型的人体表面重建。