Natural metrics and least-committed priors for articulated tracking

In articulated tracking, one is concerned with estimating the pose of a person in every frame of a film. This pose is most often represented as a kinematic skeleton where the joint angles are the degrees of freedom. Least-committed predictive models are then phrased as a Brownian motion in joint angle space. However, the metric of the joint angle space is rather unintuitive as it ignores both bone lengths and how bones are connected. As Brownian motion is strongly linked with the underlying metric, this has severe impact on the predictive models. We introduce the spatial kinematic manifold of joint positions, which is embedded in a high dimensional Euclidean space. This Riemannian manifold inherits the metric from the embedding space, such that distances are measured as the combined physical length that joints travel during movements. We then develop a least-committed Brownian motion model on the manifold that respects the natural metric. This model is expressed in terms of a stochastic differential equation, which we solve using a novel numerical scheme. Empirically, we validate the new model in a particle filter based articulated tracking system. Here, we not only outperform the standard Brownian motion in joint angle space, we are also able to specialise the model in ways that otherwise are both difficult and expensive in joint angle space.     

Automatic reconstruction of 3D human motion pose from uncalibrated monocular video sequences based on markerless human motion tracking

We present a method to reconstruct human motion pose from uncalibrated monocular video sequences based on the morphing appearance model matching. The human pose estimation is made by integrated human joint tracking with pose reconstruction in depth-first order. Firstly, the Euler angles of joint are estimated by inverse kinematics based on human skeleton constrain. Then, the coordinates of pixels in the body segments in the scene are determined by forward kinematics, by projecting these pixels in the scene onto the image plane under the assumption of perspective projection to obtain the region of morphing appearance model in the image. Finally, the human motion pose can be reconstructed by histogram matching. The experimental results show that this method can obtain favorable reconstruction results on a number of complex human motion sequences.     

基于模糊神经网络的手势识别

本文通过对人手结构和手关节运动的分析,建立了手模型及其相应的数据结构。     

Robust Articulated‐ICP for Real‐Time Hand Tracking

We present a robust method for capturing articulated hand motions in realtime using a single depth camera. Our system is based on a realtime registration process that accurately reconstructs hand poses by fitting a 3D articulated hand model to depth images. We register the hand model using depth, silhouette, and temporal information. To effectively map low‐quality depth maps to realistic hand poses, we regularize the registration with kinematic and temporal priors, as well as a data‐driven prior built from a database of realistic hand poses. We present a principled way of integrating such priors into our registration optimization to enable robust tracking without severely restricting the freedom of motion. A core technical contribution is a new method for computing tracking correspondences that directly models occlusions typical of single‐camera setups. To ensure reproducibility of our results and facilitate future research, we fully disclose the source code of our implementation.     

A joint-constraint model for human joints using signed distance-fields

We present a local joint-constraint model for a single joint which is based on distance fields. Our model is fast, general, and well suited for modeling human joints. In this work, we take a geometric approach and model the geometry of the boundary of the feasible region, i.e., the boundary of all allowed poses. A region of feasible poses can be built by embedding motion captured data points in a signed distance field. The only assumption is that the feasible poses form a single connected set of angular values. We show how signed distance fields can be used to generate fast and general joint-constraint models for kinematic figures. Our model is compared to existing joint-constraint models, both in terms of generality and computational cost. The presented method supports joint-constraints of up to three degrees of freedom and works well with sampled motion data. Our model can be extended to handle inter-joint dependencies, or joints with more than three degrees of freedom. The resolution of the joint-constraints can be tweaked individually for each degree of freedom, which can be used to optimize memory usage. We perform a comparative study of the key-properties of various joint-constraint models, as well as a performance study of our model compared to the fastest alternative, the box limit model. The study is performed on the shoulder joint, using a motion captured jumping motion as reference.     

Influence of selected modeling and computational issues on muscle force estimates

Knowledge of muscle forces and joint reaction forces during human movement can provide insight into the underlying control and tissue loading. Since direct measurement of the internal loads is generally not feasible, non-invasive methods based on musculoskeletal modeling and computer simulations have been extensively developed. By applying observed motion data to the musculoskeletal models, inverse dynamic analysis allow to determine the resultant joint torques, transformed then into estimates of individual muscle forces by means of different optimization procedures. Assessment of the joint reaction forces and other internal loads is further possible. Comparison of the muscle force estimates obtained for different modeling assumptions and parameters in the model can be valuable for the improvement of validity of the model-based estimations. The present study is another contribution to this field. Using a sagittal plane model of an upper limb with a weight carried in hand, and applying the data of recorded flexion and extension movement of the upper limb, the resultant muscular forces are predicted using different modeling assumptions and simulation tools. This study relates to different coordinates (joint and natural coordinates) used to built the mathematical model, muscle path modeling, muscle decomposition (change in number of the modeled muscles), and different optimization methods used to share the joint torques into individual muscles.     

Real-Time Physics-Based 3D Biped Character Animation Using an Inverted Pendulum Model

We present a physics-based approach to generate 3D biped character animation that can react to dynamical environments in real time. Our approach utilizes an inverted pendulum model to online adjust the desired motion trajectory from the input motion capture data. This online adjustment produces a physically plausible motion trajectory adapted to dynamic environments, which is then used as the desired motion for the motion controllers to track in dynamics simulation. Rather than using Proportional-Derivative controllers whose parameters usually cannot be easily set, our motion tracking adopts a velocity-driven method which computes joint torques based on the desired joint angular velocities. Physically correct full-body motion of the 3D character is computed in dynamics simulation using the computed torques and dynamical model of the character. Our experiments demonstrate that tracking motion capture data with real-time response animation can be achieved easily. In addition, physically plausible motion style editing, automatic motion transition, and motion adaptation to different limb sizes can also be generated without difficulty.     

一种基于传感器的人体上肢运动实时跟踪方法

实时跟踪人体运动是人机交互的重要研究课题,可以广泛应用于虚拟现实、虚拟人运动合成、聋人手语自动生成、计算机3D动画、机器人运动控制、远程人机交互等领域。文中介绍了一种基于传感器的人体上肢运动实时跟踪方法,给出了该方法的虚拟人模型、计算原理与校正方法,最后介绍了整个方法的实现以及在中国聋人手语自动合成中的应用。该方法具有使用传感器少,运动跟踪精度高、计算过程简单而且速度快等特点。     

增强现实应用中基于三维模型的手形追踪

本文介绍了一种基于三维模型的分步迭代法来实现对全局和局部手运动的估计追踪。手部位置由ICP(Iterative Closed point)算法和因式分解法求得的掌形近似。结合自然手运动限制，本文采用基于序列的Monte Carlo算法追踪手指运动。最后采用在姿态估计和手指关节追踪之间的迭代算法得到一个精确的结构估计。实验证实本方法对自然手势运动具有较好的精确性和鲁棒性。     

Robot teaching system based on hand-robot contact state detection and motion intention recognition

This paper presents a robot teaching system based on hand-robot contact state detection and human motion intent recognition. The system can detect the contact state of the hand-robot joint and extracts motion intention information from the human surface electromyography (sEMG) signals to control the robot's motion. First, a hand-robot contact state detection method is proposed based on the fusion of the virtual robot environment with the physical environment. With the use of a target detection algorithm, the position of the human hand in the color image of the physical environment can be identified and its pixel coordinates can be calculated. Meanwhile, the synthetic images of the virtual robot environment are combined with those of the physical robot scene to determine whether the human hand is in contact with the robot. Besides, a human motion intention recognition model based on deep learning is designed to recognize human motion intention with the input of sEMG signals. Moreover, a robot motion mode selection module is built to control the robot for single-axis motion, linear motion, or repositioning motion by combining the hand-robot contact state and human motion intention. The experimental results indicate that the proposed system can perform online robot teaching for the three motion modes.     

Humanoid motion analysis and control based on COG viscoelasticity

This paper proposes a concept of center of gravity (COG) viscoelasticity to associate joint viscoelasticity with the inverted pendulum model of humanoid dynamics. Although COG viscoelasticity is based on the well-known kinematic relationship between joint stiffness and end-effector stiffness, it provides practical advantages for both analysis and control of humanoid motions. There are two main contributions. The first is that the COG viscoelasticity allows us to analyze fall risk. In a previous study, the author proposed a fall detection method based on the maximal output admissible (MOA) set, which is computed from feedback gain of the inverted pendulum model. The COG viscoelasticity associates joint viscoelasticity with the feedback gain and allows us to compute the corresponding MOA set when an arbitrary joint viscoelasticity is given. The second contribution is that the COG viscoelasticity can be also utilized in motion control. After we design a feedback gain in the inverted pendulum model utilizing the control theory, the COG viscoelasticity can directly transform it to the joint viscoelasticity. The validity of the COG viscoelasticity is verified with whole-body dynamics simulations.     

人手食指运动学建模

通过分析人手解剖学结构和观察人手运动特性，提出如下观点：（1）食指指掌关节具有两个轴线垂直相交的旋转自由度，两轴线相对于手掌不同的排列顺序对应于两种不同的运动学模型，指掌关节只能是其中的一种；（2）食指指掌关节的两个自由度在运动极限位置具有相关性．通过实验测绘出人手食指的工作空间形状，将之与两种可能的食指运动学模型对应的工作空间相比较，判别出指掌关节的真实运动学结构，并验证了指掌关节极限位置上两个自由度运动相关性．在上述工作基础上建立了符合人手实际结构的手指运动学模型．     

A multi-view vision-based hand motion capturing system

Vision-based hand motion capturing approaches play a critical role in human computer interface owing to its non-invasiveness, cost effectiveness, and user friendliness. This work presents a multi-view vision-based method to capture hand motion. A 3-D hand model with structural and kinematical constraints is developed to ensure that the proposed hand model behaves similar to an ordinary human hand. Human hand motion in a high degree of freedom space is estimated by developing a separable state based particle filtering (SSBPF) method to track the finger motion. By integrating different features, including silhouette, Chamfer distance, and depth map in different view angles, the proposed motion tracking system can capture the hand motion parameter effectively and solve the self-occlusion problem of the finger motion. Experimental results indicate that the hand joint angle estimation generates an average error of 11°.     

Adaptable Anatomical Models for Realistic Bone Motion Reconstruction

We present a system to reconstruct subject‐specific anatomy models while relying only on exterior measurements represented by point clouds. Our model combines geometry, kinematics, and skin deformations (skinning). This joint model can be adapted to different individuals without breaking its functionality, i.e., the bones and the skin remain well‐articulated after the adaptation. We propose an optimization algorithm which learns the subject‐specific (anthropometric) parameters from input point clouds captured using commodity depth cameras. The resulting personalized models can be used to reconstruct motion of human subjects. We validate our approach for upper and lower limbs, using both synthetic data and recordings of three different human subjects. Our reconstructed bone motion is comparable to results obtained by optical motion capture (Vicon) combined with anatomically‐based inverse kinematics (OpenSIM). We demonstrate that our adapted models better preserve the joint structure than previous methods such as OpenSIM or Anatomy Transfer.     

Analyzing and capturing articulated hand motion in image sequences

Capturing the human hand motion from video involves the estimation of the rigid global hand pose as well as the nonrigid finger articulation. The complexity induced by the high degrees of freedom of the articulated hand challenges many visual tracking techniques. For example, the particle filtering technique is plagued by the demanding requirement of a huge number of particles and the phenomenon of particle degeneracy. This paper presents a novel approach to tracking the articulated hand in video by learning and integrating natural hand motion priors. To cope with the finger articulation, this paper proposes a powerful sequential Monte Carlo tracking algorithm based on importance sampling techniques, where the importance function is based on an initial manifold model of the articulation configuration space learned from motion-captured data. In addition, this paper presents a divide-and-conquer strategy that decouples the hand poses and finger articulations and integrates them in an iterative framework to reduce the complexity of the problem. Our experiments show that this approach is effective and efficient for tracking the articulated hand. This approach can be extended to track other articulated targets.     

Performance Capture of High‐Speed Motion Using Staggered Multi‐View Recording

We present a markerless performance capture system that can acquire the motion and the texture of human actors performing fast movements using only commodity hardware. To this end we introduce two novel concepts: First, a staggered surround multi‐view recording setup that enables us to perform model‐based motion capture on motion‐blurred images, and second, a model‐based deblurring algorithm which is able to handle disocclusion, self‐occlusion and complex object motions. We show that the model‐based approach is not only a powerful strategy for tracking but also for deblurring highly complex blur patterns.     

手部运动计算机仿真的实现

根据手部自身的物理特性和生理特性,采用分层技术,严格按照手部解剖学,先建立其骨骼模型,在此基础上形成肌肉模型,最后在肌肉模型上附加一层皮肤。这样,骨骼运动引起肌肉的变形,随着皮肤也跟着发生相应的变化,使虚拟手模型无论从形状上还是运动上都能达到更高程度的逼真效果。     

基于学习的群体动画生成技术研究

为了降低群体动画中生成大量自然而又相似的人体运动的难度和复杂性,研究了一种基于学习的群体动画生成技术。该技术首先通过建立基于高斯过程隐变量模型和隐空间动态模型的运动姿势学习模型,将高维运动姿势映射到低维隐空间中,并在低维隐空间对相邻姿势的动态演化进行建模;然后通过对已有运动数据的学习来获得组成该运动的姿势的概率分布,再通过隐空间中的动态预测和Hybrid Monte Carlo采样来得到符合给定概率分布的隐轨迹;最后通过姿势重构来得到与原运动非常相似但又不同的一系列自然的运动,以产生群体动画,从而避开了传统的基于几何和物理约束的逆运动方法固有的困难和复杂性。     

基于字词分类的层次分词方法研究

中文分词是自然语言处理的基础性问题。条件随机场模型分词过程中出现的切分粒度过小和多字粘连造成的错分问题,是影响分词结果的两个主要原因。提出了一个基于字词分类的层次分词模型,该模型采用多部有效词典进行处理,在外层分词系统中解决切分粒度过小问题;在内层核心层,条件随机场分词后再处理多字粘连问题。实验结果表明,采用加入多词典的字词结合层次分类模型F-测度值有较大的提高,有助于得到好的分词结果。