Enriching a motion database by analogous combination of partial human motions

We have synthesized new human body motions from existing motion data, by dividing the body of an animated character into several parts, such as upper and lower body, and partitioning the motion of the character into corresponding partial motions. By combining different partial motions, we can generate new motion sequences. We select the most natural-looking combinations by analyzing the similarity of partial motions, using techniques such as motion segmentation, dimensionality reduction, and clustering. These new combinations can dramatically increase the size of a motion database, allowing more score in selecting motions to meet constraints, such as collision avoidance. We verify the naturalness and physical plausibility of the new motions using an SVM learning model and by analysis of static and dynamic balance.     

基于非线性流形学习的3维人体运动合成

为了实现3维人体运动的有效合成,提出了一种基于非线性流形学习的3维人体运动合成框架及算法,并可应用于方便、快捷、用户可控的3维人体运动合成。该合成算法框架先采用非线性流形降维方法将高维运动样本映射到低维流形上,同时求解其本征运动语义参数空间的表达,然后将用户在低维运动语义参数空间中交互生成的样本通过逆向映射重建得到具有新运动语义特征的3维运动序列。实验结果表明该方法不仅能够对运动物理参数（如特定关节的运动位置、物理运动特征）进行较为精确的控制,还可用于合成具有高层运动语义（运动风格）的新运动数据。与现有运动合成方法比较,该方法具有用户可控、交互性强等优点,能够应用于常见3维人体运动数据的高效生成。     

Primitive Object Grasping for Finger Motion Synthesis

We developed a new framework to generate hand and finger grasping motions. The proposed framework provides online adaptation to the position and orientation of objects and can generate grasping motions even when the object shape differs from that used during motion capture. This is achieved by using a mesh model, which we call primitive object grasping (POG), to represent the object grasping motion. The POG model uses a mesh deformation algorithm that keeps the original shape of the mesh while adapting to varying constraints. These characteristics are beneficial for finger grasping motion synthesis that satisfies constraints for mimicking the motion capture sequence and the grasping points reflecting the shape of the object. We verify the adaptability of the proposed motion synthesizer according to its position/orientation and shape variations of different objects by using motion capture sequences for grasping primitive objects, namely, a sphere, a cylinder, and a box. In addition, a different grasp strategy called a three‐finger grasp is synthesized to validate the generality of the POG‐based synthesis framework.     

基于学习的群体动画生成技术研究

为了降低群体动画中生成大量自然而又相似的人体运动的难度和复杂性,研究了一种基于学习的群体动画生成技术。该技术首先通过建立基于高斯过程隐变量模型和隐空间动态模型的运动姿势学习模型,将高维运动姿势映射到低维隐空间中,并在低维隐空间对相邻姿势的动态演化进行建模;然后通过对已有运动数据的学习来获得组成该运动的姿势的概率分布,再通过隐空间中的动态预测和Hybrid Monte Carlo采样来得到符合给定概率分布的隐轨迹;最后通过姿势重构来得到与原运动非常相似但又不同的一系列自然的运动,以产生群体动画,从而避开了传统的基于几何和物理约束的逆运动方法固有的困难和复杂性。     

Natural motion animation through constraining and deconstraining at will

This paper presents a computational technique for creating whole-body motions of human and animal characters without reference motion. Our work enables animators to generate a natural motion by dragging a link to an arbitrary position with any number of links pinned in the global frame, as well as other constraints such as desired joint angles and joint motion ranges. The method leads to an intuitive pin-and-drag interface where the user can generate whole-body motions by simply switching on or off or strengthening or weakening the constraints. This work is based on a new interactive inverse kinematics technique that allows more flexible attachment of pins and various types of constraints. Editing or retargeting captured motion requires only a small modification to the original method, although it can also create natural motions from scratch. We demonstrate the usefulness and advantage of our method with a number of example motion clips.     

基于多分辨率模型的快速运动过渡生成算法

运动捕获技术可以记录人体关节运动的细节,是当前最有前景的计算机动画技术.然而,运动数据的重用性一直是个难点,为此,多种运动编辑手段被提出.运动过渡是一种常见的编辑技术,它可以将输入的两端运动序列拼接,形成新的运动序列.其中,过渡点选择的合理与否直接影响着结果运动的质量.在两运动间选择过渡点,需要对输入运动的每一对帧之间分别计算帧间的距离,其计算复杂度是O(n2)的,通过引入多分辨率模型,文中将该复杂度降低到O(nlog2n),同时试验结果表明,此方法并未损害到结果运动的质量.     

Natural metrics and least-committed priors for articulated tracking

In articulated tracking, one is concerned with estimating the pose of a person in every frame of a film. This pose is most often represented as a kinematic skeleton where the joint angles are the degrees of freedom. Least-committed predictive models are then phrased as a Brownian motion in joint angle space. However, the metric of the joint angle space is rather unintuitive as it ignores both bone lengths and how bones are connected. As Brownian motion is strongly linked with the underlying metric, this has severe impact on the predictive models. We introduce the spatial kinematic manifold of joint positions, which is embedded in a high dimensional Euclidean space. This Riemannian manifold inherits the metric from the embedding space, such that distances are measured as the combined physical length that joints travel during movements. We then develop a least-committed Brownian motion model on the manifold that respects the natural metric. This model is expressed in terms of a stochastic differential equation, which we solve using a novel numerical scheme. Empirically, we validate the new model in a particle filter based articulated tracking system. Here, we not only outperform the standard Brownian motion in joint angle space, we are also able to specialise the model in ways that otherwise are both difficult and expensive in joint angle space.     

Synthesizing Character Animation with Smoothly Decomposed Motion Layers

The processing of captured motion is an essential task for undertaking the synthesis of high-quality character animation. The motion decomposition techniques investigated in prior work extract meaningful motion primitives that help to facilitate this process. Carefully selected motion primitives can play a major role in various motion-synthesis tasks, such as interpolation, blending, warping, editing or the generation of new motions. Unfortunately, for a complex character motion, finding generic motion primitives by decomposition is an intractable problem due to the compound nature of the behaviours of such characters. Additionally, decomposed motion primitives tend to be too limited for the chosen model to cover a broad range of motion-synthesis tasks. To address these challenges, we propose a generative motion decomposition framework in which the decomposed motion primitives are applicable to a wide range of motion-synthesis tasks. Technically, the input motion is smoothly decomposed into three motion layers. These are base-level motion, a layer with controllable motion displacements and a layer with high-frequency residuals. The final motion can easily be synthesized simply by changing a single user parameter that is linked to the layer of controllable motion displacements or by imposing suitable temporal correspondences to the decomposition framework. Our experiments show that this decomposition provides a great deal of flexibility in several motion synthesis scenarios: denoising, style modulation, upsampling and time warping.     

Probabilistic Character Motion Synthesis using a Hierarchical Deep Latent Variable Model

We present a probabilistic framework to generate character animations based on weak control signals, such that the synthesized motions are realistic while retaining the stochastic nature of human movement. The proposed architecture, which is designed as a hierarchical recurrent model, maps each sub-sequence of motions into a stochastic latent code using a variational autoencoder extended over the temporal domain. We also propose an objective function which respects the impact of each joint on the pose and compares the joint angles based on angular distance. We use two novel quantitative protocols and human qualitative assessment to demonstrate the ability of our model to generate convincing and diverse periodic and non-periodic motion sequences without the need for strong control signals.     

三维人体行走模型的研究与实现

本文采用多面体组合建立人体模型，用三角函数拟合步行时的关节活动轨迹，设计ｂｅｎｄ函数来实现关节体和部位体的转动控制，采用逐节调整转动角度的方法实现人体旋转坐标变换，整体进行人体的坐标平移变换，生成了步行过程中的一幅幅三维人体活动模型图形，使之在连续播映时产生逼真的人体行走画效果。     

Pen-to-mime: Pen-based interactive control of a human figure

In this paper a pen-based intuitive interface is presented, that controls a virtual human figure interactively. Recent commercial pen devices can detect not only the pen positions but also the pressure and tilt of the pen. We utilize such information to make a human figure perform various types of motions in response to the pen movements manipulated by the user. The figure walks, runs, turns and steps along the trajectory and speed of the pen. The figure also bends, stretches and tilts in response to the tilt of the pen. Moreover, it ducks and jumps in response to the pen pressure. Using our interface, the user controls a virtual human figure intuitively as if he or she were holding a virtual puppet and playing with it.

In addition to the interface design and implementation, this paper describes a motion generation engine to produce various motion based on varying parameters that are given by the pen interface. We take a motion blending approach and construct motion blending modules with a set of small number of motion capture data for each type of motions. Finally, we present the results from user experiments and comparison with a transitional gamepad-based interface.     

Interpolation synthesis of articulated figure motion

Most conventional media depend on engaging and appealing characters. Empty spaces and buildings would not fare well as television or movie programming, yet virtual reality usually offers up such spaces. The problem lies in the difficulty of creating computer generated characters that display real time, engaging interaction and realistic motion. Articulated figure motion for real time computer graphics offers one solution to this problem. A common approach stores a set of motions and lets you choose one particular motion at a time. The article describes a process that greatly expands the range of possible motions. Mixing motions selected from a database lets you create a new motion to exact specifications. The synthesized motion retains the original motions' subtle qualities, such as the realism of motion capture or the expressive, exaggerated qualities of artistic animation. Our method provides a new way to achieve inverse kinematics capability-for example, placing the hands or feet of an articulated figure in specific positions. It proves useful for both real time graphics and prerendered animation production. The method, called interpolation synthesis, is based on motion capture data and it provides real time character motion for interactive entertainment or avatars in virtual worlds     

Multi-activity 3D human motion recognition and tracking in composite motion model with synthesized transition bridges

Recognizing and tracking multiple activities are all extremely challenging machine vision tasks due to diverse motion types included and high-dimensional (HD) state space. To overcome these difficulties, a novel generative model called composite motion model (CMM) is proposed. This model contains a set of independent, low-dimensional (LD), and activity-specific manifold models that effectively constrain the state search space for 3D human motion recognition and tracking. This separate modeling of activity-specific movements can not only allow each manifold model to be optimized in accordance with only its respective movement, but also improve the scalability of the models. For accurate tracking with our CMM, a particle filter (PF) method is thus employed and then the particles can be distributed in all manifold models at each time step. In addition, an efficient activity switching strategy is proposed to dominate the particle distribution on all LD manifolds. To diffuse the particles amongst manifold models and respond quickly to the sudden changes in the activity, a set of visually-reasonable and kinematically-realistic transition bridges are synthesized by using the good properties of LD latent space and HD observation space, which enables the inter-activity motions seem more natural and realistic. Finally, a pose hypothesis that can best interpret the visual observation is selected and then used to recognize the activity that is currently observed. Extensive experiments, via qualitative and quantitative analyses, verify the effectiveness and robustness of our proposed CMM in the tasks of multi-activity 3D human motion recognition and tracking.     

Tracking People on a Torus

We present a framework for monocular 3D kinematic pose tracking and viewpoint estimation of periodic and quasi-periodic human motions from an uncalibrated camera. The approach we introduce here is based on learning both the visual observation manifold and the kinematic manifold of the motion using a joint representation. We show that the visual manifold of the observed shape of a human performing a periodic motion, observed from different viewpoints, is topologically equivalent to a torus manifold. The approach we introduce here is based on the supervised learning of both the visual and kinematic manifolds. Instead of learning an embedding of the manifold, we learn the geometric deformation between an ideal manifold (conceptual equivalent topological structure) and a twisted version of the manifold (the data). Experimental results show accurate estimation of the 3D body posture and the viewpoint from a single uncalibrated camera.     

Animating with style: defining expressive semantics of motion

Actions performed by a virtual character can be controlled with verbal commands such as ‘walk five steps forward’. Similar control of the motion style, meaning how the actions are performed, is complicated by the ambiguity of describing individual motions with phrases such as ‘aggressive walking’. In this paper, we present a method for controlling motion style with relative commands such as ‘do the same, but more sadly’. Based on acted example motions, comparative annotations, and a set of calculated motion features, relative styles can be defined as vectors in the feature space. We present a new method for creating these style vectors by finding out which features are essential for a style to be perceived and eliminating those that show only incidental correlations with the style. We show with a user study that our feature selection procedure is more accurate than earlier methods for creating style vectors, and that the style definitions generalize across different actors and annotators. We also present a tool enabling interactive control of parametric motion synthesis by verbal commands. As the control method is independent from the generation of motion, it can be applied to virtually any parametric synthesis method.     

A path browser for exploratory motion assembly

We present an interactive method for creating animation sequences of characters based on captured motion data in an exploratory way as in assembling construction toys. The key component of our method is a path browser that can retrieve and visualize paths as diverse as possible connecting a given pair of initial and final motion fragments instantiated in the space. With the aid of our path browser, the user can develop large‐scale assembly of motions gradually through iterations of arranging and putting together motion fragments. For the efficient retrieval of connecting paths, we use a bidirectional search tree that grows from the initial and final configurations simultaneously under the guidance of a mixed strategy for both global exploration and local optimization. The usefulness of our approach is demonstrated through experiments with a variety of motion data including box moving, basketball, and breakdance data. Copyright © 2016 John Wiley & Sons, Ltd.     

Style‐Based Motion Synthesis†

Representing motions as linear sums of principal components has become a widely accepted animation technique. While powerful, the simplest version of this approach is not particularly well suited to modeling the specific style of an individual whose motion had not yet been recorded when building the database: it would take an expert to adjust the PCA weights to obtain a motion style that is indistinguishable from his. Consequently, when realism is required, the current practice is to perform a full motion capture session each time a new person must be considered. In this paper, we extend the PCA approach so that this requirement can be drastically reduced: for whole classes of cyclic and noncyclic motions such as walking, running or jumping, it is enough to observe the newcomer moving only once at a particular speed or jumping a particular distance using either an optical motion capture system or a simple pair of synchronized video cameras. This one observation is used to compute a set of principal component weights that best approximates the motion and to extrapolate in real‐time realistic animations of the same person walking or running at different speeds, and jumping a different distance.     

Adaptive control of redundant multiple robots in cooperative motion

A redundant robot has more degrees of freedom than what is needed to uniquely position the robot end-effector. In practical applications the extra degrees of freedom increase the orientation and reach of the robot. Also the load carrying capacity of a single robot can be increased by cooperative manipulation of the load by two or more robots. In this paper, we develop an adaptive control scheme for kinematically redundant multiple robots in cooperative motion.In a usual robotic task, only the end-effector position trajectory is specified. The joint position trajectory will therefore be unknown for a redundant multi-robot system and it must be selected from a self-motion manifold for a specified end-effector or load motion. In this paper, it is shown that the adaptive control of cooperative multiple redundant robots can be addressed as a reference velocity tracking problem in the joint space. A stable adaptive velocity control law is derived. This controller ensures the bounded estimation of the unknown dynamic parameters of the robots and the load, the exponential convergence to zero of the velocity tracking errors, and the boundedness of the internal forces. The individual robot joint motions are shown to be stable by decomposing the joint coordinates into two variables, one which is homeomorphic to the load coordinates, the other to the coordinates of the self-motion manifold. The dynamics on the self-motion manifold are directly shown to be related to the concept of zero-dynamics. It is shown that if the reference joint trajectory is selected to optimize a certain type of objective functions, then stable dynamics on the self-motion manifold result. The overall stability of the joint positions is established from the stability of two cascaded dynamic systems involving the two decomposed coordinates.     

Combined Direct and Inverse Kinematic Control for Articulated Figure Motion Editing

A new approach for the animation of articulated figures is presented. We propose a system of articulated motion design which offers a full combination of both direct and inverse kinematic control of the joint parameters. Such an approach allows an animator to specify interactively goal-directed changes to existing sampled joint motions, resulting in a more general and expressive class of possible joint motions. The fundamental idea is to consider any desired-joint space motion as a reference model inserted into the secondary task of an inverse kinematic control scheme. This approach profits from the use of half-space Cartesian main tasks in conjunction with a parallel control of the articulated figure called the coach-trainee metaphor. In addition, a transition function is introduced so as to guarantee the continuity of the control. The resulting combined kinematic control scheme leads to a new methodology of joint-motion editing which is demonstrated through the improvement of a functional model of human walking.