基于逆运动学和重构式ICA的人体运动风格分析与合成

使用独立成分分析（Independent component analysis,ICA）来建模运动风格、合成风格化的人体运动,是一种有效且有前景的手段.为了避免现有方法在设定独立成分个数或子空间结构时的人为影响,并提高风格成分的质量,提出一种基于重构式独立成分分析的运动风格分析方法.由于放弃了混合矩阵的正交性约束,一方面,拥有了更多的自由度来表示各独立成分;另一方面,利用特征的过完备性以及自身在特征选择时的稀疏特性,能够自动地确立独立成分数目.此外,通过结合基于主测地线分析的逆运动学与运动过渡技术,该方法能够合成包含多种风格、任意长度的行走运动,同时还能通过编辑特定帧的人体姿势来约束合成的结果.实验结果表明,该方法能够有效地分析出行走、跳跃和踢腿等运动中代表风格的独立成分,并根据用户对风格的编辑,实时地生成自然、平滑的运动.     

Flexible editing of human motion by three‐way decomposition

This paper proposes a new generative model for flexible editing of human motion. Different from previous work, three intuitive factors of motion, namely, content, identity and style, can be manipulated directly with the new model. With the new generative model, motion editing can be achieved in various aspects, including transferring an unknown style from an actor to another, synthesizing other styles for an unknown actor and generating a new motion with other content. Copyright © 2013 John Wiley & Sons, Ltd.     

结合低维运动模型和逆运动学的风格化人体运动合成

如何得到满足用户指定约束的风格化运动是近年来计算机动画领域的研究难点,针对这个问题,提出一个基于独立特征子空间的低维运动模型,该模型可以较好地参数化运动风格,并在此基础上提出了一种合成满足约束的风格化人体运动的算法.该算法在低维空间中求解反向运动学问题,并在风格子空间中响应用户输入的风格参数,使用户可以在指定关键帧末端约束的同时对风格进行编辑.实验结果表明:文中算法效率高,具有良好的交互性,能够用于动画的交互式编辑和与合成.     

基于多分辨率模型的快速运动过渡生成算法

运动捕获技术可以记录人体关节运动的细节,是当前最有前景的计算机动画技术.然而,运动数据的重用性一直是个难点,为此,多种运动编辑手段被提出.运动过渡是一种常见的编辑技术,它可以将输入的两端运动序列拼接,形成新的运动序列.其中,过渡点选择的合理与否直接影响着结果运动的质量.在两运动间选择过渡点,需要对输入运动的每一对帧之间分别计算帧间的距离,其计算复杂度是O(n2)的,通过引入多分辨率模型,文中将该复杂度降低到O(nlog2n),同时试验结果表明,此方法并未损害到结果运动的质量.     

基于学习的群体动画生成技术研究

为了降低群体动画中生成大量自然而又相似的人体运动的难度和复杂性,研究了一种基于学习的群体动画生成技术。该技术首先通过建立基于高斯过程隐变量模型和隐空间动态模型的运动姿势学习模型,将高维运动姿势映射到低维隐空间中,并在低维隐空间对相邻姿势的动态演化进行建模;然后通过对已有运动数据的学习来获得组成该运动的姿势的概率分布,再通过隐空间中的动态预测和Hybrid Monte Carlo采样来得到符合给定概率分布的隐轨迹;最后通过姿势重构来得到与原运动非常相似但又不同的一系列自然的运动,以产生群体动画,从而避开了传统的基于几何和物理约束的逆运动方法固有的困难和复杂性。     

Creating and retargetting motion by the musculoskeletal human body model

Recently, optimization has been used in various ways to interpolate or retarget human body motions obtained by motion-capturing systems. However, in such cases, the inner structure of a human body has rarely been taken into account, and hence there have been difficulties in simulating physiological effects such as fatigue or injuries. In this paper, we propose a method to create/retarget human body motions using a musculoskeletal human body model. Using our method, it is possible to create dynamically and physiologically feasible motions. Since a muscle model based on Hill's model is included in our system, it is also possible to retarget the original motion by changing muscular parameters. For example, using the muscle fatigue model, a motion where a human body gradually gets tired can be simulated. By increasing the maximal force exertable by the muscles, or decreasing it to zero, training or displacement effects of muscles can also be simulated. Our method can be used for biomechanically correct inverse kinematics, interpolation of motions, and physiological retargetting of the human body motion.     

An Integrated System for Modeling,Animating and Rendering Hair

There are basically four problems to solve in order to produce realistic animated synthetic actors with hair: hair modeling and creation, hair motion, collision detection and hair rendering. This paper describes a complete methodology to solve these basic four problems. We present how hair styles may be designed with our Hair Styler module. Then we survey the animation model and emphasize a method of collision processing. Finally, we explain how hair may be rendered using an extension of a standard ray-tracing program. We also show applications of our synthetic actors with various hair styles and different styles of mustaches and beards.     

基于非线性流形学习的3维人体运动合成

为了实现3维人体运动的有效合成,提出了一种基于非线性流形学习的3维人体运动合成框架及算法,并可应用于方便、快捷、用户可控的3维人体运动合成。该合成算法框架先采用非线性流形降维方法将高维运动样本映射到低维流形上,同时求解其本征运动语义参数空间的表达,然后将用户在低维运动语义参数空间中交互生成的样本通过逆向映射重建得到具有新运动语义特征的3维运动序列。实验结果表明该方法不仅能够对运动物理参数（如特定关节的运动位置、物理运动特征）进行较为精确的控制,还可用于合成具有高层运动语义（运动风格）的新运动数据。与现有运动合成方法比较,该方法具有用户可控、交互性强等优点,能够应用于常见3维人体运动数据的高效生成。     

Animating with style: defining expressive semantics of motion

Actions performed by a virtual character can be controlled with verbal commands such as ‘walk five steps forward’. Similar control of the motion style, meaning how the actions are performed, is complicated by the ambiguity of describing individual motions with phrases such as ‘aggressive walking’. In this paper, we present a method for controlling motion style with relative commands such as ‘do the same, but more sadly’. Based on acted example motions, comparative annotations, and a set of calculated motion features, relative styles can be defined as vectors in the feature space. We present a new method for creating these style vectors by finding out which features are essential for a style to be perceived and eliminating those that show only incidental correlations with the style. We show with a user study that our feature selection procedure is more accurate than earlier methods for creating style vectors, and that the style definitions generalize across different actors and annotators. We also present a tool enabling interactive control of parametric motion synthesis by verbal commands. As the control method is independent from the generation of motion, it can be applied to virtually any parametric synthesis method.     

Constrained synthesis of textural motion for articulated characters

Obtaining high-quality, realistic motions of articulated characters is both time consuming and expensive, necessitating the development of easy-to-use and effective tools for motion editing and reuse. We propose a new simple technique for generating constrained variations of different lengths from an existing captured or otherwise animated motion. Our technique is applicable to textural motions, such as walking or dancing, where the motion sequence can be decomposed into shorter motion segments without an obvious temporal ordering among them. Inspired by previous work on texture synthesis and video textures, our method essentially produces a reordering of these shorter segments. Discontinuities are eliminated by carefully choosing the transition points and applying local adaptive smoothing in their vicinity, if necessary. The user is able to control the synthesis process by specifying a small number of simple constraints.     

基于小波变换的运动分析及其应用

对捕获的运动数据进行编辑处理 ,是生成新的复杂人体动画和提高运动捕获数据重用性的关键 ,但目前大多数运动编辑技术不具备对运动进行高层控制处理的能力 ,为此 ,提出了一种基于小波变换的运动编辑新算法 ,即将小波变换引入运动编辑 ,并对运动信号进行多分辨率分析 ,从而实现了运动特征增强、运动融合及运动特征提取与综合 .实验结果表明 ,该算法非常适合对运动特征进行处理 ,由于其能够在高层次上对运动进行有效的编辑 ,因而提高了动画师的工作效率 .     

Diversifying Semantic Image Synthesis and Editing via Class- and Layer-wise VAEs

Semantic image synthesis is a process for generating photorealistic images from a single semantic mask. To enrich the diversity of multimodal image synthesis, previous methods have controlled the global appearance of an output image by learning a single latent space. However, a single latent code is often insufficient for capturing various object styles because object appearance depends on multiple factors. To handle individual factors that determine object styles, we propose a class- and layer-wise extension to the variational autoencoder (VAE) framework that allows flexible control over each object class at the local to global levels by learning multiple latent spaces. Furthermore, we demonstrate that our method generates images that are both plausible and more diverse compared to state-of-the-art methods via extensive experiments with real and synthetic datasets in three different domains. We also show that our method enables a wide range of applications in image synthesis and editing tasks.     

基于运动图的路径编辑技术

改进了过渡运动的生成算法和路径搜索算法,提出了一种基于运动图的路径编辑的新方法。其中,针对过渡运动的构造,通过最小化融合帧之间的平均帧间距来自动确定用于运动融合的运动片段,并提出了改进动态时间变形(EDTW)算法来解决这一最优化问题;针对运动图上的路径搜索,提出了基于路径曲线所夹面积的目标函数并改进了分段搜索算法和剪枝策略。实验结果表明,该方法能够编辑生成与用户指定路径高度匹配的人物运动。     

Style-Controllable Speech-Driven Gesture Synthesis Using Normalising Flows

Automatic synthesis of realistic gestures promises to transform the fields of animation, avatars and communicative agents. In off-line applications, novel tools can alter the role of an animator to that of a director, who provides only high-level input for the desired animation; a learned network then translates these instructions into an appropriate sequence of body poses. In interactive scenarios, systems for generating natural animations on the fly are key to achieving believable and relatable characters. In this paper we address some of the core issues towards these ends. By adapting a deep learning-based motion synthesis method called MoGlow, we propose a new generative model for generating state-of-the-art realistic speech-driven gesticulation. Owing to the probabilistic nature of the approach, our model can produce a battery of different, yet plausible, gestures given the same input speech signal. Just like humans, this gives a rich natural variation of motion. We additionally demonstrate the ability to exert directorial control over the output style, such as gesture level, speed, symmetry and spacial extent. Such control can be leveraged to convey a desired character personality or mood. We achieve all this without any manual annotation of the data. User studies evaluating upper-body gesticulation confirm that the generated motions are natural and well match the input speech. Our method scores above all prior systems and baselines on these measures, and comes close to the ratings of the original recorded motions. We furthermore find that we can accurately control gesticulation styles without unnecessarily compromising perceived naturalness. Finally, we also demonstrate an application of the same method to full-body gesticulation, including the synthesis of stepping motion and stance.     

Example-based automatic music-driven conventional dance motion synthesis

We introduce a novel method for synthesizing dance motions that follow the emotions and contents of a piece of music. Our method employs a learning-based approach to model the music to motion mapping relationship embodied in example dance motions along with those motions' accompanying background music. A key step in our method is to train a music to motion matching quality rating function through learning the music to motion mapping relationship exhibited in synchronized music and dance motion data, which were captured from professional human dance performance. To generate an optimal sequence of dance motion segments to match with a piece of music, we introduce a constraint-based dynamic programming procedure. This procedure considers both music to motion matching quality and visual smoothness of a resultant dance motion sequence. We also introduce a two-way evaluation strategy, coupled with a GPU-based implementation, through which we can execute the dynamic programming process in parallel, resulting in significant speedup. To evaluate the effectiveness of our method, we quantitatively compare the dance motions synthesized by our method with motion synthesis results by several peer methods using the motions captured from professional human dancers' performance as the gold standard. We also conducted several medium-scale user studies to explore how perceptually our dance motion synthesis method can outperform existing methods in synthesizing dance motions to match with a piece of music. These user studies produced very positive results on our music-driven dance motion synthesis experiments for several Asian dance genres, confirming the advantages of our method.     

Smart Motion Synthesis

Creating long motion sequences is a time‐consuming task even when motion capture equipment or motion editing tools are used. In this paper, we propose a system for creating a long motion sequence by combining elementary motion clips. The user is asked to first input motions on a timeline. The system then automatically generates a continuous and natural motion. Our system employs four motion synthesis methods: motion transition, motion connection, motion adaptation, and motion composition. Based on the constraints between the feet of the animated character and the ground, and the timing of the input motions, the appropriate method is determined for each pair of overlapped or sequential motions. As the user changes the arrangement of the motion clips, the system interactively changes the output motion. Alternatively, the user can make the system execute an input motion as soon as possible so that it follows the previous motion smoothly. Using our system, users can make use of existing motion clips. Because the entire process is automatic, even novices can easily use our system. A prototype system demonstrates the effectiveness of our approach.     

Human motion retrieval using topic model

Content‐based human motion retrieval is important for animators with the development of motion editing and synthesis, which need to search similar motions in large databases. Obtaining text‐based representation from quantization of mocap data turned out to be efficient. It becomes a fundamental step of many researches in human motion analysis. Geometric features are one of these techniques, which involve much prior knowledge and reduce data redundancy of numerical data. We describe geometric features as basic unit to define human motions (also called mo‐words) and view a human motion as a generative process. Therefore, we obtain topic motions, which possess more semantic information using latent Dirichlet allocation by learning from massive training examples in order to understand motions better. We combine probabilistic model with human motion retrieval and come up with a new representation of human motions and a new retrieval framework. Our experiments demonstrate its advantages, both for understanding motions and retrieval. Copyright © 2011 John Wiley & Sons, Ltd.