Hybrid motion graph for character motion synthesis

ObjectiveThis paper proposes a novel framework of Hybrid Motion Graph (HMG) for creating character animations, which enhances the graph-based structural control by motion field representations for efficient motion synthesis of diverse and interactive character animations.MethodsIn HMG framework, the motion template of each class is automatically derived from the training motions for capturing the general spatio-temporal characteristics of an entire motion class. Typical motion field for each class is then constructed. The smooth transitions among motion classes are then generated by interpolating the related motion templates with spacetime constraints. Finally, a hybrid motion graph is built by integrating the separate motion fields for each motion class into the global structural control of motion graph through smooth transition.ResultsIn motion synthesis stage, a character may freely ‘switch’ among different motion classes in the hybrid motion graph via smooth transitions between motion templates and ‘flow’ within each class through the continuous space of motion field with agile and the continuous control process.ConclusionExperimental results show that our framework realizes the fast connectivity among different motion classes and high responsiveness and interactivity for creating realistic character animation of rich behaviors with limited motion data and computational resources.     

基于非线性流形学习的3维人体运动合成

为了实现3维人体运动的有效合成,提出了一种基于非线性流形学习的3维人体运动合成框架及算法,并可应用于方便、快捷、用户可控的3维人体运动合成。该合成算法框架先采用非线性流形降维方法将高维运动样本映射到低维流形上,同时求解其本征运动语义参数空间的表达,然后将用户在低维运动语义参数空间中交互生成的样本通过逆向映射重建得到具有新运动语义特征的3维运动序列。实验结果表明该方法不仅能够对运动物理参数（如特定关节的运动位置、物理运动特征）进行较为精确的控制,还可用于合成具有高层运动语义（运动风格）的新运动数据。与现有运动合成方法比较,该方法具有用户可控、交互性强等优点,能够应用于常见3维人体运动数据的高效生成。     

Effectively Discriminating Fighting Shots in Action Movies

Fighting shots are the highlights of action movies and an effective approach to discriminating fighting shots is very useful for many applications,such as movie trailer construction,movie content filtering,and movie content retrieval. In this paper,we present a novel method for this task.Our approach first extracts the reliable motion information of local invariant features through a robust keypoint tracking computation;then foreground keypoints are distinguished from background keypoints by a sophisticated voting process;further,the parameters of the camera motion model is computed based on the motion information of background keypoints,and this model is then used as a reference to compute the actual motion of foreground keypoints;finally,the corresponding feature vectors are extracted to characterizing the motions of foreground keypoints,and a support vector machine(SVM) classifier is trained based on the extracted feature vectors to discriminate fighting shots.Experimental results on representative action movies show our approach is very effective.     

基于小波变换的运动分析及其应用

对捕获的运动数据进行编辑处理 ,是生成新的复杂人体动画和提高运动捕获数据重用性的关键 ,但目前大多数运动编辑技术不具备对运动进行高层控制处理的能力 ,为此 ,提出了一种基于小波变换的运动编辑新算法 ,即将小波变换引入运动编辑 ,并对运动信号进行多分辨率分析 ,从而实现了运动特征增强、运动融合及运动特征提取与综合 .实验结果表明 ,该算法非常适合对运动特征进行处理 ,由于其能够在高层次上对运动进行有效的编辑 ,因而提高了动画师的工作效率 .     

IMoS: Intent-Driven Full-Body Motion Synthesis for Human-Object Interactions

Can we make virtual characters in a scene interact with their surrounding objects through simple instructions? Is it possible to synthesize such motion plausibly with a diverse set of objects and instructions? Inspired by these questions, we present the first framework to synthesize the full-body motion of virtual human characters performing specified actions with 3D objects placed within their reach. Our system takes textual instructions specifying the objects and the associated ‘intentions’ of the virtual characters as input and outputs diverse sequences of full-body motions. This contrasts existing works, where full-body action synthesis methods generally do not consider object interactions, and human-object interaction methods focus mainly on synthesizing hand or finger movements for grasping objects. We accomplish our objective by designing an intent-driven full-body motion generator, which uses a pair of decoupled conditional variational auto-regressors to learn the motion of the body parts in an autoregressive manner. We also optimize the 6-DoF pose of the objects such that they plausibly fit within the hands of the synthesized characters. We compare our proposed method with the existing methods of motion synthesis and establish a new and stronger state-of-the-art for the task of intent-driven motion synthesis.     

Robot motion command simplification and scaling

It has been observed that human limb motions are not very accurate, leading to the hypothesis that the human motor control system may have simplified motion commands at the expense of motion accuracy. Inspired by this hypothesis, we propose learning schemes that trade motion accuracy for motion command simplification. When the original complex motion commands capable of tracking motion accurately are reduced to simple forms, the simplified motion commands can then be stored and manipulated by using learning mechanisms with simple structures and scanty memory resources, and they can be executed quickly and smoothly. We also propose learning schemes that can perform motion command scaling, so that simplified motion commands can be provided for a number of similar motions of different movement distances and velocities without recalculating system dynamics. Simulations based on human motions are reported that demonstrate the effectiveness of the proposed learning schemes in implementing this accuracy-simplification tradeoff.     

Robot motion design using bunraku emotional expressions – focusing on Jo-Ha-Kyū in sounds and movements*

One of the UNESCO intangible cultural heritages Bunraku puppets can play one of the most beautiful puppet motions in the world. The Bunraku puppet motions can express emotions without the so-called ‘Uncanny Valley.’ We try to convert these emotional motions into robot affective motions so that robots can interact with human beings more comfortable. In so doing, in the present paper, we present a robot motion design framework using Bunraku affective motions that are based on the so-called ‘Jo-Ha-Kyū,’ and convert a few simple Bunraku motions into a robot motions using one of deep learning methods. Our primitive experiments show that Jo-Ha-Kyū can be incorporated into robot motion design smoothly, and some simple affective robot motions can be designed using our proposed framework.     

An efficient control over human running animation with extension of planar hopper model

The most important goal of character animation is to efficiently control the motions of a character. Until now, many techniques have been proposed for human gait animation. Some techniques have been created to control the emotions in gaits such as ‘tired walking’ and ‘brisk walking’ by using parameter interpolation or motion data mapping. Since it is very difficult to automate the control over the emotion of a motion, the emotions of a character model have been generated by creative animators. This paper proposes a human running model based on a one‐legged planar hopper with a self‐balancing mechanism. The proposed technique exploits genetic programming to optimize movement and can be easily adapted to various character models. We extend the energy minimization technique to generate various motions in accordance with emotional specifications. Copyright © 1999 John Wiley & Sons, Ltd.     

Online real‐time locomotive motion transformation based on biomechanical observations

In the paper, we present an online real‐time method for automatically transforming a basic locomotive motion to a desired motion of the same type, based on biomechanical results. Given an online request for a motion of a certain type with desired moving speed and turning angle, our method first extracts a basic motion of the same type from a motion graph, and then transforms it to achieve the desired moving speed and turning angle by exploiting the following biomechanical observations: contact‐driven center‐of‐mass control, anticipatory reorientation of upper body segments, moving speed adjustment, and whole‐body leaning. Exploiting these observations, we propose a simple but effective method to add physical and behavioral naturalness to the resulting locomotive motions without preprocessing. Through experiments, we show that our method enables a character to respond agilely to online user commands while efficiently generating walking, jogging, and running motions with a compact motion library. Our method can also deal with certain dynamical motions such as forward roll. Copyright © 2016 John Wiley & Sons, Ltd.     

Modification via averaging of partial-energy-shaping control for creating oscillations: cart-pendulum example

We consider the challenging problem of creating oscillations in underactuated mechanical systems. Target oscillatory motions of the indirectly actuated degree of freedom of a mechanical system can often be achieved via a straightforward to design feedback transformation. Moreover, the corresponding part of the dynamics can be forced to match a desired second-order system possessing the target periodic solution (Aracil, J., Gordillo, F., and Acosta, J.A. (2002), ‘Stabilization of Oscillations in the Inverted Pendulum’, in Proceedings of the 15th IFAC World Congress, Barcelona, Spain; Canudas-de-Wit, C., Espiau, B., and Urrea, C. (2002), ‘Orbital Stabilisation of Underactuated Mechanical Systems’, in Proceedings of the 15th IFAC World Congress, Barcelona, Spain). Sometimes, it is possible to establish the presence of periodic or bounded motions for the remaining degrees of freedom in the transformed system. However, typically this motion planning procedure leads to an open-loop unstable orbit and by necessity should be followed by a feedback control design. We propose a new approach for synthesis of a (practically) stabilising feedback controller, which ensures convergence of the solutions of the closed-loop system into a narrow tube around the preplanned orbit. The method is illustrated in detail by shaping oscillations in the inverted pendulum on a cart around its upright equilibrium. The complete analysis is based on application of a non-standard higher-order averaging technique assuming sufficiently high frequency of oscillations and is presented for this particular example.     

Repairing CAD model errors based on the design history

For users of CAD data, few things are as frustrating as receiving unusable, poor quality data. Users often waste time fixing or rebuilding such data from scratch on the basis of paper drawings. While previous studies use the boundary representation (B-Rep) of CAD models, we propose an approach to repairing CAD model errors that is based on the design history. CAD model errors can be corrected by an interdependency analysis of the feature commands or of the parametric data of each feature command, as well as by a reconstruction of the feature commands through rule-based reasoning of an expert system. Unlike other correction methods based on B-Rep models, our method repairs parametric feature models without translating them to a B-Rep shape, and it also preserves parametric information.     

Style-Controllable Speech-Driven Gesture Synthesis Using Normalising Flows

Automatic synthesis of realistic gestures promises to transform the fields of animation, avatars and communicative agents. In off-line applications, novel tools can alter the role of an animator to that of a director, who provides only high-level input for the desired animation; a learned network then translates these instructions into an appropriate sequence of body poses. In interactive scenarios, systems for generating natural animations on the fly are key to achieving believable and relatable characters. In this paper we address some of the core issues towards these ends. By adapting a deep learning-based motion synthesis method called MoGlow, we propose a new generative model for generating state-of-the-art realistic speech-driven gesticulation. Owing to the probabilistic nature of the approach, our model can produce a battery of different, yet plausible, gestures given the same input speech signal. Just like humans, this gives a rich natural variation of motion. We additionally demonstrate the ability to exert directorial control over the output style, such as gesture level, speed, symmetry and spacial extent. Such control can be leveraged to convey a desired character personality or mood. We achieve all this without any manual annotation of the data. User studies evaluating upper-body gesticulation confirm that the generated motions are natural and well match the input speech. Our method scores above all prior systems and baselines on these measures, and comes close to the ratings of the original recorded motions. We furthermore find that we can accurately control gesticulation styles without unnecessarily compromising perceived naturalness. Finally, we also demonstrate an application of the same method to full-body gesticulation, including the synthesis of stepping motion and stance.     

Human motion generation with multifactor models

To generate human motions with various specific attributes is a difficult task because of high dimensionality and complexity of human motions. This paper presents a novel human motion model for generating and editing motions with multiple factors. A set of motions performed by several actors with various styles was captured for constructing a well‐structured motion database. Subsequently, MICA (multilinear independent component analysis) model that combines ICA and conventional multilinear framework was adopted for the construction of a multifactor model. With this model, new motions can be synthesized by interpolation and through solving optimization problems for the specific factors. Our method offers a practical solution to edit stylistic human motions in a parametric space learnt with MICA model. We demonstrated the power of our method by generating and editing sideways stepping, reaching, and striding over obstructions using different actors with various styles. The experimental results show that our method can be used for interactive stylistic motion synthesis and editing. Copyright © 2011 John Wiley & Sons, Ltd.     

基于支持向量机多分类的眼电辅助肌电的人机交互

针对单一肌电信号在控制系统中正确识别率不高问题,设计并实现了一种基于支持向量机(SVM)多分类的眼电(EOG)辅助肌电(EMG)的人机交互(HCI)系统。该系统采用改进小波包算法和阈值法分别对EMG信号和EOG信号进行特征提取,并对特征向量融合;然后提取特征参数作为SVM的输入来识别EMG信号和EOG信号动作模式,根据分类结果生成控制命令。实验证明,该系统比单一肌电控制系统更便于操作,稳定性好,正确识别率高。     

视频监控系统中基于MPEG编码域的运动检测方法

运动检测是视频监控系统中极有实用价值的技术，该文简单介绍了MPEG编码的基本特征，并利用这些特征在编码域中进行运行检测，提出了一种利用运动矢量进行运动检测的方法，该方法不但能检测出运动，还能区分出摄像头和物体的运动，并判断运动类型。     

风格强度可变的人脸风格迁移网络

目的 针对人脸风格迁移算法StarGAN （star generative adversarial network）、MSGAN （mode seeking generative adversarial network）等存在细节风格学习不佳、迁移效果单一和生成图像失真等缺点，提出一种能够降低失真并生成不同风格强度图像的人脸风格迁移算法MStarGAN （multilayer StarGAN）。方法 首先，通过特征金字塔网络（feature pyramid network，FPN）构建前置编码器，生成蕴含有图像细节特征的多层特征向量，增强生成图像在风格传输时能学习到的风格图像的细节风格；其次，使用前置编码器对原图像及风格图像各生成一个风格向量并进行组合，利用组合后的风格向量进行风格传输，使生成图像具有不同的风格迁移强度；最后，采用权重解调算法作为生成器中的风格传输模块，通过对卷积权重的操作代替在特征图上的归一化操作，消除特征图中的特征伪影，减少生成图像中的失真。结果 在Celeba_HQ数据集上进行实验，与MSGAN、StarGAN v2等对比算法相比，在参考引导合成实验中，MStarGAN的FID （Frechét inception distance score）指标分别降低了18.9和3.1，LPIPS （learnedperceptual image patch similarity）指标分别提升了0.094和0.018。在潜在引导合成实验中，MStarGAN的FID指标分别降低了20.2和0.8，LPIPS指标分别提升了0.155和0.92，并能够生成具有不同风格强度的结果图像。结论 提出的算法能够传输图像的细节风格，生成具有不同强度的输出图像，并减少生成图像的失真。     

Real Time Animation of Virtual Humans: A Trade‐off Between Naturalness and Control

Virtual humans are employed in many interactive applications using 3D virtual environments, including (serious) games. The motion of such virtual humans should look realistic (or ‘natural’) and allow interaction with the surroundings and other (virtual) humans. Current animation techniques differ in the trade‐off they offer between motion naturalness and the control that can be exerted over the motion. We show mechanisms to parametrize, combine (on different body parts) and concatenate motions generated by different animation techniques. We discuss several aspects of motion naturalness and show how it can be evaluated. We conclude by showing the promise of combinations of different animation paradigms to enhance both naturalness and control.     

Talk—an interactive system for a small computer

TALK is an interactive file creation and editing system implemented on a very small computer. It provides facilities for FORTRAN syntax checking and an interactive ‘desk-calculator’. Major computing tasks are sent over a high speed link to CDC 6600/6400 computers A number of teleprinters can be controlled by TALK in the usual way, but also the card reader/line printer pair may be used as a terminal. This is useful for creating or printing large files When a user has logged in he can work in one of several modes. These include INPUT, EDIT and CALCULATE modes. It is possible to nest modes, for example, by entering CALCULATE mode while in INPUT mode without losing one's place in the input file An unusual feature of TALK is the ‘inclusion’ facility. This enables a user to specify previously filed text to be included within another file, either as the file is being created or dynamically whenever the file is used. It also gives users the ability to build up macros of commonly used commands. Many system commands are in fact macros with only simple primitives provided as executable code.