Style Invariant Locomotion Classification for Character Control

We present a real‐time system for character control that relies on the classification of locomotive actions in skeletal motion capture data. Our method is both progress dependent and style invariant. Two deep neural networks are used to correlate body shape and implicit dynamics to locomotive types and their respective progress. In comparison to related work, our approach does not require a setup step and enables the user to act in a natural, unconstrained manner. Also, our method displays better performance than the related work in scenarios where the actor performs sharp changes in direction and highly stylized motions while maintaining at least as good performance in other scenarios. Our motivation is to enable character control of non‐bipedal characters in virtual production and live immersive experiences, where mannerisms in the actor's performance may be an issue for previous methods.     

基于多自主智能体的群体动画创作

群体动画一直是计算机动画界一个具有挑战性的研究方向，提出了一个基于多自主智能体的群体动画创作框架：群体中的各角色作为自主智能体，能感知环境信息，产生意图，规划行为，最后通过运动系统产生运动来完成行为和实现意图，与传统的角色运动生成机理不同，首先采用运动捕获系统建立基本运动库，然后通过运动编辑技术对基本运动进行处理以最终得到角色运动，应用本技术，动画师只需“拍摄”角色群体的运动就能创作群体动画，极大地提高了制作效率。     

基于非线性流形学习的3维人体运动合成

为了实现3维人体运动的有效合成,提出了一种基于非线性流形学习的3维人体运动合成框架及算法,并可应用于方便、快捷、用户可控的3维人体运动合成。该合成算法框架先采用非线性流形降维方法将高维运动样本映射到低维流形上,同时求解其本征运动语义参数空间的表达,然后将用户在低维运动语义参数空间中交互生成的样本通过逆向映射重建得到具有新运动语义特征的3维运动序列。实验结果表明该方法不仅能够对运动物理参数（如特定关节的运动位置、物理运动特征）进行较为精确的控制,还可用于合成具有高层运动语义（运动风格）的新运动数据。与现有运动合成方法比较,该方法具有用户可控、交互性强等优点,能够应用于常见3维人体运动数据的高效生成。     

一种实时虚拟人反应式动画生成算法

使用运动捕获数据驱动与动力学模拟相结合的控制方法,可以产生既真实又能对外界施加的作用力作出反应的人体运动.为减少以前方法中反应式运动数据搜索的时间开销并去除动画师需要的手工调节工作,采用并行计算,并引入人工神经网络的方法,根据虚拟人主要关节的位姿对反应运动类型进行预测,得到需搜索的反应运动子类型库.另外,对搜索匹配的算法进行改善以提高搜索效率.实验结果表明:系统中的虚拟人的运动能在两种控制方式之间灵活切换,并能实时响应外界的交互作用.     

Memory-Based Human Motion Simulation for Computer-Aided Ergonomic Design

A novel memory-based motion simulation (MBMS) model was developed as a general framework for simulating natural human motions for computer-aided ergonomic design. The MBMS model utilizes real human motion samples recorded in motion capture experiments as templates for simulating novel motions. Such human motion samples are stored in a motion database. When a user submits an input simulation scenario to the model, a motion search engine termed the ldquoroot motion finderrdquo in the model searches the motion database and retrieves the motion samples that closely match the given scenario. The retrieved motions, referred to as root motions, may significantly differ from one another in the underlying movement technique. Such variability within the root motion set is analyzed and graphically summarized by a model component termed the motion variability analyzer. This analysis helps users rapidly identify alternative movement techniques for the given input simulation scenario and simulate human motions based on alternative movement techniques. Since root motions do not exactly satisfy but only closely match the input simulation scenario, a motion modification (MoM) algorithm adapts them to fit the scenario by systematically deforming them in the joint angle-time domain. The MoM algorithm retains the root motions' fundamental spatial-temporal structure and minimizes deviations from the root motions during such deformations. The MBMS model overcomes limitations of existing simulation models and achieves the following: 1) simulation of categorically different motions based on a single unified model; 2) simple and efficient learning of new motion behaviors; and 3) representation and simulation of human motion variability.     

Pen-to-mime: Pen-based interactive control of a human figure

In this paper a pen-based intuitive interface is presented, that controls a virtual human figure interactively. Recent commercial pen devices can detect not only the pen positions but also the pressure and tilt of the pen. We utilize such information to make a human figure perform various types of motions in response to the pen movements manipulated by the user. The figure walks, runs, turns and steps along the trajectory and speed of the pen. The figure also bends, stretches and tilts in response to the tilt of the pen. Moreover, it ducks and jumps in response to the pen pressure. Using our interface, the user controls a virtual human figure intuitively as if he or she were holding a virtual puppet and playing with it.

In addition to the interface design and implementation, this paper describes a motion generation engine to produce various motion based on varying parameters that are given by the pen interface. We take a motion blending approach and construct motion blending modules with a set of small number of motion capture data for each type of motions. Finally, we present the results from user experiments and comparison with a transitional gamepad-based interface.     

一种基于传感器的人体上肢运动实时跟踪方法

实时跟踪人体运动是人机交互的重要研究课题,可以广泛应用于虚拟现实、虚拟人运动合成、聋人手语自动生成、计算机3D动画、机器人运动控制、远程人机交互等领域。文中介绍了一种基于传感器的人体上肢运动实时跟踪方法,给出了该方法的虚拟人模型、计算原理与校正方法,最后介绍了整个方法的实现以及在中国聋人手语自动合成中的应用。该方法具有使用传感器少,运动跟踪精度高、计算过程简单而且速度快等特点。     

基于四元数描述和EMD的人体运动捕获数据检索

针对运动捕获数据的高效匹配问题,提出了一种新的基于四元数描述和EMD（ Earth Mover＆#39;s Distance）的人体运动检索算法。该算法主要包括特征提取和运动匹配两部分。在特征提取部分,为了解决高维数据检索效率低的问题,引入了四元数描述符对关节点的数据信息特征进行描述,通过映射姿态分布的原始数据,并采取K-means聚类方法对待查询动作和运动数据库的特征数据进行降维并归类。在运动匹配部分,根据聚类结果,建立每个特征数据集的距离矩阵,将匹配问题转换为运输优化问题。然后,用EMD算法度量待查询动作和数据库动作之间的相似值。仿真实验结果证明了提出的算法是有效的。     

Dynamics-based analysis and synthesis of human locomotion

One of the best ways to synthesize realistic human motions is to animate characters from captured motion data that inherently respect motion laws. Retargeting and interpolation methods are often used to adapt these motions to different representations of the character and to various environmental constraints but they may introduce physical inaccuracies, although the synthesized motions are natural looking. This paper presents a method for evaluating the physical correctness of retargeted and interpolated locomotions using an inverse dynamics analysis. Furthermore, we propose to improve an initial database with analysed motions that are synthesized again by using a forward dynamics approach. The analysis algorithm consists in determining the resulting forces and torques at joints. With this intention, we develop an automatic creation process of the mass/inertia model of the character. Then using support phase recognition, we compute resulting forces and torques by an inverse dynamics method. The retargeting and the interpolation methods change the physics of the motions. This change is evaluated by using the results of our analysis on artificial and real motions and by using literature results and experimental data from force plates. The evaluation relies on the study of several retargeting and interpolation parameters such as the global size of the character or the structure of the model. The output of this evaluation, the resulting forces and torques at joints, are used to produce physically valid motions by using forward dynamics simulation. With this purpose, we introduce forces and torques normalizations, and finally the synthesized motions may improve the initial database.     

Constrained synthesis of textural motion for articulated characters

Obtaining high-quality, realistic motions of articulated characters is both time consuming and expensive, necessitating the development of easy-to-use and effective tools for motion editing and reuse. We propose a new simple technique for generating constrained variations of different lengths from an existing captured or otherwise animated motion. Our technique is applicable to textural motions, such as walking or dancing, where the motion sequence can be decomposed into shorter motion segments without an obvious temporal ordering among them. Inspired by previous work on texture synthesis and video textures, our method essentially produces a reordering of these shorter segments. Discontinuities are eliminated by carefully choosing the transition points and applying local adaptive smoothing in their vicinity, if necessary. The user is able to control the synthesis process by specifying a small number of simple constraints.     

基于视频的现实人物运动创作

提出一种根据用户指定的人体运动和观察视角生成真实感视频的方法.首先采集演员进行少数基本运动时的多视角视频数据库,并使用无标记运动捕捉的方法获得任意时刻人体对应的骨骼和3D模型.其次,用户对人体骨架指定运动并设定视角,以此定义目标视频.实验结果验证了文中方法能够利用有限的数据库合成演员在用户指定运动和视角下的真实感视频.     

视觉感知的端到端自动驾驶运动规划综述

视觉感知模块能够利用摄像机等视觉传感器获取丰富的图像和视频信息,进而检测自动驾驶汽车视野中的车辆、行人与交通标识等信息,是自动驾驶最有效、成本最低的感知方式之一。运动规划为自主车辆提供从车辆初始状态到目标状态的一系列运动参数和驾驶动作,而端到端的模型能够直接从感知的数据获取车辆的运动参数,因而受到广泛的关注。为了全面反映视觉感知的端到端自动驾驶运动规划方法的研究进展,本文对国内外公开发表的具有代表性和前沿的论文进行了概述。首先分析端到端方法的应用,以及视觉感知和运动规划在端到端自动驾驶中的作用,然后以自主车辆的学习方式作为分类依据,将视觉感知的端到端自动驾驶运动规划的实现方法分为模仿学习和强化学习两大类,并对各类方法的不同算法进行了归纳和分析;考虑到现阶段端到端模型的研究面临着虚拟到现实的任务,故对基于迁移学习的方法进行了梳理。最后列举与自动驾驶相关的数据集和仿真平台,总结存在的问题和挑战,对未来的发展趋势进行思考和展望。视觉感知的端到端自动驾驶运动规划模型的普适性强且结构简单,这类方法具有广阔的应用前景和研究价值,但是存在不可解释和难以保证绝对安全的问题,未来需要更多的研究改善端到端模型存在的局限性。     

Recurrent Motion Refiner for Locomotion Stitching

Stitching different character motions is one of the most commonly used techniques as it allows the user to make new animations that fit one's purpose from pieces of motion. However, current motion stitching methods often produce unnatural motion with foot sliding artefacts, depending on the performance of the interpolation. In this paper, we propose a novel motion stitching technique based on a recurrent motion refiner (RMR) that connects discontinuous locomotions into a single natural locomotion. Our model receives different locomotions as input, in which the root of the last pose of the previous motion and that of the first pose of the next motion are aligned. During runtime, the model slides through the sequence, editing frames window by window to output a smoothly connected animation. Our model consists of a two-layer recurrent network that comes between a simple encoder and decoder. To train this network, we created a sufficient number of paired data with a newly designed data generation. This process employs a K-nearest neighbour search that explores a predefined motion database to create the corresponding input to the ground truth. Once trained, the suggested model can connect various lengths of locomotion sequences into a single natural locomotion.     

智能车运动仿真平台设计

为了辅助开发人员进行智能车PID控制参数的设定及修正, 设计了一种智能车运动仿真平台, 该仿真平台由三部分模块组成. 轨道模块内部记录有当前仿真环境下的跑道形状及参数信息; 车辆模块内建有智能车的运动模型及参数; 运行仿真模块用于模拟物理运行环境并建立有坐标信息. 运行过程中, 运行仿真模块接收来自用户的车辆运动及PID控制参数, 在车辆模块的协助下, 实现对车辆位置以及误差信息的计算. 实验表明, 与现有算法相比, 本平台不受传感器类型的限制, 可以更真实的仿真车辆的运动状态, 同时为开发人员进行PID参数调整提供了实践依据.     

Reconstructing whole-body motions with wrist trajectories

Reconstructing whole-body motions using only a low-dimensional input reduces the cost of and efforts for performance capture significantly, and yet remains a challenging problem. We introduce a novel technique that synthesizes whole-body motion using the two wrist trajectories. Given the wrist trajectories, we first determine the optimal ankle trajectories from a large number of candidate ankle paths obtained from example poses in the motion database. The optimal trajectory is efficiently achieved by solving for the shortest path problem in a directed acyclic graph. Next, we use both the wrist and ankle trajectories as the low-dimensional control signals to achieve the whole-body pose at each time step. We show that our method can reconstruct various whole-body motions that can be recognized by arm motions, such as walking, stepping, and in-place upper-body motions. Comparisons with ground truth motions and with other methods are provided.     

An efficient approach to human motion modeling for the verification of human-centric product design and manufacturing in virtual environments

The efficient and reliable human-integrated design of products and processes is a major goal of the manufacturing industry. Thus, numerous human-related product functionality and manufacturability aspects need to be verified, using simulation, in the context of the product development procedures. The natural representation of human motions in virtual environments is crucial for the reliability of the simulation results. In this context, the paper presents an efficient approach to human motion analysis and modeling with respect to the anthropometric parameters, based on real motion data. Statistical methods employed for the analysis of the data and additive motion models are derived. The models are capable of predicting human motion and driving digital humans in product and worker simulation environments. A specific test case is presented to demonstrate the application of the suggested methodology on a real industrial problem.     

A motion capture‐based control‐space approach for walking mannequins

Virtual mannequins need to navigate in order to interact with their environment. Their autonomy to accomplish navigation tasks is ensured by locomotion controllers. Control inputs can be user‐defined or automatically computed to achieve high‐level operations (e.g. obstacle avoidance). This paper presents a locomotion controller based on a motion capture edition technique. Controller inputs are the instantaneous linear and angular velocities of the walk. Our solution works in real time and supports at any time continuous changes of inputs. The controller combines three main components to synthesize locomotion animations in a four‐stage process. First, the Motion Library stores motion capture samples. Motion captures are analysed to compute quantitative characteristics. Second, these characteristics are represented in a linear control space. This geometric representation is appropriate for selecting and weighting three motion samples with respect to the input state. Third, locomotion cycles are synthesized by blending the selected motion samples. Blending is done in the frequency domain. Lastly, successive postures are extracted from the synthesized cycles in order to complete the animation of the moving mannequin. The method is demonstrated in this paper in a locomotion‐planning context. Copyright © 2006 John Wiley & Sons, Ltd.     

Interpolation synthesis of articulated figure motion

Most conventional media depend on engaging and appealing characters. Empty spaces and buildings would not fare well as television or movie programming, yet virtual reality usually offers up such spaces. The problem lies in the difficulty of creating computer generated characters that display real time, engaging interaction and realistic motion. Articulated figure motion for real time computer graphics offers one solution to this problem. A common approach stores a set of motions and lets you choose one particular motion at a time. The article describes a process that greatly expands the range of possible motions. Mixing motions selected from a database lets you create a new motion to exact specifications. The synthesized motion retains the original motions' subtle qualities, such as the realism of motion capture or the expressive, exaggerated qualities of artistic animation. Our method provides a new way to achieve inverse kinematics capability-for example, placing the hands or feet of an articulated figure in specific positions. It proves useful for both real time graphics and prerendered animation production. The method, called interpolation synthesis, is based on motion capture data and it provides real time character motion for interactive entertainment or avatars in virtual worlds     

Real-time Unsupervised Segmentation of human whole-body motion and its application to humanoid robot acquisition of motion symbols

An interactive loop between motion recognition and motion generation is a fundamental mechanism for humans and humanoid robots. We have been developing an intelligent framework for motion recognition and generation based on symbolizing motion primitives. The motion primitives are encoded into Hidden Markov Models (HMMs), which we call “motion symbols”. However, to determine the motion primitives to use as training data for the HMMs, this framework requires a manual segmentation of human motions. Essentially, a humanoid robot is expected to participate in daily life and must learn many motion symbols to adapt to various situations. For this use, manual segmentation is cumbersome and impractical for humanoid robots. In this study, we propose a novel approach to segmentation, the Real-time Unsupervised Segmentation (RUS) method, which comprises three phases. In the first phase, short human movements are encoded into feature HMMs. Seamless human motion can be converted to a sequence of these feature HMMs. In the second phase, the causality between the feature HMMs is extracted. The causality data make it possible to predict movement from observation. In the third phase, movements having a large prediction uncertainty are designated as the boundaries of motion primitives. In this way, human whole-body motion can be segmented into a sequence of motion primitives. This paper also describes an application of RUS to AUtonomous Symbolization of motion primitives (AUS). Each derived motion primitive is classified into an HMM for a motion symbol, and parameters of the HMMs are optimized by using the motion primitives as training data in competitive learning. The HMMs are gradually optimized in such a way that the HMMs can abstract similar motion primitives. We tested the RUS and AUS frameworks on captured human whole-body motions and demonstrated the validity of the proposed framework.     

基于虚拟人合成技术的中国手语合成方法

介绍了一种中国手语合成方法,实现了文本到中国手语的自动翻译,并使用虚拟人合成技术,实现了中国手语的合成与显示,以此帮助聋人与听力正常人之间实现自然交流.在该方法中,首先应用两只数据手套和3个6自由度位置跟踪器,基于运动跟踪的原理,记录真实人体演示每个手语词的运动数据,建立一个初始的手语词运动数据库.然后,应用一种基于控制点的人体运动编辑方法,对每个手语词的运动数据进行编辑与微调,最后得到一个高质量的手语词运动数据库.当给定一个文本句子时,应用人体运动合成方法,对每个手语词的手语运动片段进行拼接合成,最终生成一个完整的手语运动,并基于VRML的人体运动显示方法将合成的运动逼真地显示出来.基于该方法,在PC/Windows/VC6.0环境下实现了一个中国聋人手语合成系统.该系统采集了<中国手语>(含续集)中收录的5 596个手语词,可以合成一般生活与教学用语.经聋校的老师和学生确认,合成手语准确逼真,可以广泛应用于教学、电视、Internet 等多种大众媒体,帮助聋人参与其他听力正常人的活动,具有广泛的应用前景和重要的社会意义.