Assessing similarity models for human‐motion retrieval applications

The development of motion capturing devices poses new challenges in the exploitation of human‐motion data for various application fields, such as computer animation, visual surveillance, sports, or physical medicine. Recently, a number of approaches dealing with motion data have been proposed, suggesting characteristic motion features to be extracted and compared on the basis of similarity. Unfortunately, almost each approach defines its own set of motion features and comparison methods; thus, it is hard to fairly decide which similarity model is the most suitable for a given kind of human‐motion retrieval application. To cope with this problem, we propose the human motion model evaluator, which is a generic framework for assessing candidate similarity models with respect to the purpose of the target application. The application purpose is specified by a user in form of a representative sample of categorized motion data. Respecting such categorization, the similarity models are assessed from the effectiveness and efficiency points of view using a set of space‐complexity, information‐retrieval, and performance measures. The usability of the framework is demonstrated by case studies of three practical examples of retrieval applications focusing on recognition of actions, detection of similar events, and identification of subjects. Copyright © 2015 John Wiley & Sons, Ltd.     

基于非线性流形学习的3维人体运动合成

为了实现3维人体运动的有效合成,提出了一种基于非线性流形学习的3维人体运动合成框架及算法,并可应用于方便、快捷、用户可控的3维人体运动合成。该合成算法框架先采用非线性流形降维方法将高维运动样本映射到低维流形上,同时求解其本征运动语义参数空间的表达,然后将用户在低维运动语义参数空间中交互生成的样本通过逆向映射重建得到具有新运动语义特征的3维运动序列。实验结果表明该方法不仅能够对运动物理参数（如特定关节的运动位置、物理运动特征）进行较为精确的控制,还可用于合成具有高层运动语义（运动风格）的新运动数据。与现有运动合成方法比较,该方法具有用户可控、交互性强等优点,能够应用于常见3维人体运动数据的高效生成。     

基于运动能量模型的人体运动捕捉数据库的检索

不同于经典的基于几何空间位置的检索方法,首先提出基于运动能量的描述人体运动的模型,在此基础上引入熵的概念,提取能体现运动特征的关键关节作为衡量动作相似性的基准;然后利用Keogh定界算法建立索引,以加快检索速度;最后,利用动态时间变形算法计算运动例子和检索集之间的相似度,确定检索结果集.实验结果表明,该方法速度快、准确性高.     

A path browser for exploratory motion assembly

We present an interactive method for creating animation sequences of characters based on captured motion data in an exploratory way as in assembling construction toys. The key component of our method is a path browser that can retrieve and visualize paths as diverse as possible connecting a given pair of initial and final motion fragments instantiated in the space. With the aid of our path browser, the user can develop large‐scale assembly of motions gradually through iterations of arranging and putting together motion fragments. For the efficient retrieval of connecting paths, we use a bidirectional search tree that grows from the initial and final configurations simultaneously under the guidance of a mixed strategy for both global exploration and local optimization. The usefulness of our approach is demonstrated through experiments with a variety of motion data including box moving, basketball, and breakdance data. Copyright © 2016 John Wiley & Sons, Ltd.     

Indexing and Retrieving Motions of Characters in Close Contact

Human motion indexing and retrieval are important for animators due to the need to search for motions in the database which can be blended and concatenated. Most of the previous researches of human motion indexing and retrieval compute the Euclidean distance of joint angles or joint positions. Such approaches are difficult to apply for cases in which multiple characters are closely interacting with each other, as the relationships of the characters are not encoded in the representation. In this research, we propose a topology-based approach to index the motions of two human characters in close contact. We compute and encode how the two bodies are tangled based on the concept of rational tangles. The encoded relationships, which we define as {it TangleList}, are used to determine the similarity of the pairs of postures. Using our method, we can index and retrieve motions such as one person piggy-backing another, one person assisting another in walking, and two persons dancing / wrestling. Our method is useful to manage a motion database of multiple characters. We can also produce motion graph structures of two characters closely interacting with each other by interpolating and concatenating topologically similar postures and motion clips, which are applicable to 3D computer games and computer animation.     

Generalized Model‐Based Human Motion Recognition with Body Partition Index Maps

Content‐based human motion analysis has captured extensive concerns of researchers from the domains of computer animation, human‐machine interaction, entertainment, etc. However, it is a non‐trivial task due to the spatial and temporal variations in the motion data. In this paper, we propose a generalized model (GM)‐based approach to model the variations and accurately recognize motion patterns. We partition the human character model into five parts, and extract the features of the submotions of each specific body part using clustering techniques. These features from the training trials in each class are combined to build the GM. We propose a new penalty based similarity measure for DTW to be used with the GMs for isolated motion recognition. On the other hand, from the GMs five body partition index maps are constructed and used for matching together with a flexible end point detection scheme during continuous motion recognition. In the experiments, we examine the effectiveness and efficiency of the approach in both isolated motion and continuous motion recognition. The results show that our proposed method has good performance compared with other state‐of‐the‐art methods in recognition accuracy and processing speed.     

运动串:一种用于行为分割的运动捕获数据表示方法

运动数据的行为分割是运动捕获过程中非常重要的一环.针对现有分割方法的不足,提出了一种可用于行为分割的运动数据表示方法,并基于该表示实现了数据的行为分割.运动数据经过谱聚类(spectral clustering)、时序恢复和最大值滤波法(max filtering)后生成一个字符串,该字符串称为运动串,然后采用后缀树(suffix tree)分析运动串,提取出所有静态子串和周期子串,对这些子串进行行为标注,从而实现运动数据的行为分割.实验表明,基于运动串的分割具有较好的鲁棒性和分割效果.     

基于检索的人体运动识别和模拟

运动识别和模拟是人体运动分析中重要的研究内容，实现了以检索为基础的实验性的运动分析系统．小波矩具有平移、旋转和缩放不变性，能够提取局部多层次特征，被用来作为特征描述运动序列以及动作，根据相似性实现运动识别，利用动态时间变形（DTW）实现序列的动作匹配，在poser建模、依据正多面体分割的金字塔模型得到多视点投影视频的数据库中进行识别和匹配，并以三维模型的形式显示出来．实验结果可以模拟人体运动以及为进一步分析提供初始分析数据．     

基于优化的DTW算法的人体运动数据检索

随着大量三维人体运动数据库的建立,使得在数据库中实现基于内容的三维人体运动检索面临着诸多困难,文中提出一种分阶段的动态时间变形(DTW)优化算法的人体运动数据检索技术,可有效检索出逻辑上相似的运动。该算法首先对齐两个运动序列的坐标位置,基于窗口距离构造距离矩阵。其次采用基于全局和局部约束的DTW优化算法进行相似度匹配,得到两个运动间的对应关系。最后通过归一化相似度和DTW平均距离分阶段判断运动的相似性。实验结果表明,分阶段的DTW优化算法在提高效率的同时对长度不等的运动能取得较好的检索结果。     

一种基于时空兴趣点的人体动作识别方法

人体动作识别是计算机视觉中一个具有挑战性的课题,同时也具有广阔的应用前景。本文采用基于时空兴趣点的人体运动表示方法,实现了时空兴趣点的检测算法和人体动作识别,解决了检测过程中检测尺度的选择问题,取得了较好的识别结果。与经典的时空兴趣点检测方法相比,这种方法对比较简单的动作更加有效。     

Finding Syntactic Structures from Human Motion Data

We present a new approach to motion rearrangement that preserves the syntactic structures of an input motion automatically by learning a context‐free grammar from the motion data. For grammatical analysis, we reduce an input motion into a string of terminal symbols by segmenting the motion into a series of subsequences, and then associating a group of similar subsequences with the same symbol. To obtain the most repetitive and precise set of terminals, we search for an optimial segmentation such that a large number of subsequences can be clustered into groups with little error. Once the input motion has been encoded as a string, a grammar induction algorithm is employed to build up a context‐free grammar so that the grammar can reconstruct the original string accurately as well as generate novel strings sharing their syntactic structures with the original string. Given any new strings from the learned grammar, it is straightforward to synthesize motion sequences by replacing each terminal symbol with its associated motion segment, and stitching every motion segment sequentially. We demonstrate the usefulness and flexibility of our approach by learning grammars from a large diversity of human motions, and reproducing their syntactic structures in new motion sequences.     

Probabilistic nod generation model based on speech and estimated utterance categories

ABSTRACT

We proposed and evaluated a probabilistic model that generates nod motions based on utterance categories estimated from the speech input. The model comprises two main blocks. In the first block, dialog act-related categories are estimated from the input speech. Considering the correlations between dialog acts and head motions, the utterances are classified into three categories having distinct nod distributions. Linguistic information extracted from the input speech is fed to a cluster of classifiers which are combined to estimate the utterance categories. In the second block, nod motion parameters are generated based on the categories estimated by the classifiers. The nod motion parameters are represented as probability distribution functions (PDFs) inferred from human motion data. By using speech energy features, the parameters are sampled from the PDFs belonging to the estimated categories. The effectiveness of the proposed model was evaluated using an android robot, through subjective experiments. Experiment results indicated that the motions generated by our proposed approach are considered more natural than those of a previous model using fixed nod shapes and hand-labeled utterance categories.     

面向内容的三维人体运动检索技术研究综述

三维人体运动数据检索技术是计算机动画领域的研究热点,系统分析了当前基于内容的三维人体运动数据检索技术的研究现状。分析比较了基于运动数值特征、基于运动能量特征和基于语义信息特征三种运动数据的特征表示方法;从局部相似性度量和全局相似性判断两个角度阐述了运动相似性匹配算法;总结目前基于内容的人体运动数据检索技术中存在的问题,探讨了未来开展基于语义的检索技术所需的工作。     

Human motion generation with multifactor models

To generate human motions with various specific attributes is a difficult task because of high dimensionality and complexity of human motions. This paper presents a novel human motion model for generating and editing motions with multiple factors. A set of motions performed by several actors with various styles was captured for constructing a well‐structured motion database. Subsequently, MICA (multilinear independent component analysis) model that combines ICA and conventional multilinear framework was adopted for the construction of a multifactor model. With this model, new motions can be synthesized by interpolation and through solving optimization problems for the specific factors. Our method offers a practical solution to edit stylistic human motions in a parametric space learnt with MICA model. We demonstrated the power of our method by generating and editing sideways stepping, reaching, and striding over obstructions using different actors with various styles. The experimental results show that our method can be used for interactive stylistic motion synthesis and editing. Copyright © 2011 John Wiley & Sons, Ltd.     

Human motion recognition exploiting radar with stacked recurrent neural network

We develop a novel radar-based human motion recognition technique that exploits the temporal sequentiality of human motions. The stacked recurrent neural network (RNN) with long short-term memory (LSTM) units is employed to extract sequential features for automatic motion classification. The spectrogram of raw radar data is used as the network input to utilize the time-varying Doppler and micro-Doppler signatures for human motion characterization. Based on experimental data, we verified that a stacked RNN with two 36-cell LSTM layers successfully classifies six different types of human motions.     

人体运动的函数数据分析与合成

在许多虚拟现实的应用中,虚拟人作为人在计算机中的表示,是提高其交互能力和沉浸感的重要因素之一.然而,对于虚拟人建模而言,合成逼真、可控的虚拟人运动仍然是具有挑战性的课题.为此,提出了一种基于函数数据分析的人体运动合成方法.通过对一组样本运动进行函数主成分分析,构建出一个由特征运动构成的低维函数子空间.该低维子空间不仅能够有效地刻画样本序列内在的变化规律,而且也为有目的的运动合成提供了方法.在该空间中,通过控制各特征运动的系数即可合成出逼真、平滑的运动序列.该合成过程没有耗时的计算,因此能够满足各种实时应用     

UKI: universal Kinect‐type controller by ICE Lab

Universal Kinect‐type‐controller by ICE Lab (UKI, pronounced as ‘You‐key’) was developed to allow users to control any existing application by using body motions as inputs. The middleware works by converting detected motions into keyboard and/or mouse‐click events and sending them to a target application. This paper presents the structure and design of core modules, along with examples from real cases to illustrate how the middleware can be configured to fit a variety of applications. We present our designs for interfaces that decode all configuration details into a human‐interpretable language, and these interfaces significantly promote user experience and eliminate the need for programming skill. The performance of the middleware is evaluated on fighting‐game motion data, and we make the data publicly available so that they can be used in other researches. UKI welcomes its use by everyone without any restrictions on use; for instance, it can be used to promote healthy life through a means of gaming and/or used to conduct serious research on motion systems. The middleware serves as a shortcut in the development of motion applications—coding of an application to detect motions can be replaced with simple clicks on UKI. Copyright © 2017 John Wiley & Sons, Ltd.     

人体运动的非线性降维及新运动生成

为了提高现有运动数据的可重用性,生成更为丰富的新运动,提出快速自适应比例高斯过程隐变量模型,以及基于该模型的人体运动数据降维及运动生成方法.通过对运动数据进行统计学习,获得运动数据在隐空间的一个低维映射来实现非线性降维,同时获得了该运动的姿态空间的概率分布,其大小反映了该姿态的自然逼真程度;在给定末端约束条件下求取满足约束的、同时概率最大的姿态,并将其作为逆向运动学的解,克服了传统逆向运动学算法计算烦琐、效果不逼真的缺点.实验结果表明,该模型具有更快的收敛速度和更高的收敛精度,同时能够自适应运动编辑的方向,有效地扩大运动的可编辑幅度.     

Combining inverse blending and Jacobian‐based inverse kinematics to improve accuracy in human motion generation

We present in the paper a hybrid method for motion editing combining motion blending and Jacobian‐based inverse kinematics (IK). When the original constraints are changed, a blending‐based IK solver is first employed to find an adequate joint configuration coarsely. Using linear motion blending, this search corresponds to a gradient‐based minimization in the weight space. The found solution is then improved by a Jacobian‐based IK solver by further minimizing the distance between the end effectors and constraints. To accelerate the searching in the weight space, we introduce a weight map, which pre‐computes the good starting positions for the gradient‐based minimization. The advantages of our approach are threefold: first, more realistic motions can be generated by utilizing motion blending techniques, compared with pure Jacobian‐based IK. The blended results also increase the rate of convergence of the Jacobian‐based IK solver. Second, the Jacobian‐based IK solver modifies poses in the pose configuration space and the computational cost does not scale with the number of examples. Third, it is possible to extrapolate the given example motions with a Jacobian‐based IK solver, while it is generally difficult with pure blending‐based techniques. Copyright © 2014 John Wiley & Sons, Ltd.