基于非线性流形学习的3维人体运动合成

为了实现3维人体运动的有效合成,提出了一种基于非线性流形学习的3维人体运动合成框架及算法,并可应用于方便、快捷、用户可控的3维人体运动合成。该合成算法框架先采用非线性流形降维方法将高维运动样本映射到低维流形上,同时求解其本征运动语义参数空间的表达,然后将用户在低维运动语义参数空间中交互生成的样本通过逆向映射重建得到具有新运动语义特征的3维运动序列。实验结果表明该方法不仅能够对运动物理参数（如特定关节的运动位置、物理运动特征）进行较为精确的控制,还可用于合成具有高层运动语义（运动风格）的新运动数据。与现有运动合成方法比较,该方法具有用户可控、交互性强等优点,能够应用于常见3维人体运动数据的高效生成。     

Indexing and Integrating Multiple Features for WWW Images

In this paper, we present a novel indexing technique called Multi-scale Similarity Indexing (MSI) to index image's multi-features into a single one-dimensional structure. Both for text and visual feature spaces, the similarity between a point and a local partition's center in individual space is used as the indexing key, where similarity values in different features are distinguished by different scale. Then a single indexing tree can be built on these keys. Based on the property that relevant images have similar similarity values from the center of the same local partition in any feature space, certain number of irrelevant images can be fast pruned based on the triangle inequity on indexing keys. To remove the “dimensionality curse” existing in high dimensional structure, we propose a new technique called Local Bit Stream (LBS). LBS transforms image's text and visual feature representations into simple, uniform and effective bit stream (BS) representations based on local partition's center. Such BS representations are small in size and fast for comparison since only bit operation are involved. By comparing common bits existing in two BSs, most of irrelevant images can be immediately filtered. To effectively integrate multi-features, we also investigated the following evidence combination techniques—Certainty Factor, Dempster Shafer Theory, Compound Probability, and Linear Combination. Our extensive experiment showed that single one-dimensional index on multi-features improves multi-indices on multi-features greatly. Our LBS method outperforms sequential scan on high dimensional space by an order of magnitude. And Certainty Factor and Dempster Shafer Theory perform best in combining multiple similarities from corresponding multiple features.     

一种可编辑的三维人体蒙皮网格动画合成方法*

为了使虚拟人情感表达更加逼真,提出了一种可编辑的三维人体动画合成方法。首先建立了三维网格和人体骨架之间的蒙皮关系,构造了关节运动角度的参数模型;然后,对三维人体骨架进行操作,利用线性混合蒙皮算法,驱动虚拟人体的运动。在Visual C++平台上实现了在特定情感状态下的肢体动画,验证了该方法的有效性。     

Communicating emotions and mental states to robots in a real time parallel framework using Laban movement analysis

This paper presents a parallel real time framework for emotions and mental states extraction and recognition from video fragments of human movements. In the experimental setup human hands are tracked by evaluation of moving skin-colored objects. The tracking analysis demonstrates that acceleration and frequency characteristics of the traced objects are relevant for classification of the emotional expressiveness of human movements. The outcomes of the emotional and mental states recognition are cross-validated with the analysis of two independent certified movement analysts (CMA’s) who use the Laban movement analysis (LMA) method. We argue that LMA based computer analysis can serve as a common language for expressing and interpreting emotional movements between robots and humans, and in that way it resembles the common coding principle between action and perception by humans and primates that is embodied by the mirror neuron system. The solution is part of a larger project on interaction between a human and a humanoid robot with the aim of training social behavioral skills to autistic children with robots acting in a natural environment.     

压缩视频流中主要运动的定性分析

随着视频数据量的急剧增加，如何为海量视频数据建立高效索引并提供查询和检索能力成为目前信息检索领域重要的研究课题，许多研究表明，通过对视频中主要运动的定性分析，有助于实现视频内容的索引和检索，该文根据运动矢量的分布特性，运动矢量之间的相对关系特性等，提出了一种压缩视频流中主要运动的定性分析算法，实验结果表明该算法具有速度快，精度高，鲁棒性好等特性，非常适合于视频信息检索方面的应用。     

一种基于运动的视频层次化分析框架

运动信息是视频中最重要的特征之一.本文提出了一种新颖的基于运动的视频层次化分析框架.为了揭示视频流中的运动关系和在拍摄过程摄像机的运动信息即拍摄者的兴趣所在,提出了视频层次化结构来对视频流进行描述和检索.该运动层次化结构引入一个新的概念:基本运动单元(Basic Motion Unit,BMU).BMU反映的是MPEG-7中定义的基本摄像机操作,这样就可以把视频分割成BMU的集合.在BMU的基础上发现并挖掘出视频流中潜在的运动语义模式,最后根据运动活力(Motion Activity)将BMU聚类为运动节奏快慢的运动活力单元(Motion Activity Unit,MAU).该基于运动的视频分析框架不仅能够直观地捕获视频流在拍摄过程中拍摄者的注意力,而且为视频流的快速浏览和检索提供了新的思路.     

Compression of Human Motion Capture Data Using Motion Pattern Indexing

In this work, a novel scheme is proposed to compress human motion capture data based on hierarchical structure construction and motion pattern indexing. For a given sequence of 3D motion capture data of human body, the 3D markers are first organized into a hierarchy where each node corresponds to a meaningful part of the human body. Then, the motion sequence corresponding to each body part is coded separately. Based on the observation that there is a high degree of spatial and temporal correlation among the 3D marker positions, we strive to identify motion patterns that form a database for each meaningful body part. Thereafter, a sequence of motion capture data can be efficiently represented as a series of motion pattern indices. As a result, higher compression ratio has been achieved when compared with the prior art, especially for long sequences of motion capture data with repetitive motion styles. Another distinction of this work is that it provides means for flexible and intuitive global and local distortion controls.     

Generalized Model‐Based Human Motion Recognition with Body Partition Index Maps

Content‐based human motion analysis has captured extensive concerns of researchers from the domains of computer animation, human‐machine interaction, entertainment, etc. However, it is a non‐trivial task due to the spatial and temporal variations in the motion data. In this paper, we propose a generalized model (GM)‐based approach to model the variations and accurately recognize motion patterns. We partition the human character model into five parts, and extract the features of the submotions of each specific body part using clustering techniques. These features from the training trials in each class are combined to build the GM. We propose a new penalty based similarity measure for DTW to be used with the GMs for isolated motion recognition. On the other hand, from the GMs five body partition index maps are constructed and used for matching together with a flexible end point detection scheme during continuous motion recognition. In the experiments, we examine the effectiveness and efficiency of the approach in both isolated motion and continuous motion recognition. The results show that our proposed method has good performance compared with other state‐of‐the‐art methods in recognition accuracy and processing speed.     

Example-based automatic music-driven conventional dance motion synthesis

We introduce a novel method for synthesizing dance motions that follow the emotions and contents of a piece of music. Our method employs a learning-based approach to model the music to motion mapping relationship embodied in example dance motions along with those motions' accompanying background music. A key step in our method is to train a music to motion matching quality rating function through learning the music to motion mapping relationship exhibited in synchronized music and dance motion data, which were captured from professional human dance performance. To generate an optimal sequence of dance motion segments to match with a piece of music, we introduce a constraint-based dynamic programming procedure. This procedure considers both music to motion matching quality and visual smoothness of a resultant dance motion sequence. We also introduce a two-way evaluation strategy, coupled with a GPU-based implementation, through which we can execute the dynamic programming process in parallel, resulting in significant speedup. To evaluate the effectiveness of our method, we quantitatively compare the dance motions synthesized by our method with motion synthesis results by several peer methods using the motions captured from professional human dancers' performance as the gold standard. We also conducted several medium-scale user studies to explore how perceptually our dance motion synthesis method can outperform existing methods in synthesizing dance motions to match with a piece of music. These user studies produced very positive results on our music-driven dance motion synthesis experiments for several Asian dance genres, confirming the advantages of our method.     

基于注意力机制的3D DenseNet人体动作识别方法

传统人体动作识别算法无法充分利用视频中人体动作的时空信息,且识别准确率较低。提出一种新的三维密集卷积网络人体动作识别方法。将双流网络作为基本框架,在空间网络中运用添加注意力机制的三维密集网络提取视频中动作的表观信息特征,结合时间网络对连续视频序列运动光流的运动信息进行特征提取,经过时空特征和分类层的融合后得到最终的动作识别结果。同时为更准确地提取特征并对时空网络之间的相互作用进行建模,在双流网络之间加入跨流连接对时空网络进行卷积层的特征融合。在UCF101和HMDB51数据集上的实验结果表明,该模型识别准确率分别为94.52%和69.64%,能够充分利用视频中的时空信息,并提取运动的关键信息。     

基于特征向量匹配的舞蹈实时姿态分析方法

针对人体运动姿态可自由编辑的特性,提取人体骨架模型,建立人体运动姿态模型库,分析了运动捕捉系统在舞蹈训练中的应用,提出基于特征平面间相似性匹配的方法来计算模型各部件间运动数据参数。经过验证,该方法对于人体姿态的分析具有较高的准确性、鲁棒性,使得舞蹈演员能够准确的对比出与标准舞蹈动作的差异,为舞蹈者进行科学舞蹈训练提供了理论支持。     

Target detection algorithm for dance moving images based on sensor and motion capture data

The effectiveness of classification based on motion capture data to identify the human skeleton dance types. The goal is based on the body joint's information is to perform the characteristic posture obtained by the ultra-high Kinect sensor, identifying for each dance. The proposed Target Detection (TD) algorithm is used to dance moving object identification based on the improved Gaussian Mixture Model (GMM). The proposed Target Detection (TD) algorithm based on a Gaussian Mixture Model is a widely used method for modeling background from a Kinect sensor moving objects. The used data set contains six folk dance sequences and their variations. Gesture recognition scheme using a plurality of time constraints, spatial information, and spatial distribution characteristics to create a training data set appropriate application. The Gaussian Mixed Model distribution background model account, the algorithm, and their frame difference can be extracted in straight lines to obtain target dance areas with less background and background photo station under motor damage conditions. Through real-time Target Detection (TD) algorithm dance moving images based on the Gaussian Mixture Model (GMM), these two algorithms effectively detect dance moving image targets.     

Acquiring and validating motion qualities from live limb gestures

This paper presents a neural computing model that can automatically extract motion qualities from live performance. The motion qualities are in terms of laban movement analysis (LMA) Effort factors. The model inputs both 3D motion capture and 2D video projections. The output is a classification of motion qualities that are detected in the input. The neural nets are trained with professional LMA notators to ensure valid analysis and have achieved an accuracy of about 90% in motion quality recognition. The combination of this system with the EMOTE motion synthesis system provides a capability for automating both observation and analysis processes, to produce natural gestures for embodied communicative agents.     

State of the Art in Example-Based Motion Synthesis for Virtual Characters in Interactive Applications

Animated virtual human characters are a common feature in interactive graphical applications, such as computer and video games, online virtual worlds and simulations. Due to dynamic nature of such applications, character animation must be responsive and controllable in addition to looking as realistic and natural as possible. Though procedural and physics-based animation provide a great amount of control over motion, they still look too unnatural to be of use in all but a few specific scenarios, which is why interactive applications nowadays still rely mainly on recorded and hand-crafted motion clips. The challenge faced by animation system designers is to dynamically synthesize new, controllable motion by concatenating short motion segments into sequences of different actions or by parametrically blending clips that correspond to different variants of the same logical action. In this article, we provide an overview of research in the field of example-based motion synthesis for interactive applications. We present methods for automated creation of supporting data structures for motion synthesis and describe how they can be employed at run-time to generate motion that accurately accomplishes tasks specified by the AI or human user.     

Intelligent multimedia data: data + indices + inference

In this paper, we present a novel approach for multimedia data indexing and retrieval that is machine independent and highly flexible for sharing multimedia data across applications. Traditional multimedia data indexing and retrieval problems have been attacked using the central data server as the main focus, and most of the indexing and query-processing for retrieval are highly application dependent. This precludes the use of created indices and query processing mechanisms for multimedia data which, in general, have a wide variety of uses across applications. The approach proposed in this paper addresses three issues: 1. multimedia data indexing; 2. inference or query processing; and 3. combining indices and inference or query mechanism with the data to facilitate machine independence in retrieval and query processing. We emphasize the third issue, as typically multimedia data are huge in size and requires intra-data indexing. We describe how the proposed approach addresses various problems faced by the application developers in indexing and retrieval of multimedia data. Finally, we present two applications developed based on the proposed approach: video indexing; and video content authorization for presentation.     

基于三维特征提取的人体运动数据分析和检索

基于各关节点三维空间和时间关系的三维特征,提出了一种描述各关节点之间三维空间关系的时空特征,可以分别处理每一个关节点的局部时空特征。同时,三维时空特征避开与原始数据的直接接触,从而很大程度避免了维数灾难。实验结果显示,三维特征提高运动数据检索的效率和精度,可应用到运动语义智能分析等领域。     

Indexing spatiotemporal archives

Spatiotemporal objects – that is, objects that evolve over time – appear in many applications. Due to the nature of such applications, storing the evolution of objects through time in order to answer historical queries (queries that refer to past states of the evolution) requires a very large specialized database, what is termed in this article a spatiotemporal archive. Efficient processing of historical queries on spatiotemporal archives requires equally sophisticated indexing schemes. Typical spatiotemporal indexing techniques represent the objects using minimum bounding regions (MBR) extended with a temporal dimension, which are then indexed using traditional multidimensional index structures. However, rough MBR approximations introduce excessive overlap between index nodes, which deteriorates query performance. This article introduces a robust indexing scheme for answering spatiotemporal queries more efficiently. A number of algorithms and heuristics are elaborated that can be used to preprocess a spatiotemporal archive in order to produce finer object approximations, which, in combination with a multiversion index structure, will greatly improve query performance in comparison to the straightforward approaches. The proposed techniques introduce a query efficiency vs. space tradeoff that can help tune a structure according to available resources. Empirical observations for estimating the necessary amount of additional storage space required for improving query performance by a given factor are also provided. Moreover, heuristics for applying the proposed ideas in an online setting are discussed. Finally, a thorough experimental evaluation is conducted to show the merits of the proposed techniques. Edited by B. Seeger A short version of this article appeared as “Efficient indexing of spatiotemporal objects” in the Proceedings of Extending Database Technology 2002 [19]. This work was partially supported by NSF grants IIS-9907477, EIA-9983445, NSF IIS 9984729, NSF ITR 0220148, NSF IIS-0133825, NRDRP, and the U.S. Department of Defense.     

融合运动特征和深度学习的跌倒检测算法

为了利用计算机视觉技术准确检测老年人的跌倒状况,针对现有跌倒检测算法中人为设计特征造成的不完备性以及跌倒检测过程中前后景分离困难、目标混淆、运动目标丢失、跌倒检测准确率低等问题,提出了一种融合人体运动信息的深度学习跌倒检测算法对人体跌倒状态进行检测。首先,通过改进YOLOv3网络进行前景与背景的分离,并根据YOLOv3网络的检测结果对前景人体目标进行最小外接矩形标记;其次,分析人体跌倒过程中的运动特征,将人体运动特征向量化并通过Sigmoid激活函数转化为0到1之间的运动权重信息;最后,通过全连接层将将运动特征与卷积神经网络（CNN）提取的特征进行拼接和融合从而实现人体跌倒分类判别。将所提跌倒检测算法与背景差分、高斯混合、VIBE、方向梯度直方图（HOG）等人体目标检测算法及阈值法、分级法、支持向量机（SVM）分类和CNN分类等人体跌倒判断方案进行了对比实验,并将所提跌倒检测算法在不同光照条件下和混合日常噪声运动干扰下进行了实验,结果表明所提算法在环境适应性和跌倒检测准确率上都优于传统的人体跌倒检测方法。该算法能有效检测出视频中的人体并对人体跌倒状态进行准确检测,进一步验证了融合运动信息的深度学习识别方法在视频跌倒行为分析上的可行性与高效性。