Stereovision depth analysis by two-dimensional motion charge memories

Several strategies to retrieve depth information from a sequence of images have been described so far. In this paper a method that turns around the existing symbiosis between stereovision and motion is introduced; motion minimizes correspondence ambiguities, and stereovision enhances motion information. The central idea behind our approach is to transpose the spatially defined problem of disparity estimation into the spatial–temporal domain. Motion is analyzed in the original sequences by means of the so-called permanency effect and the disparities are calculated from the resulting two-dimensional motion charge maps. This is an important contribution to the traditional stereovision depth analysis, where disparity is got from the image luminescence. In our approach, disparity is studied from a motion-based persistency charge measure.     

基于卷积神经网络的Leap Motion运动数据优化网络

为提高Leap Motion设备的采集精准度,解决自遮挡、采样频率不稳定等设备固有问题,首先,设计了使用Leap Motion和动作捕捉设备的手部多模态同步运动采集方案,采集了日常动作数据集;其次,提出了基于卷积神经网络(convolutional neural network,CNN)的Leap Motion手部运动...     

Spatio-temporal structure of human motion primitives and its application to motion prediction

This paper proposes a novel approach to structuring behavioral knowledge based on symbolization of human whole body motions, hierarchical classification of the motions, and extraction of the causality among the motions. The motion patterns are encoded into parameters of corresponding Hidden Markov Models (HMMs), where each HMM abstracts the dynamics of motion pattern, and hereafter is referred to as “motion symbol”. The motion symbols allow motion recognition and synthesis. The motion symbols are organized into a hierarchical tree structure representing the property of spatial similarity among the motion patterns, and this tree is referred to as “motion symbol tree”. Seamless motion is segmented into a sequence of motion primitives, each of which is classified as a motion symbol based on the motion symbol tree. The seamless motion results in a sequence of the motion symbols, which is stochastically represented as transitions between the motion symbols by an N-gram model. The motion symbol N-gram model is referred to as “motion symbol graph”. The motion symbol graph extracts the temporal causality among the human behaviors. The integration of the motion symbol tree and the motion symbol graph makes it possible to recognize motion patterns fast and predict human behavior during observation. The experiments on a motion dataset of radio calisthenics and on a large motion dataset provided by CMU motion database validate the proposed framework.     

Human motion database with a binary tree and node transition graphs

Database of human motion has been widely used for recognizing human motion and synthesizing humanoid motions. In this paper, we propose a data structure for storing and extracting human motion data and demonstrate that the database can be applied to the recognition and motion synthesis problems in robotics. We develop an efficient method for building a human motion database from a collection of continuous, multi-dimensional motion clips. The database consists of a binary tree representing the hierarchical clustering of the states observed in the motion clips, as well as node transition graphs representing the possible transitions among the nodes in the binary tree. Using databases constructed from real human motion data, we demonstrate that the proposed data structure can be used for human motion recognition, state estimation and prediction, and robot motion planning.     

Cross-view action matching using a novel projective invariant on non-coplanar space-time points

Existing action matching methods from the geometric respect typically assume the collinearity or coplanarity for view invariance. These assumptions curb the application to uncontrolled action patterns. In this paper, a new projective invariant named characteristic number (CN) is used, which can be used to describe 3D non-coplanar points. For motion trajectories of actions, we propose the temporal CN (TCN) for individual joint point of a human body in temporal series. This view-invariant feature can characterize an action well with limited number of joints(a single one in our experiments). In addition to TCN, we are also able to define the spatial characteristic number (SCN) on several (five in our paper) joint points in the spatial domain for one frame. SCN works complementary to temporal features, when limited snapshots of an action are available. We validate both SCN and TCN on the widely used CMU Motion Capture Database (Mocap) database, KTH Multiview Football Dataset II and IXMAS dataset. The promising recognition results indicate the invariance to varying viewpoints compared with the state-of-the-art. The results on CMU and KTH database corrupted by noise show the robustness to noise.     

融合时空图卷积网络与非自回归模型的三维人体运动预测

当前人体运动预测的方法大多采用基于图卷积网络的自回归模型,没有充分考虑关节间的特有关系和自回归网络性能的限制,从而产生平均姿态和误差累积等问题。为解决以上问题,提出融合时空图卷积网络和非自回归的模型对人体运动进行预测。一方面利用时空图卷积的网络提取人体运动序列的局部特征,可以有效减少三维人体运动预测场景中的平均姿态问题和过度堆叠图卷积层引起的过平滑问题的发生;另一方面将非自回归模型与时空图卷积网络进行结合,减少误差累计问题的发生。利用Human3.6M的数据集进行80 ms、160 ms、320 ms和400 ms的人体运动预测实验。结果表明,NAS-GCN模型与现有方法相比,能预测出更精确的结果。     

Extracting representative motion flows for effective video retrieval

In this paper, we propose a novel motion-based video retrieval approach to find desired videos from video databases through trajectory matching. The main component of our approach is to extract representative motion features from the video, which could be broken down to the following three steps. First, we extract the motion vectors from each frame of videos and utilize Harris corner points to compensate the effect of the camera motion. Second, we find interesting motion flows from frames using sliding window mechanism and a clustering algorithm. Third, we merge the generated motion flows and select representative ones to capture the motion features of videos. Furthermore, we design a symbolic based trajectory matching method for effective video retrieval. The experimental results show that our algorithm is capable to effectively extract motion flows with high accuracy and outperforms existing approaches for video retrieval.     

时空域融合的骨架动作识别与交互研究

在人体骨架结构动作识别方法中,很多研究工作在提取骨架结构上的空间信息和运动信息后进行融合,没有对具有复杂时空关系的人体动作进行高效表达。本文提出了基于姿态运动时空域融合的图卷积网络模型(PM-STFGCN)。对于在时域上存在大量的干扰信息,定义了一种基于局部姿态运动的时域关注度模块(LPM-TAM),用于抑制时域上的干扰并学习运动姿态的表征。设计了基于姿态运动的时空域融合模块(PM-STF),融合时域运动和空域姿态特征并进行自适应特征增强。通过实验验证,本文提出的方法是有效性的,与其他方法相比,在识别效果上具有很好的竞争力。设计的人体动作交互系统,验证了在实时性和准确率上优于语音交互系统。     

基于上肢力学信息的人体运动意图检测

运动意图检测在人机交互中具有重要作用,是人机交互自然性和可靠性的保障.上肢力学信息与上肢运动意图有某种潜在的关系,探索人体在外界约束条件下上肢力学信息产生和变化规律,通过开发运动意图信息采集系统并实验,可实现基于上肢力学信息的运动意图检测.通过多维力传感器检测得到的人体上肢力信息,并与人的运动意图相结合,找出上肢动作、运动意图、上肢力信息三者之间的关系,探索出一种基于上肢力学信息的人体运动意图检测方法.     

一个基于运动的视频检索系统

给出了一种基于运动在空间 时间域上体现的运动特征的设计方案 ,主要涉及两个模块 ,即 :运动检测模块 (Detec tionModule)和任何检索系统都必需的匹配模块 (MatchingModule)。初步试验表明 ,检测的结果基本和我们的直觉相符合。     

3D human pose estimation in motion based on multi-stage regression

3D human pose estimation in motion is a hot research direction in the field of computer vision. However, the performance of the algorithm is affected by the complexity of 3D spatial information, self-occlusion of human body, mapping uncertainty and other problems. In this paper, we propose a 3D human joint localization method based on multi-stage regression depth network and 2D to 3D point mapping algorithm. First of all, we use a single RGB image as the input, through the introduction of heatmap and multi-stage regression to constantly optimize the coordinates of human joint points. Then we input the 2D joint points into the mapping network for calculation, and get the coordinates of 3D human body joint points, and then to complete the 3D human body pose estimation task. The MPJPE of the algorithm in Human3.6 M dataset is 40.7. The evaluation of dataset shows that our method has obvious advantages.     

面向编钟乐舞的舞蹈动作虚拟仿真的研究

本文简述了面向编钟乐舞的虚拟角色的动作库制作过程.文章着重分析了舞蹈文法下的动作归类及其对应关系.探讨了基于情感模型下的动作风格鉴定理论,提出了通过对特征动作单元属性和关联约束的定义来实现动作衔接的方法,并给出该方法在动作编排系统中的应用框架.     

数据库系统课程的教学内容和方法探讨

针对大同大学数据库原理教学中存在的理论和实践脱节的问题,提出了以数据库系统设计为主线的教学方法和内容的改进,引进了以小组课题为主的CDIO教学模式,旨在提高学生分析问题和解决问题的能力,使学生通过本课程的学习,能够设计出规范的数据库。     

Human motion generation with multifactor models

To generate human motions with various specific attributes is a difficult task because of high dimensionality and complexity of human motions. This paper presents a novel human motion model for generating and editing motions with multiple factors. A set of motions performed by several actors with various styles was captured for constructing a well‐structured motion database. Subsequently, MICA (multilinear independent component analysis) model that combines ICA and conventional multilinear framework was adopted for the construction of a multifactor model. With this model, new motions can be synthesized by interpolation and through solving optimization problems for the specific factors. Our method offers a practical solution to edit stylistic human motions in a parametric space learnt with MICA model. We demonstrated the power of our method by generating and editing sideways stepping, reaching, and striding over obstructions using different actors with various styles. The experimental results show that our method can be used for interactive stylistic motion synthesis and editing. Copyright © 2011 John Wiley & Sons, Ltd.     

基于计算机视觉的人运动捕获综述

人体运动的捕获是计算机视觉领域的热点之一,它是指从大范围上从图像序列中提取并描述人体轮廓的运动,然后对其进行跟踪识别。介绍了运动捕获的潜在应用,从初始化、检测、跟踪、姿态评估、识别5个方面分析了有关运动捕获的分类及研究现状,最后简要探讨了该领域面临的难点问题及发展趋势。     

A database-based framework for gesture recognition

Gestures are an important modality for human–machine communication. Computer vision modules performing gesture recognition can be important components of intelligent homes, assistive environments, and human–computer interfaces. A key problem in recognizing gestures is that the appearance of a gesture can vary widely depending on variables such as the person performing the gesture, or the position and orientation of the camera. This paper presents a database-based approach for addressing this problem. The large variability in appearance among different examples of the same gesture is addressed by creating large gesture databases, that store enough exemplars from each gesture to capture the variability within that gesture. This database-based approach is applied to two gesture recognition problems: handshape categorization and motion-based recognition of American Sign Language signs. A key aspect of our approach is the use of database indexing methods, in order to address the challenge of searching large databases without violating the time constraints of an online interactive system, where system response times of over a few seconds are oftentimes considered unacceptable. Our experiments demonstrate the benefits of the proposed database-based framework, and the feasibility of integrating large gesture databases into online interacting systems.     

Multiscale motion saliency for keyframe extraction from motion capture sequences

Motion capture is an increasingly popular animation technique; however data acquired by motion capture can become substantial. This makes it difficult to use motion capture data in a number of applications, such as motion editing, motion understanding, automatic motion summarization, motion thumbnail generation, or motion database search and retrieval. To overcome this limitation, we propose an automatic approach to extract keyframes from a motion capture sequence. We treat the input sequence as motion curves, and obtain the most salient parts of these curves using a new proposed metric, called 'motion saliency'. We select the curves to be analysed by a dimension reduction technique, Principal Component Analysis (PCA). We then apply frame reduction techniques to extract the most important frames as keyframes of the motion. With this approach, around 8% of the frames are selected to be keyframes for motion capture sequences. Copyright © 2010 John Wiley & Sons, Ltd.     

基于符号图的高维时间序列检索

提出一种基于符号图的高维时间序列数据库索引方法,通过等分空间法对高维时间序列数据进行预处理,将其转化为高维符号序列,利用符号状态转换图对数据库进行索引。应用CMU Graphics lab发布的Motion Capture Database动作数据库进行实验,取得了较好的结果。