局部特征与全局特征结合的HMM静态手势识别

针对静态手势识别问题,提出了一种综合考虑局部形状与全局轮廓的隐马尔科夫模型(HMM)静态手势识别算法。该算法提取局部形状熵特征与上层轮廓特征分别作为训练数据训练每类手势的HMM参数。测试时,先凭借局部形状熵特征得出初步识别结果,然后根据初步识别结果的模糊性,附加与局部特征互补的上层轮廓特征进行再识别,得出最终识别结果。实验结果表明,该算法对于形状差异占主导地位的手势库有很好的效果,并且将静态手势的空间序列模拟成时间序列使得静态手势识别具有空间尺度不变性;同时该算法合理控制特征维数,一定程度上弱化了HMM训练时间长的弊端,加快了识别的速度。     

Maximum mutual information training for an online neural predictive handwritten word recognition system

In this paper, we present a hybrid online handwriting recognition system based on hidden Markov models (HMMs). It is devoted to word recognition using large vocabularies. An adaptive segmentation of words into letters is integrated with recognition, and is at the heart of the training phase. A word-model is a left-right HMM in which each state is a predictive multilayer perceptron that performs local regression on the drawing (i.e., the written word) relying on a context of observations. A discriminative training paradigm related to maximum mutual information is used, and its potential is shown on a database of 9,781 words. Received June 19, 2000 / Revised October 16, 2000     

Combining multi-visual features for efficient indexing in a large image database

The optimized distance-based access methods currently available for multidimensional indexing in multimedia databases have been developed based on two major assumptions: a suitable distance function is known a priori and the dimensionality of the image features is low. It is not trivial to define a distance function that best mimics human visual perception regarding image similarity measurements. Reducing high-dimensional features in images using the popular principle component analysis (PCA) might not always be possible due to the non-linear correlations that may be present in the feature vectors. We propose in this paper a fast and robust hybrid method for non-linear dimensions reduction of composite image features for indexing in large image database. This method incorporates both the PCA and non-linear neural network techniques to reduce the dimensions of feature vectors so that an optimized access method can be applied. To incorporate human visual perception into our system, we also conducted experiments that involved a number of subjects classifying images into different classes for neural network training. We demonstrate that not only can our neural network system reduce the dimensions of the feature vectors, but that the reduced dimensional feature vectors can also be mapped to an optimized access method for fast and accurate indexing. Received 11 June 1998 / Accepted 25 July 2000 Published online: 13 February 2001     

基于Leap Motion的动态手势识别方法

伴随虚拟现实（Virtual Reality,VR）技术的发展,以及人们对人机交互性能和体验感的要求提高,手势识别作为影响虚拟现实中交互操作的重要技术之一,其精确度急需提升^[1].针对当前手势识别方法在一些动作类似的手势识别中表现欠佳的问题,提出了一种多特征动态手势识别方法.该方法首先使用体感控制器Leap Motion追踪动态手势获取数据,然后在特征提取过程中增加对位移向量角度和拐点判定计数的提取,接着进行动态手势隐马尔科夫模型（Hidden Markov Model,HMM）的训练,最后根据待测手势与模型的匹配率进行识别.从实验结果中得出,该多特征识别方法能够提升相似手势的识别率.     

基于拉普拉斯脸和隐马尔可夫的视频人脸识别

提出了一种基于拉普拉斯脸和隐马尔可夫模型的视频人脸识别方法。在训练过程中，采用拉普拉斯脸方法将每一视频序列中的人脸图像映射到拉普拉斯空间，将降维后的特征作为观测值，通过隐马尔可夫模型得到每一训练视频的统计特性和时间动态特性。在识别过程中，用每一个训练视频的隐马尔可夫模型来分析测试视频的时间动态特性，计算出每一训练模型产生该序列的概率，概率最大值所对应的模型就是待识别序列所属的类别。实验结果表明，该方法能够很好地进行视频人脸识别。     

The reconstruction of dynamic 3D structure of biological objects using stereo microscope images

In this paper, we address the analysis of 3D shape and shape change in non-rigid biological objects imaged via a stereo light microscope. We propose an integrated approach for the reconstruction of 3D structure and the motion analysis for images in which only a few informative features are available. The key components of this framework are: 1) image registration using a correlation-based approach, 2) region-of-interest extraction using motion-based segmentation, and 3) stereo and motion analysis using a cooperative spatial and temporal matching process. We describe these three stages of processing and illustrate the efficacy of the proposed approach using real images of a live frog's ventricle. The reconstructed dynamic 3D structure of the ventricle is demonstrated in our experimental results, and it agrees qualitatively with the observed images of the ventricle.     

Temporal and spatial data mining with second-order hidden markov models

In the frame of designing a knowledge discovery system, we have developed stochastic models based on high-order hidden Markov models. These models are capable to map sequences of data into a Markov chain in which the transitions between the states depend on the n previous states according to the order of the model. We study the process of achieving information extraction from spatial and temporal data by means of an unsupervised classification. We use therefore a French national database related to the land use of a region, named Ter Uti, which describes the land use both in the spatial and temporal domain. Land-use categories (wheat, corn, forest, ...) are logged every year on each site regularly spaced in the region. They constitute a temporal sequence of images in which we look for spatial and temporal dependencies. The temporal segmentation of the data is done by means of a second-order Hidden Markov Model (HMM2) that appears to have very good capabilities to locate stationary segments, as shown in our previous work in speech recognition. The spatial classification is performed by defining a fractal scanning of the images with the help of a Hilbert–Peano curve that introduces a total order on the sites, preserving the relation of neighborhood between the sites. We show that the HMM2 performs a classification that is meaningful for the agronomists. Spatial and temporal classification may be achieved simultaneously by means of a two levels HMM2 that measures the a posteriori probability to map a temporal sequence of images onto a set of hidden classes.     

Extracting characters of license plates from video sequences

In this paper, we present a new approach to extract characters on a license plate of a moving vehicle, given a sequence of perspective-distortion-corrected license plate images. Different from many existing single-frame approaches, our method simultaneously utilizes spatial and temporal information. We first model the extraction of characters as a Markov random field (MRF), where the randomness is used to describe the uncertainty in pixel label assignment. With the MRF modeling, the extraction of characters is formulated as the problem of maximizing a posteriori probability based on a given prior knowledge and observations. A genetic algorithm with local greedy mutation operator is employed to optimize the objective function. Experiments and comparison study were conducted and some of our experimental results are presented in the paper. It is shown that our approach provides better performance than other single frame methods. Received: 13 August 1997 / Accepted: 7 October 1997     

Independent shape component-based human activity recognition via Hidden Markov Model

In proactive computing, human activity recognition from image sequences is an active research area. In this paper, a novel human activity recognition method is proposed, which utilizes Independent Component Analysis (ICA) for activity shape information extraction from image sequences and Hidden Markov Model (HMM) for recognition. Various human activities are represented by shape feature vectors from the sequence of activity shape images via ICA. Based on these features, each HMM is trained and activity recognition is achieved by the trained HMMs of different activities. Our recognition performance has been compared to the conventional method where Principal Component Analysis (PCA) is typically used to derive activity shape features. Our results show that superior recognition is achieved with the proposed method especially for activities (e.g., skipping) that cannot be easily recognized by the conventional method. Furthermore, by employing Linear Discriminant Analysis (LDA) on IC features, the recognition results further improved significantly in the recognition performance.     

人脸识别中PCA方法的推广

主成分分析(PrincipalComponentAnalysis,PCA)是公认的特征抽取的最为重要的工具之一,目前仍然被广泛地应用在人脸等图像识别领域。基于PCA,该文提出了分块PCA的人脸识别方法。分块PCA方法先对图像矩阵进行分块,对分块得到的子图像矩阵利用PCA进行鉴别分析。其特点是能有效地抽取图像的局部特征,对人脸表情和光照条件变化较大的图像表现尤为突出。与PCA方法相比,由于使用子图像矩阵,分块PCA可以避免使用奇异值分解理论,过程简便。此外,PCA是分块PCA的特殊情况。在Yale和NUST603人脸库上的试验结果表明,所提出的方法在识别性能上明显优于经典的PCA方法,识别率可以分别提高6.7和4个百分点。     

多点触控的沙画手势识别

针对多点触控手势间接指令问题,提出了基于多点触控的沙画手势识别系统,该识别系统由时间、空间、形状信息控制。提出一种手势图形建模方法,测量手势的笔划之间的空间和时间关系。采用聚类算法标记手势图形中笔划的形状信息作为局部形状特征;利用基准方法HBF49特征提取全局形状特征。通过一组有10种不同多点触控的沙画手势的数据集评估基于多点触控的沙画手势识别系统,使用图嵌入方法和SVM分类进行手势识别,识别的准确率达到94.75%。实验结果证明,此研究对完成基于多点触控的沙画虚拟系统有重要作用。     

Parametric hidden Markov models for gesture recognition

A method for the representation, recognition, and interpretation of parameterized gesture is presented. By parameterized gesture we mean gestures that exhibit a systematic spatial variation; one example is a point gesture where the relevant parameter is the two-dimensional direction. Our approach is to extend the standard hidden Markov model method of gesture recognition by including a global parametric variation in the output probabilities of the HMM states. Using a linear model of dependence, we formulate an expectation-maximization (EM) method for training the parametric HMM. During testing, a similar EM algorithm simultaneously maximizes the output likelihood of the PHMM for the given sequence and estimates the quantifying parameters. Using visually derived and directly measured three-dimensional hand position measurements as input, we present results that demonstrate the recognition superiority of the PHMM over standard HMM techniques, as well as greater robustness in parameter estimation with respect to noise in the input features. Finally, we extend the PHMM to handle arbitrary smooth (nonlinear) dependencies. The nonlinear formulation requires the use of a generalized expectation-maximization (GEM) algorithm for both training and the simultaneous recognition of the gesture and estimation of the value of the parameter. We present results on a pointing gesture, where the nonlinear approach permits the natural spherical coordinate parameterization of pointing direction     

基于3D卷积神经网络的动态手势识别

传统的2D卷积神经网络在进行视频识别时容易丢失目标在时间维度上的相关特征信息,导致识别准确率降低。针对该问题,本文采用3D卷积网络作为基本的网络框架,使用3D卷积核进行卷积操作提取视频中的时空特征,同时集成多个3D卷积神经网络模型对动态手势进行识别。为了提高模型的收敛速度和训练的稳定性,运用批量归一化（BN）技术优化网络,使优化后的网络训练时间缩短。实验结果表明,本文方法对于动态手势的识别具有较好的识别结果,在Sheffield Kinect Gesture (SKIG)数据集上识别准确率达到98.06%。与单独使用RGB信息、深度信息以及传统2D CNN相比,手势识别率均有所提高,验证了本文方法的可行性和有效性。     

基于手势特征融合的操作动作识别

针对动态复杂场景下的操作动作识别,提出一种基于手势特征融合的动作识别框架,该框架主要包含RGB视频特征提取模块、手势特征提取模块与动作分类模块。其中RGB视频特征提取模块主要使用I3D网络提取RGB视频的时间和空间特征;手势特征提取模块利用Mask R-CNN网络提取操作者手势特征;动作分类模块融合上述特征,并输入到分类器中进行分类。在EPIC-Kitchens数据集上,提出的方法识别抓取手势的准确性高达89.63%,识别综合动作的准确度达到了74.67%。     

基于ICA与HMM的表情识别

独立分量分析(independent component analysis,ICA)是一种盲源分离的有效方法,为了进一步有效提取表情图像中隐藏的信息和提高表情识别率,可将它应用于人脸表情识别。由于脸部表情为人类情感、认知过程的研究提供了极为重要的测量依据,因此表情特征的提取和特征序列所代表的表情状态是表情识别过程中的重要步骤。为了更好地进行表情和情感的分类,提出了一种ICA结合隐马尔可夫模型(HMM)识别表情的情感分类系统,该系统首先利用ICA算法进行表情特征提取,为了加快特征提取的速度,这里采用了FastICA算法;然后通过7个训练好的HMM进行表情识别。实验结果显示,该系统使人脸表情识别的整体效果有了提高,取得了令人满意的效果,可以用来识别人脸表情。     

表情和姿态的双模态情感识别

多模态情感识别是当前情感计算研究领域的重要内容,针对人脸表情和动作姿态开展双模态情感识别研究,提出一种基于双边稀疏偏最小二乘的表情和姿态的双模态情感识别方法.首先,从视频图像系列中分别提取表情和姿态两种模态的空时特征作为情感特征矢量.然后,通过双边稀疏偏最小二乘(BSPLS)的数据降维方法来进一步提取两组模态中的情感特征,并组合成新的情感特征向量.最后,采用了两种分类器来进行情感的分类识别.以国际上广泛采用的FABO表情和姿态的双模态情感数据库为实验数据,并与多种子空间方法(主成分分析、典型相关分析、偏最小二乘回归)进行对比实验来评估本文方法的识别性能.实验结果表明,两种模态融合后相比单模态更加有效,双边稀疏偏最小二乘(BSPLS)算法在几种方法中得到最高的情感识别率.     

A probabilistic model for state sequence analysis in hidden Markov model for hand gesture recognition

The role of gesture recognition is significant in areas like human‐computer interaction, sign language, virtual reality, machine vision, etc. Among various gestures of the human body, hand gestures play a major role to communicate nonverbally with the computer. As the hand gesture is a continuous pattern with respect to time, the hidden Markov model (HMM) is found to be the most suitable pattern recognition tool, which can be modeled using the hand gesture parameters. The HMM considers the speeded up robust feature features of hand gesture and uses them to train and test the system. Conventionally, the Viterbi algorithm has been used for training process in HMM by discovering the shortest decoded path in the state diagram. The recursiveness of the Viterbi algorithm leads to computational complexity during the execution process. In order to reduce the complexity, the state sequence analysis approach is proposed for training the hand gesture model, which provides a better recognition rate and accuracy than that of the Viterbi algorithm. The performance of the proposed approach is explored in the context of pattern recognition with the Cambridge hand gesture data set.     

基于HMM-FNN模型的复杂动态手势识别

复杂动态手势识别是利用视频手势进行人机交互的关键问题.提出一种HMM-FNN模型结构.它整合了隐马尔可夫模型对时序数据的建模能力与模糊神经网络的模糊规则构建与推理能力,并将其应用到复杂动态手势的识别中.复杂动态手势具备两大特点:运动特征的可分解性与定义描述的模糊性.针对这两种特性,复杂手势被分解为手形变化、2D平面运动与Z轴方向运动3个子部分,分别利用HMM进行建模,HMM模型对观察子序列的似然概率被作为FNN的模糊隶属度,通过模糊规则推理,最终得到手势的分类类别.HMM-FNN方法将高维手势特征分解为低维子特征序列,降低了模型的复杂度.此外,它还可以充分利用人的经验辅助模型结构的创建与优化.实验表明,该方法是一种有效的复杂动态手势识别方法,并且优于传统的HMM模型方法.