Video-based signer-independent Arabic sign language recognition using hidden Markov models

Sign language in Arab World has been recently recognized and documented. There have been no serious attempts to develop a recognition system that can be used as a communication means between hearing-impaired and other people. This paper introduces the first automatic Arabic sign language (ArSL) recognition system based on hidden Markov models (HMMs). A large set of samples has been used to recognize 30 isolated words from the Standard Arabic sign language. The system operates in different modes including offline, online, signer-dependent, and signer-independent modes. Experimental results on using real ArSL data collected from deaf people demonstrate that the proposed system has high recognition rate for all modes. For signer-dependent case, the system obtains a word recognition rate of 98.13%, 96.74%, and 93.8%, on the training data in offline mode, on the test data in offline mode, and on the test data in online mode respectively. On the other hand, for signer-independent case the system obtains a word recognition rate of 94.2% and 90.6% for offline and online modes respectively. The system does not rely on the use of data gloves or other means as input devices, and it allows the deaf signers to perform gestures freely and naturally.     

基于二阶隐马尔可夫模型的连续手语识别

在传统的一阶隐马尔可夫模型(HMM1)中,状态序列中的每一个状态被假设只与前一个状态有关,这样虽然可以简单、有效地推导出模型的学习和识别算法,但也丢失了许多从上文传递下来的信息.因此,在传统一阶隐马尔可夫模型的基础上,为了解决手语识别困难、正确率低的问题,提出了一种基于二阶隐马尔可夫模型(HMM2)的连续手语识别方法....     

A cloud-based monitoring system via face recognition using Gabor and CS-LBP features

Face detection and recognition is an important topic in security. Currently, ubiquitous monitoring has received a large amount of attention. This paper proposes a cloud-based ubiquitous monitoring system via face recognition. It consists of a monitoring client module for face detection and recognition and a cloud storage module for data visualization. In the monitoring client module, the center-symmetric local Gabor binary pattern feature extraction method is proposed for face recognition, which combines improved multi-scale Gabor and center-symmetric local binary pattern (CS-LBP) features. This method maintains crucial local features, reduces the Gabor filter complexity, and adds rotational invariance and more precise texture information. A large number of experiments on the ORL, Yale-B, and Yale databases show that the proposed method obtains significantly better recognition rates than the LBP, CS-LBP, and Scale Gabor methods. Furthermore, we propose a Web browser-based data visualization that renders the geographic locations of the face detection and recognition results.     

基于小波分析和几何特征的人脸识别方法研究

在主成分分析法(PCA)和独立成分分析法(ICA)等理论基础上,提出一种结合人脸几何特征和独立Gabor小波特征分析的人脸识别方法.在对人脸图像进行二维小波分解的基础上,从人脸图像的下采样Gabor小波图像中得到一个Gabor小波特征向量并利用PCA法降维,在ICA法的基础上得到独立Gabor小波特征,并结合人脸面部器官的位置和轮廓及器官距离等所构成的几何特征进行人脸识别.     

A Hidden Markov Model approach for appearance-based 3D object recognition

In this paper, a new appearance-based 3D object classification method is proposed based on the Hidden Markov Model (HMM) approach. Hidden Markov Models are a widely used methodology for sequential data modelling, of growing importance in the last years. In the proposed approach, each view is subdivided in regular, partially overlapped sub-images, and wavelet coefficients are computed for each window. These coefficients are then arranged in a sequential fashion to compose a sequence vector, which is used to train a HMM, paying particular attention to the model selection issue and to the training procedure initialization. A thorough experimental evaluation on a standard database has shown promising results, also in presence of image distortions and occlusions, the latter representing one of the most severe problems of the recognition methods. This analysis suggests that the proposed approach represents an interesting alternative to classic appearance-based methods to 3D object classification.     

Neutralizing lighting non-homogeneity and background size in PCNN image signature for Arabic Sign Language recognition

Many feature generation methods have been developed using pulse-coupled neural network. Most of these methods succeeded to achieve the invariance against object translation, rotation and scaling but could not neutralize the bright background effect and non-uniform light on the quality of the generated features. To overcome the shortcomings, the paper proposes a new method to enhance the features’ quality. The “Continuity Factor” is defined and considered as a weight factor of the current pulse in signature generation process. This factor measures the simultaneous firing strength for connected pixels. The proposed new method is applied and compared to the previous methods. Through Arabic Sign Language recognition experiments, the superiority of the new method is shown.

     

A face detection and recognition system in color image series

A human face detection and recognition system for color image series is presented in this paper. The system is composed of two subsystems: human face detection subsystem and human face recognition subsystem. The face detection subsystem includes two modules: face finding and face verification. The human face finding module determines the face regions of a number of subjects from color image series using skin color analysis and motion analysis. The human face verification module is developed to verify the detected human faces by judging of eclipse and support vector machine (SVM), and precisely localize human faces by locating eyes and mouths based on Generalized Symmetry Transform. The features characterizing the relation between face patterns can be extracted and selected by Principal Component Analysis. Using these selected features to train multiple SVMs, we can finally classify human faces. Moreover, in these modules, several simple and complex methods are used to reduce the searching space. So the system can work at a high speed and high detection and recognition rate. Human face detection accuracy of the system is 97.2% under controllable lightning condition. Human face recognition accuracy of the system for 70 persons is 96.5% (with 20 eigenvectors) and 98.3% (with 30 eigenvectors).     

HMM在自然语言处理领域中的应用研究

隐马尔可夫模型（HMM）是一种强大的统计学机器学习技术,该模型已经成功地应用于连续语音识别、在线手写识别,在生物学信息中也得到了广泛的应用。由于该模型的强大的学习能力,在自然语言处理领域逐渐得到了应用。对隐马尔可夫模型在词性标注、命名实体识别、信息抽取应用中的关键问题进行了分析。着重分析了在信息抽取时使用隐马尔可夫模型的重点和难点问题,期望让更多的研究人员进一步认识和了解HMM。最后分析了隐马尔可夫模型在应用中的不足之处和改进研究。     

Hidden Markov modeling using a dominant state sequence with application to speech recognition

Approximate maximum likelihood (ML) hidden Markov modeling using the most likely state sequence (MLSS) is examined and compared with the exact ML approach that considers all possible state sequences. It is shown that for any hidden Markov model (HMM), the difference between the approximate and the exact normalized likelihood functions cannot exceed the logarithm of the number of states divided by the dimension of the output vectors (frame length), which is negligible for typically used values of vector dimension (128–256) and number of states (2–30). Furthermore, for Gaussian HMMs and a given observation sequence, the MLSS is typically the sequence of nearest neighbor states in the Itakura-Saito sense, and the posterior probability of any state sequence which departs from the MLSS in a single time instant decays exponentially with the frame length. Hence, for a sufficiently large frame length the exact and approximate ML approaches provide similar model estimates and likelihood values. The results and their implications on speech recognition are demonstrated in a set of experiments.     

HMM在自然语言处理领域中的应用研究

隐马尔可夫模型（HMM）是一种强大的统计学机器学习技术,该模型已经成功地应用于连续语音识别、在线手写识别,在生物学信息中也得到了广泛的应用。由于该模型的强大的学习能力,在自然语言处理领域逐渐得到了应用。对隐马尔可夫模型在词性标注、命名实体识别、信息抽取应用中的关键问题进行了分析。着重分析了在信息抽取时使用隐马尔可夫模型的重点和难点问题,期望让更多的研究人员进一步认识和了解HMM。最后分析了隐马尔可夫模型在应用中的不足之处和改进研究。     

基于HMM-SVM的图像型火焰识别

针对单一的隐马尔科夫模型在图像型火灾探测中误报率偏高的问题,提出了隐马尔科夫模型和支持向量机相结合的图像型火焰识别算法。对捕获到的图像进行运动区域检测和颜色分析,提取疑似火焰区域,利用隐马尔科夫模型计算疑似区域与火焰模型的相似度,并输入到训练好的支持向量机进行二次识别。实验结果表明,与传统单一隐马尔科夫模型相比,该方法可以有效地降低误报率,提高火焰识别准确性。     

An integrated automatic face detection and recognition system

This paper proposes an integrated system for unconstrained face recognition in complex scenes. The scale and orientation tolerant system comprises a face detector followed by a recognizer. Given a color input image of a person, the face detector encloses the face from the complex scene within a circular boundary, and locates the position of the nose. A radial grid mapping centered on the nose is then performed to extract a feature vector within the boundary. The feature vector is input to a radial basis function neural network classifier for face identification. The proposed face detector achieved an average detection rate of 95.8% while the face recognizer achieved an average recognition rate of 97.5% on a database of 21 persons with variations in scale, orientation, natural illumination and background. The two modules were combined to form an automatic face recognition system that was evaluated in the context of a security system using a video database of 21 users and 10 intruders, acquired in an unconstrained environment. A recognition rate of 93.5% with 0% false acceptance rate was achieved.     

Robust Arabic speech recognition in noisy environments using prosodic features and formant

This paper investigates the contribution of formants and prosodic features such as pitch and energy in Arabic speech recognition under real-life conditions. Our speech recognition system based on Hidden Markov Models (HMMs) is implemented using the HTK Toolkit. The front-end of the system combines features based on conventional Mel-Frequency Cepstral Coefficient (MFFC), prosodic information and formants. The experiments are performed on the ARADIGIT corpus which is a database of Arabic spoken words. The obtained results show that the resulting multivariate feature vectors, in noisy environment, lead to a significant improvement, up to 27%, in word accuracy relative the word accuracy obtained from the state-of-the-art MFCC-based system.     

American Sign Language word recognition with a sensory glove using artificial neural networks

An American Sign Language (ASL) recognition system is being developed using artificial neural networks (ANNs) to translate ASL words into English. The system uses a sensory glove called the Cyberglove™ and a Flock of Birds^® 3-D motion tracker to extract the gesture features. The data regarding finger joint angles obtained from strain gauges in the sensory glove define the hand shape, while the data from the tracker describe the trajectory of hand movements. The data from these devices are processed by a velocity network with noise reduction and feature extraction and by a word recognition network. Some global and local features are extracted for each ASL word. A neural network is used as a classifier of this feature vector. Our goal is to continuously recognize ASL signs using these devices in real time. We trained and tested the ANN model for 50 ASL words with a different number of samples for every word. The test results show that our feature vector extraction method and neural networks can be used successfully for isolated word recognition. This system is flexible and open for future extension.     

A cascade face recognition system using hybrid feature extraction

A novel cascade face recognition system using hybrid feature extraction is proposed. Three sets of face features are extracted. The merits of Two-Dimensional Complex Wavelet Transform (2D-CWT) are analyzed. For face recognition feature extraction, it has proved that 2D-CWT compares favorably with the traditionally used 2D Gabor transform in terms of the computational complexity and features? stability. The proposed recognition system congregates three Artificial Neural Network classifiers (ANNs) and a gating network trained by the three feature sets. A computationally efficient fitness function of the genetic algorithms is proposed to evolve the best weights of the ensemble classifier. Experiments demonstrated that the overall recognition rate and reliability have been significantly improved in both still face recognition and video-based face recognition.     

Multi-view face and eye detection using discriminant features

Multi-view face detection plays an important role in many applications. This paper presents a statistical learning method to extract features and construct classifiers for multi-view face detection. Specifically, a recursive nonparametric discriminant analysis (RNDA) method is presented. The RNDA relaxes Gaussian assumptions of Fisher discriminant analysis (FDA), and it can handle more general class distributions. RNDA also improves the traditional nonparametric discriminant analysis (NDA) by alleviating its computational complexity. The resulting RNDA features provide better accuracy than the commonly used Haar features in detecting objects of complex shapes. Histograms of extracted features are learned to represent class distributions and to construct probabilistic classifiers. RNDA features are subsequently learned and combined with AdaBoost to form a multi-view face detector. The method is applied to both multi-view face and eye detection, and experimental results demonstrate improved performance over existing methods.     

A real-time face detection and recognition system for a mobile robot in a complex background

This article presents a real-time face detection and recognition system for mobile robots based on videos with a complex background. In the visual system, we propose a multi-information method consisting of an Adaboost algorithm, and color information for the face detection part. The interesting targets in the video will first be detected by the Adaboost algorithm, which is robust to illumination. Then the skin color model in YCbCr space will be employed to select the parts that may not be skin areas from the information detected by the Adaboost algorithm. An embedded hidden Markov model (EHMM) is presented, using a 2-DCT feature vector as the observation vector, to recognize the faces detected. The whole process of detecting and recognizing a frame, which is 320 × 240, will take 1.4 s with the rapid recognition parameters and 4.2 s with the slow recognition parameters.